[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:00:02.788782   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:33961
I20260709 13:00:02.791117   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:00:02.793318   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:02.821218   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:02.822014   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:02.824736   431 server_base.cc:1061] running on GCE node
W20260709 13:00:02.822638   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:04.219940   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:04.220427   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:04.220759   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602004220719 us; error 0 us; skew 500 ppm
I20260709 13:00:04.221971   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:04.230278   431 webserver.cc:533] Webserver started at http://127.0.107.254:37293/ using document root <none> and password file <none>
I20260709 13:00:04.231500   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:04.231846   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:04.232488   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:04.239974   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "e557d7c6420746779cbef9af2411f9ba"
format_stamp: "Formatted at 2026-07-09 13:00:04 on dist-test-slave-ftjc"
I20260709 13:00:04.252206   431 fs_manager.cc:696] Time spent creating directory manager: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:00:04.259330   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:04.261327   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:00:04.261930   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "e557d7c6420746779cbef9af2411f9ba"
format_stamp: "Formatted at 2026-07-09 13:00:04 on dist-test-slave-ftjc"
I20260709 13:00:04.262375   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:04.342309   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:04.344770   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:00:04.345466   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:04.465019   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:33961
I20260709 13:00:04.465152   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:33961 every 8 connection(s)
I20260709 13:00:04.476914   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:04.507542   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba: Bootstrap starting.
I20260709 13:00:04.515801   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:04.518945   498 log.cc:826] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba: Log is configured to *not* fsync() on all Append() calls
I20260709 13:00:04.525828   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba: No bootstrap required, opened a new log
I20260709 13:00:04.554230   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e557d7c6420746779cbef9af2411f9ba" member_type: VOTER }
I20260709 13:00:04.554975   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:04.555217   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e557d7c6420746779cbef9af2411f9ba, State: Initialized, Role: FOLLOWER
I20260709 13:00:04.556567   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e557d7c6420746779cbef9af2411f9ba" member_type: VOTER }
I20260709 13:00:04.557175   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:04.557477   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:04.557816   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:04.563012   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e557d7c6420746779cbef9af2411f9ba" member_type: VOTER }
I20260709 13:00:04.563961   498 leader_election.cc:304] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e557d7c6420746779cbef9af2411f9ba; no voters: 
I20260709 13:00:04.566303   498 leader_election.cc:290] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:04.566730   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:04.570127   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 1 LEADER]: Becoming Leader. State: Replica: e557d7c6420746779cbef9af2411f9ba, State: Running, Role: LEADER
I20260709 13:00:04.571971   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e557d7c6420746779cbef9af2411f9ba" member_type: VOTER }
I20260709 13:00:04.573472   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:04.591032   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader e557d7c6420746779cbef9af2411f9ba. Latest consensus state: current_term: 1 leader_uuid: "e557d7c6420746779cbef9af2411f9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e557d7c6420746779cbef9af2411f9ba" member_type: VOTER } }
I20260709 13:00:04.590570   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e557d7c6420746779cbef9af2411f9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e557d7c6420746779cbef9af2411f9ba" member_type: VOTER } }
I20260709 13:00:04.591892   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:04.592010   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:04.597961   510 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:04.617631   510 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:04.623394   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:04.653303   510 catalog_manager.cc:1383] Generated new cluster ID: de3e11d0f5f9427eb86ae5a9a4efce9b
I20260709 13:00:04.653733   510 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:04.688424   510 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:04.691123   510 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:04.710631   510 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba: Generated new TSK 0
I20260709 13:00:04.712318   510 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:04.758795   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:04.771641   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:04.776063   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:04.781311   431 server_base.cc:1061] running on GCE node
W20260709 13:00:04.782229   524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:04.783737   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:04.784198   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:04.784528   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602004784505 us; error 0 us; skew 500 ppm
I20260709 13:00:04.785585   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:04.792582   431 webserver.cc:533] Webserver started at http://127.0.107.193:34649/ using document root <none> and password file <none>
I20260709 13:00:04.793744   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:04.794103   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:04.794549   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:04.796629   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "b8d201fd04514f4083ba854d86117733"
format_stamp: "Formatted at 2026-07-09 13:00:04 on dist-test-slave-ftjc"
I20260709 13:00:04.807328   431 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.007s	sys 0.003s
I20260709 13:00:04.814810   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:04.817039   431 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:00:04.817571   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "b8d201fd04514f4083ba854d86117733"
format_stamp: "Formatted at 2026-07-09 13:00:04 on dist-test-slave-ftjc"
I20260709 13:00:04.818084   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:04.844465   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:04.846612   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:04.851022   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:04.859403   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:04.859860   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:00:04.860249   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:04.860704   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.002s	sys 0.000s
I20260709 13:00:05.012961   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:36873
I20260709 13:00:05.013127   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:36873 every 8 connection(s)
I20260709 13:00:05.028185   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:05.045526   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.055164   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.063766   431 server_base.cc:1061] running on GCE node
W20260709 13:00:05.066393   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.067860   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:05.068291   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:05.068634   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602005068606 us; error 0 us; skew 500 ppm
I20260709 13:00:05.069654   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:05.075630   431 webserver.cc:533] Webserver started at http://127.0.107.194:43187/ using document root <none> and password file <none>
I20260709 13:00:05.076433   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:05.076711   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:05.077071   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:05.079381   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-ftjc"
I20260709 13:00:05.086778   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:33961
I20260709 13:00:05.087458   591 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:05.090039   591 heartbeater.cc:507] Master 127.0.107.254:33961 requested a full tablet report, sending...
I20260709 13:00:05.094432   431 fs_manager.cc:696] Time spent creating directory manager: real 0.014s	user 0.013s	sys 0.003s
I20260709 13:00:05.099434   463 ts_manager.cc:194] Registered new tserver with Master: b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873)
I20260709 13:00:05.100154   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.101233   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:00:05.101684   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-ftjc"
I20260709 13:00:05.102173   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:05.103325   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45164
I20260709 13:00:05.124205   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:05.126437   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:05.128643   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:05.134497   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:05.134865   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.135206   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:05.135464   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.272790   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43591
I20260709 13:00:05.273567   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43591 every 8 connection(s)
I20260709 13:00:05.281652   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:05.299566   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:05.301575   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.307399   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:33961
I20260709 13:00:05.307945   666 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:05.309180   666 heartbeater.cc:507] Master 127.0.107.254:33961 requested a full tablet report, sending...
W20260709 13:00:05.310878   672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:05.312021   431 server_base.cc:1061] running on GCE node
I20260709 13:00:05.312441   463 ts_manager.cc:194] Registered new tserver with Master: 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:05.313525   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:05.313822   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:05.314056   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602005314035 us; error 0 us; skew 500 ppm
I20260709 13:00:05.314640   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45180
I20260709 13:00:05.314961   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:05.318776   431 webserver.cc:533] Webserver started at http://127.0.107.195:36593/ using document root <none> and password file <none>
I20260709 13:00:05.319373   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:05.319585   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:05.319903   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:05.321350   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "90658a877e1e4b709c134360c4b215d1"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-ftjc"
I20260709 13:00:05.326902   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.005s
I20260709 13:00:05.331504   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.332815   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:00:05.333289   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "90658a877e1e4b709c134360c4b215d1"
format_stamp: "Formatted at 2026-07-09 13:00:05 on dist-test-slave-ftjc"
I20260709 13:00:05.333678   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:05.354245   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:05.356071   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:05.359639   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:05.363231   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:05.363533   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.363816   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:05.364014   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:05.520623   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:33023
I20260709 13:00:05.520713   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:33023 every 8 connection(s)
I20260709 13:00:05.548745   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:33961
I20260709 13:00:05.549316   740 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:05.550514   740 heartbeater.cc:507] Master 127.0.107.254:33961 requested a full tablet report, sending...
I20260709 13:00:05.553699   463 ts_manager.cc:194] Registered new tserver with Master: 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:05.554106   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.022132289s
I20260709 13:00:05.555748   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45182
I20260709 13:00:05.629400   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45198:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:05.635104   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:05.824626   628 tablet_service.cc:1511] Processing CreateTablet for tablet 272779beaada449796221508dafcfa86 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:05.823112   630 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1c4ef3bff14955abad790e2493d162 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:05.827641   629 tablet_service.cc:1511] Processing CreateTablet for tablet c5fe58805431460c834064070707cbf6 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:05.832217   705 tablet_service.cc:1511] Processing CreateTablet for tablet 6a59cc4c26364d97964e31a9fdd0875a (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:05.838100   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 272779beaada449796221508dafcfa86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.841343   703 tablet_service.cc:1511] Processing CreateTablet for tablet c5fe58805431460c834064070707cbf6 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:05.844331   631 tablet_service.cc:1511] Processing CreateTablet for tablet 6a59cc4c26364d97964e31a9fdd0875a (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:05.838166   702 tablet_service.cc:1511] Processing CreateTablet for tablet 272779beaada449796221508dafcfa86 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:05.837503   704 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1c4ef3bff14955abad790e2493d162 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:05.851320   627 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3d9a07d2a84147acf181a8aa0cee78 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:05.853696   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5fe58805431460c834064070707cbf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.855969   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a59cc4c26364d97964e31a9fdd0875a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.853914   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 272779beaada449796221508dafcfa86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.864832   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a59cc4c26364d97964e31a9fdd0875a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.864183   701 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3d9a07d2a84147acf181a8aa0cee78 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:05.865641   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3d9a07d2a84147acf181a8aa0cee78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.869261   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1c4ef3bff14955abad790e2493d162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.883366   701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3d9a07d2a84147acf181a8aa0cee78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.886034   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1c4ef3bff14955abad790e2493d162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.893863   553 tablet_service.cc:1511] Processing CreateTablet for tablet 272779beaada449796221508dafcfa86 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:05.896696   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 272779beaada449796221508dafcfa86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.899145   556 tablet_service.cc:1511] Processing CreateTablet for tablet 6a59cc4c26364d97964e31a9fdd0875a (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:05.882026   554 tablet_service.cc:1511] Processing CreateTablet for tablet c5fe58805431460c834064070707cbf6 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:05.890363   555 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1c4ef3bff14955abad790e2493d162 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:05.905790   552 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3d9a07d2a84147acf181a8aa0cee78 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:05.913853   551 tablet_service.cc:1511] Processing CreateTablet for tablet 1d666ad56cd44ad6806f4138aec1db17 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:05.920126   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a59cc4c26364d97964e31a9fdd0875a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.924037   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d666ad56cd44ad6806f4138aec1db17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.927773   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3d9a07d2a84147acf181a8aa0cee78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.930240   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1c4ef3bff14955abad790e2493d162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.932593   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5fe58805431460c834064070707cbf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:05.976393   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5fe58805431460c834064070707cbf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.019048   626 tablet_service.cc:1511] Processing CreateTablet for tablet 1d666ad56cd44ad6806f4138aec1db17 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:06.023669   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d666ad56cd44ad6806f4138aec1db17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.031935   700 tablet_service.cc:1511] Processing CreateTablet for tablet 1d666ad56cd44ad6806f4138aec1db17 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:06.039997   700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d666ad56cd44ad6806f4138aec1db17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.061829   550 tablet_service.cc:1511] Processing CreateTablet for tablet 970388cd5b7c4b438eced336ec5d35c4 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:06.065548   550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970388cd5b7c4b438eced336ec5d35c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.078298   628 tablet_service.cc:1511] Processing CreateTablet for tablet 970388cd5b7c4b438eced336ec5d35c4 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:06.080809   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970388cd5b7c4b438eced336ec5d35c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.086007   699 tablet_service.cc:1511] Processing CreateTablet for tablet 970388cd5b7c4b438eced336ec5d35c4 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:06.090467   699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970388cd5b7c4b438eced336ec5d35c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.098217   758 tablet_bootstrap.cc:492] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:06.105362   549 tablet_service.cc:1511] Processing CreateTablet for tablet 39b7e4b3af6e4950b2c622f5597c10bc (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:06.106930   549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b7e4b3af6e4950b2c622f5597c10bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.107962   591 heartbeater.cc:499] Master 127.0.107.254:33961 was elected leader, sending a full tablet report...
I20260709 13:00:06.160262   759 tablet_bootstrap.cc:492] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:06.161936   625 tablet_service.cc:1511] Processing CreateTablet for tablet 39b7e4b3af6e4950b2c622f5597c10bc (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:06.167124   625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b7e4b3af6e4950b2c622f5597c10bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.176100   548 tablet_service.cc:1511] Processing CreateTablet for tablet e8ead3ba2d074aa38cf587085d937df0 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:06.185078   548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ead3ba2d074aa38cf587085d937df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.187054   624 tablet_service.cc:1511] Processing CreateTablet for tablet e8ead3ba2d074aa38cf587085d937df0 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:06.209882   758 tablet_bootstrap.cc:654] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.211024   704 tablet_service.cc:1511] Processing CreateTablet for tablet 39b7e4b3af6e4950b2c622f5597c10bc (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:06.212262   705 tablet_service.cc:1511] Processing CreateTablet for tablet e8ead3ba2d074aa38cf587085d937df0 (DEFAULT_TABLE table=test-workload [id=709d568fb6c440ee8709b141f50a256b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:06.219308   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ead3ba2d074aa38cf587085d937df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.236028   624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ead3ba2d074aa38cf587085d937df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.267359   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b7e4b3af6e4950b2c622f5597c10bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:06.268103   760 tablet_bootstrap.cc:492] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:06.286612   758 tablet_bootstrap.cc:492] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:06.295746   759 tablet_bootstrap.cc:654] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.295905   758 ts_tablet_manager.cc:1403] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.251s	user 0.035s	sys 0.005s
I20260709 13:00:06.303578   758 raft_consensus.cc:359] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.308305   758 raft_consensus.cc:385] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.308866   758 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.312925   758 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.319537   666 heartbeater.cc:499] Master 127.0.107.254:33961 was elected leader, sending a full tablet report...
I20260709 13:00:06.327145   760 tablet_bootstrap.cc:654] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.351209   758 ts_tablet_manager.cc:1434] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.030s
I20260709 13:00:06.353253   758 tablet_bootstrap.cc:492] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:06.355125   759 tablet_bootstrap.cc:492] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:06.355911   759 ts_tablet_manager.cc:1403] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.206s	user 0.023s	sys 0.013s
I20260709 13:00:06.362538   759 raft_consensus.cc:359] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.363507   759 raft_consensus.cc:385] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.363916   759 raft_consensus.cc:740] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.366086   758 tablet_bootstrap.cc:654] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.365151   759 consensus_queue.cc:260] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.376943   759 ts_tablet_manager.cc:1434] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:00:06.378003   740 heartbeater.cc:499] Master 127.0.107.254:33961 was elected leader, sending a full tablet report...
I20260709 13:00:06.378083   760 tablet_bootstrap.cc:492] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:06.379032   760 ts_tablet_manager.cc:1403] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.112s	user 0.015s	sys 0.022s
I20260709 13:00:06.379292   759 tablet_bootstrap.cc:492] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:06.383430   760 raft_consensus.cc:359] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.384389   760 raft_consensus.cc:385] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.384747   760 raft_consensus.cc:740] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.385591   760 consensus_queue.cc:260] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.396629   760 ts_tablet_manager.cc:1434] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.017s	user 0.014s	sys 0.006s
I20260709 13:00:06.397852   760 tablet_bootstrap.cc:492] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:06.404654   758 tablet_bootstrap.cc:492] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:06.405972   758 ts_tablet_manager.cc:1403] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.022s
I20260709 13:00:06.408828   760 tablet_bootstrap.cc:654] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.416137   758 raft_consensus.cc:359] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.417337   758 raft_consensus.cc:385] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.417783   758 raft_consensus.cc:740] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.419086   759 tablet_bootstrap.cc:654] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.418733   758 consensus_queue.cc:260] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.423492   758 ts_tablet_manager.cc:1434] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:00:06.425251   758 tablet_bootstrap.cc:492] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:06.428047   760 tablet_bootstrap.cc:492] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:06.432933   760 ts_tablet_manager.cc:1403] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.000s
I20260709 13:00:06.437067   760 raft_consensus.cc:359] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.438056   760 raft_consensus.cc:385] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.438464   760 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.439320   760 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.451315   760 ts_tablet_manager.cc:1434] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:00:06.452832   760 tablet_bootstrap.cc:492] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:06.462234   760 tablet_bootstrap.cc:654] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.445888   758 tablet_bootstrap.cc:654] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.479115   759 tablet_bootstrap.cc:492] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:06.481441   759 ts_tablet_manager.cc:1403] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.103s	user 0.040s	sys 0.021s
I20260709 13:00:06.483125   758 tablet_bootstrap.cc:492] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:06.483925   758 ts_tablet_manager.cc:1403] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.003s
I20260709 13:00:06.484647   760 tablet_bootstrap.cc:492] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:06.485180   760 ts_tablet_manager.cc:1403] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.017s
I20260709 13:00:06.486088   759 raft_consensus.cc:359] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.486910   759 raft_consensus.cc:385] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.487257   759 raft_consensus.cc:740] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.487560   758 raft_consensus.cc:359] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.488495   758 raft_consensus.cc:385] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.488322   759 consensus_queue.cc:260] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.488911   758 raft_consensus.cc:740] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.489244   760 raft_consensus.cc:359] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.490216   760 raft_consensus.cc:385] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.489915   758 consensus_queue.cc:260] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.490564   760 raft_consensus.cc:740] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.491302   759 ts_tablet_manager.cc:1434] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:06.492412   759 tablet_bootstrap.cc:492] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:06.502386   759 tablet_bootstrap.cc:654] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.505379   758 ts_tablet_manager.cc:1434] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:00:06.503085   760 consensus_queue.cc:260] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.509132   758 tablet_bootstrap.cc:492] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:06.518440   759 tablet_bootstrap.cc:492] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:06.511665   760 ts_tablet_manager.cc:1434] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:00:06.519078   759 ts_tablet_manager.cc:1403] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.016s
I20260709 13:00:06.520036   760 tablet_bootstrap.cc:492] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:06.523850   759 raft_consensus.cc:359] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.524796   759 raft_consensus.cc:385] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:06.526011   592 tablet.cc:2406] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:06.528739   758 tablet_bootstrap.cc:654] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.525161   759 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.529922   759 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.535390   760 tablet_bootstrap.cc:654] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.536759   759 ts_tablet_manager.cc:1434] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.002s
I20260709 13:00:06.538014   759 tablet_bootstrap.cc:492] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:06.541538   765 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.542990   765 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.547797   765 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:06.558208   759 tablet_bootstrap.cc:654] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.583732   758 tablet_bootstrap.cc:492] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:06.588686   758 ts_tablet_manager.cc:1403] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.082s	user 0.008s	sys 0.020s
I20260709 13:00:06.590525   760 tablet_bootstrap.cc:492] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:06.591295   760 ts_tablet_manager.cc:1403] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.072s	user 0.021s	sys 0.014s
I20260709 13:00:06.594592   767 raft_consensus.cc:493] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.595292   767 raft_consensus.cc:515] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.595876   760 raft_consensus.cc:359] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.598780   760 raft_consensus.cc:385] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.599123   767 leader_election.cc:290] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:06.599635   760 raft_consensus.cc:740] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.602133   760 consensus_queue.cc:260] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
W20260709 13:00:06.607365   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:06.625015   760 ts_tablet_manager.cc:1434] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.006s
I20260709 13:00:06.627048   760 tablet_bootstrap.cc:492] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:06.628125   758 raft_consensus.cc:359] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.629217   758 raft_consensus.cc:385] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.629639   758 raft_consensus.cc:740] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.630636   758 consensus_queue.cc:260] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.640269   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:06.641381   641 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:06.645684   678 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:06.646744   765 raft_consensus.cc:2804] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.647229   765 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.647614   765 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.652652   760 tablet_bootstrap.cc:654] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.655694   762 raft_consensus.cc:493] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.656884   762 raft_consensus.cc:515] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.660297   758 ts_tablet_manager.cc:1434] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.071s	user 0.001s	sys 0.008s
I20260709 13:00:06.668613   762 leader_election.cc:290] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:06.668963   758 tablet_bootstrap.cc:492] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:06.673698   767 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.674746   767 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.678525   767 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:06.712157   759 tablet_bootstrap.cc:492] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:06.718142   767 raft_consensus.cc:493] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.717936   759 ts_tablet_manager.cc:1403] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.180s	user 0.036s	sys 0.036s
I20260709 13:00:06.721252   767 raft_consensus.cc:515] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.731964   788 raft_consensus.cc:493] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.732630   788 raft_consensus.cc:515] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.734745   788 leader_election.cc:290] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873)
I20260709 13:00:06.736760   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:06.739272   641 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:06.744179   767 leader_election.cc:290] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:06.746764   678 leader_election.cc:304] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:06.751657   788 raft_consensus.cc:2804] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.751756   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:06.754035   788 raft_consensus.cc:493] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.754798   788 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.761772   765 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.769471   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:06.782666   758 tablet_bootstrap.cc:654] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.776981   566 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:06.791862   765 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:06.795089   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:06.795776   641 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.788223   759 raft_consensus.cc:359] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.797863   759 raft_consensus.cc:385] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.798349   759 raft_consensus.cc:740] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
W20260709 13:00:06.805972   679 leader_election.cc:343] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.838006   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:06.807531   759 consensus_queue.cc:260] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.846213   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:06.839972   641 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:06.847786   760 tablet_bootstrap.cc:492] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:06.846352   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
W20260709 13:00:06.849224   679 leader_election.cc:343] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.846949   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:06.856613   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:06.851835   760 ts_tablet_manager.cc:1403] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.225s	user 0.008s	sys 0.020s
I20260709 13:00:06.857473   678 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: b8d201fd04514f4083ba854d86117733
I20260709 13:00:06.862584   765 raft_consensus.cc:2804] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:06.859951   604 leader_election.cc:343] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:06.870419   529 leader_election.cc:343] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.870734   788 raft_consensus.cc:515] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
W20260709 13:00:06.871999   529 leader_election.cc:343] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.868968   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:06.874780   715 raft_consensus.cc:2393] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8d201fd04514f4083ba854d86117733 in current term 1: Already voted for candidate 90658a877e1e4b709c134360c4b215d1 in this term.
I20260709 13:00:06.874981   639 raft_consensus.cc:2468] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 0.
I20260709 13:00:06.878132   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:06.882042   529 leader_election.cc:304] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:06.882781   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:06.883792   640 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:06.885607   529 leader_election.cc:343] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:06.876504   759 ts_tablet_manager.cc:1434] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.154s	user 0.012s	sys 0.003s
I20260709 13:00:06.885628   767 raft_consensus.cc:2804] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.888986   767 raft_consensus.cc:493] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.889405   767 raft_consensus.cc:3060] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.890002   759 tablet_bootstrap.cc:492] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:06.896687   765 raft_consensus.cc:697] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:06.898886   758 tablet_bootstrap.cc:492] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:06.899703   758 ts_tablet_manager.cc:1403] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.231s	user 0.025s	sys 0.031s
I20260709 13:00:06.896868   529 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8d201fd04514f4083ba854d86117733; no voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:06.891407   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:06.905711   788 leader_election.cc:290] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873)
I20260709 13:00:06.899912   765 consensus_queue.cc:237] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.886886   760 raft_consensus.cc:359] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.910938   760 raft_consensus.cc:385] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.911612   760 raft_consensus.cc:740] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.916682   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:06.925562   529 leader_election.cc:304] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8d201fd04514f4083ba854d86117733; no voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:06.926112   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:06.927258   565 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.916428   760 consensus_queue.cc:260] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.925469   758 raft_consensus.cc:359] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.932034   758 raft_consensus.cc:385] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:06.932107   792 raft_consensus.cc:2749] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.934798   767 raft_consensus.cc:515] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.932862   758 raft_consensus.cc:740] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:06.938071   767 leader_election.cc:290] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:06.941246   640 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:06.941623   758 consensus_queue.cc:260] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.943023   678 leader_election.cc:304] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:06.945996   765 raft_consensus.cc:2804] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:06.946609   765 raft_consensus.cc:697] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:06.947738   765 consensus_queue.cc:237] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:06.955642   758 ts_tablet_manager.cc:1434] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.001s
I20260709 13:00:06.957060   758 tablet_bootstrap.cc:492] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:06.961637   767 raft_consensus.cc:493] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.966903   788 raft_consensus.cc:493] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.966224   767 raft_consensus.cc:515] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.968993   788 raft_consensus.cc:515] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:06.970810   767 leader_election.cc:290] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:06.971453   759 tablet_bootstrap.cc:654] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:06.974807   760 ts_tablet_manager.cc:1434] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.115s	user 0.012s	sys 0.018s
I20260709 13:00:06.976675   788 leader_election.cc:290] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:06.975648   566 raft_consensus.cc:2393] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f9087c50833404182e2f6f4fdbf3eb4 in current term 1: Already voted for candidate b8d201fd04514f4083ba854d86117733 in this term.
I20260709 13:00:06.973652   462 catalog_manager.cc:5697] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:06.983592   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:06.984645   640 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:06.988152   678 leader_election.cc:304] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:06.989406   788 raft_consensus.cc:2804] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:06.989878   788 raft_consensus.cc:493] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.990320   788 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:06.982740   565 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:06.992304   605 leader_election.cc:304] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4; no voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733
I20260709 13:00:06.995210   792 raft_consensus.cc:2749] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:06.991508   760 tablet_bootstrap.cc:492] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:06.997607   784 raft_consensus.cc:3060] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:07.011529   529 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43591, user_credentials={real_user=slave}} blocked reactor thread for 86032us
I20260709 13:00:07.038669   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:07.070266   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.058094   784 raft_consensus.cc:2749] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:07.026233   758 tablet_bootstrap.cc:654] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.080412   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:07.071455   788 raft_consensus.cc:515] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.025343   759 tablet_bootstrap.cc:492] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:07.086782   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.089221   795 raft_consensus.cc:493] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.090018   795 raft_consensus.cc:515] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.091334   759 ts_tablet_manager.cc:1403] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.202s	user 0.024s	sys 0.007s
W20260709 13:00:07.090164   529 leader_election.cc:343] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.096877   759 raft_consensus.cc:359] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.084241   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:07.097971   759 raft_consensus.cc:385] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.098543   759 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
W20260709 13:00:07.099174   529 leader_election.cc:343] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.103554   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:07.104321   641 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.105465   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733"
W20260709 13:00:07.106963   679 leader_election.cc:343] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.103801   795 leader_election.cc:290] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:07.110240   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.058903   765 raft_consensus.cc:493] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.111073   639 raft_consensus.cc:2468] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 0.
I20260709 13:00:07.111609   765 raft_consensus.cc:515] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.113289   714 raft_consensus.cc:2468] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 0.
W20260709 13:00:07.116140   679 leader_election.cc:343] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.116518   788 leader_election.cc:290] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.087872   760 tablet_bootstrap.cc:654] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.116714   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:07.117440   529 leader_election.cc:304] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:07.119320   529 leader_election.cc:304] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:07.099743   759 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.118950   795 raft_consensus.cc:2804] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:07.125147   795 raft_consensus.cc:493] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.125299   792 raft_consensus.cc:2804] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:07.125756   795 raft_consensus.cc:3060] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.125823   792 raft_consensus.cc:493] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.126617   759 ts_tablet_manager.cc:1434] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.007s
I20260709 13:00:07.126498   792 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:07.126860   529 leader_election.cc:343] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.128016   759 tablet_bootstrap.cc:492] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:07.130496   758 tablet_bootstrap.cc:492] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:07.131173   758 ts_tablet_manager.cc:1403] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.174s	user 0.010s	sys 0.015s
I20260709 13:00:07.142618   641 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:07.144528   678 leader_election.cc:304] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: b8d201fd04514f4083ba854d86117733
I20260709 13:00:07.145926   788 raft_consensus.cc:2804] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.146518   788 raft_consensus.cc:697] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:07.147626   788 consensus_queue.cc:237] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.155149   758 raft_consensus.cc:359] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.161754   759 tablet_bootstrap.cc:654] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.171805   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.172945   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.174077   765 leader_election.cc:290] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.176837   640 raft_consensus.cc:2468] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 1.
I20260709 13:00:07.183099   792 raft_consensus.cc:515] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.186461   529 leader_election.cc:304] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:07.187909   794 raft_consensus.cc:2804] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.189127   794 raft_consensus.cc:697] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:07.190254   794 consensus_queue.cc:237] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.197469   759 tablet_bootstrap.cc:492] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:07.197168   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:07.198269   640 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.198669   760 tablet_bootstrap.cc:492] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:07.199246   760 ts_tablet_manager.cc:1403] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.208s	user 0.022s	sys 0.026s
I20260709 13:00:07.203732   760 raft_consensus.cc:359] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.208739   760 raft_consensus.cc:385] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.207523   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:07.208904   759 ts_tablet_manager.cc:1403] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.014s
I20260709 13:00:07.156225   758 raft_consensus.cc:385] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.209555   792 leader_election.cc:290] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:07.204547   795 raft_consensus.cc:515] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.231083   565 raft_consensus.cc:2393] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90658a877e1e4b709c134360c4b215d1 in current term 1: Already voted for candidate b8d201fd04514f4083ba854d86117733 in this term.
W20260709 13:00:07.231438   529 leader_election.cc:343] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.233870   640 raft_consensus.cc:2468] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 1.
I20260709 13:00:07.237655   529 leader_election.cc:304] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:07.233429   759 raft_consensus.cc:359] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.238854   758 raft_consensus.cc:740] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.239068   792 raft_consensus.cc:2804] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.238413   760 raft_consensus.cc:740] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.240175   792 raft_consensus.cc:697] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:07.240499   758 consensus_queue.cc:260] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.240974   760 consensus_queue.cc:260] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.241813   641 raft_consensus.cc:2468] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:07.243394   758 ts_tablet_manager.cc:1434] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.111s	user 0.010s	sys 0.003s
I20260709 13:00:07.244640   758 tablet_bootstrap.cc:492] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:07.246142   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:07.241168   792 consensus_queue.cc:237] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.248054   714 raft_consensus.cc:3060] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.239925   759 raft_consensus.cc:385] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.248808   759 raft_consensus.cc:740] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.249879   795 leader_election.cc:290] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:07.250963   759 consensus_queue.cc:260] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.257128   760 ts_tablet_manager.cc:1434] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.058s	user 0.016s	sys 0.006s
I20260709 13:00:07.257923   759 ts_tablet_manager.cc:1434] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.002s
I20260709 13:00:07.258155   760 tablet_bootstrap.cc:492] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:07.259727   759 tablet_bootstrap.cc:492] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:07.272190   765 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.276798   765 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.281309   758 tablet_bootstrap.cc:654] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.282405   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.284286   565 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:07.288192   679 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:07.289197   765 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.290198   765 raft_consensus.cc:2804] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:07.290828   765 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.291440   765 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.362340   788 raft_consensus.cc:493] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.363102   788 raft_consensus.cc:515] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.401211   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:07.401328   678 leader_election.cc:304] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: b8d201fd04514f4083ba854d86117733
I20260709 13:00:07.402156   641 raft_consensus.cc:3060] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.423362   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.423405   714 raft_consensus.cc:2468] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 1.
I20260709 13:00:07.425168   791 raft_consensus.cc:2764] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:00:07.425691   529 leader_election.cc:304] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:07.426451   641 raft_consensus.cc:2468] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 1.
I20260709 13:00:07.426647   792 raft_consensus.cc:2804] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.427237   792 raft_consensus.cc:697] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:07.432193   788 leader_election.cc:290] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.432791   792 consensus_queue.cc:237] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.433915   765 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.438467   758 tablet_bootstrap.cc:492] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:07.439235   758 ts_tablet_manager.cc:1403] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.195s	user 0.015s	sys 0.008s
I20260709 13:00:07.439042   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.442397   760 tablet_bootstrap.cc:654] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.444988   758 raft_consensus.cc:359] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.445979   758 raft_consensus.cc:385] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:07.445976   678 leader_election.cc:343] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.450081   758 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.453583   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.459645   640 raft_consensus.cc:2393] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90658a877e1e4b709c134360c4b215d1 in current term 1: Already voted for candidate b8d201fd04514f4083ba854d86117733 in this term.
I20260709 13:00:07.461011   765 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.459095   758 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.462906   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:07.466933   565 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:07.468916   679 consensus_peers.cc:597] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:07.468699   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:07.469626   765 consensus_queue.cc:1048] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:07.472327   641 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.472558   760 tablet_bootstrap.cc:492] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:07.473245   760 ts_tablet_manager.cc:1403] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.215s	user 0.013s	sys 0.011s
I20260709 13:00:07.474850   758 ts_tablet_manager.cc:1434] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.005s
I20260709 13:00:07.476270   758 tablet_bootstrap.cc:492] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:07.477396   760 raft_consensus.cc:359] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.478238   760 raft_consensus.cc:385] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.478770   760 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.471966   678 leader_election.cc:304] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1; no voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733
I20260709 13:00:07.480197   759 tablet_bootstrap.cc:654] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.479923   760 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.481352   641 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:07.483180   678 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:07.483494   565 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:07.484436   765 raft_consensus.cc:3060] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.488940   788 raft_consensus.cc:2804] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.489723   788 raft_consensus.cc:697] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:07.491016   760 ts_tablet_manager.cc:1434] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 13:00:07.491592   788 consensus_queue.cc:237] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.504472   759 tablet_bootstrap.cc:492] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:07.506064   759 ts_tablet_manager.cc:1403] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.247s	user 0.022s	sys 0.016s
I20260709 13:00:07.511281   792 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.511997   792 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.512385   759 raft_consensus.cc:359] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.513094   759 raft_consensus.cc:385] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.513413   759 raft_consensus.cc:740] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.517203   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.518106   640 raft_consensus.cc:2393] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8d201fd04514f4083ba854d86117733 in current term 1: Already voted for candidate 90658a877e1e4b709c134360c4b215d1 in this term.
I20260709 13:00:07.518183   759 consensus_queue.cc:260] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.521621   759 ts_tablet_manager.cc:1434] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 13:00:07.522783   759 tablet_bootstrap.cc:492] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:07.525449   791 consensus_queue.cc:1048] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.525801   792 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:07.529044   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:07.532495   529 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8d201fd04514f4083ba854d86117733; no voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:07.530130   765 raft_consensus.cc:2749] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:07.535346   792 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.546394   758 tablet_bootstrap.cc:654] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.549997   815 raft_consensus.cc:493] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.550673   815 raft_consensus.cc:515] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.552424   788 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:07.556972   760 tablet_bootstrap.cc:492] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:07.557020   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.563347   815 leader_election.cc:290] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.562928   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.564496   641 raft_consensus.cc:2393] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90658a877e1e4b709c134360c4b215d1 in current term 1: Already voted for candidate b8d201fd04514f4083ba854d86117733 in this term.
I20260709 13:00:07.568871   794 raft_consensus.cc:493] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.569445   794 raft_consensus.cc:515] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.568490   678 leader_election.cc:304] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1; no voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733
I20260709 13:00:07.574561   794 leader_election.cc:290] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:07.573596   760 tablet_bootstrap.cc:654] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.577701   815 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.582110   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.583657   759 tablet_bootstrap.cc:654] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.583673   641 raft_consensus.cc:2393] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8d201fd04514f4083ba854d86117733 in current term 1: Already voted for candidate 90658a877e1e4b709c134360c4b215d1 in this term.
I20260709 13:00:07.577405   461 catalog_manager.cc:5697] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:07.592213   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:07.593261   461 catalog_manager.cc:5697] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:07.594575   529 leader_election.cc:304] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8d201fd04514f4083ba854d86117733; no voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:07.599964   795 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.612703   792 raft_consensus.cc:2749] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:07.612848   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:07.614846   795 raft_consensus.cc:2749] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:07.628731   815 raft_consensus.cc:2749] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:07.638947   758 tablet_bootstrap.cc:492] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:07.639664   758 ts_tablet_manager.cc:1403] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.164s	user 0.027s	sys 0.031s
I20260709 13:00:07.652757   758 raft_consensus.cc:359] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.654109   758 raft_consensus.cc:385] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.654620   758 raft_consensus.cc:740] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.655642   758 consensus_queue.cc:260] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.658053   758 ts_tablet_manager.cc:1434] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:00:07.659385   758 tablet_bootstrap.cc:492] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4: Bootstrap starting.
I20260709 13:00:07.659757   759 tablet_bootstrap.cc:492] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
I20260709 13:00:07.660274   759 ts_tablet_manager.cc:1403] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.138s	user 0.018s	sys 0.014s
I20260709 13:00:07.663488   759 raft_consensus.cc:359] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.663017   461 catalog_manager.cc:5697] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } }
I20260709 13:00:07.664253   760 tablet_bootstrap.cc:492] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:07.664928   760 ts_tablet_manager.cc:1403] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.108s	user 0.020s	sys 0.009s
I20260709 13:00:07.664208   759 raft_consensus.cc:385] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.665477   759 raft_consensus.cc:740] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.666841   759 consensus_queue.cc:260] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.668869   759 ts_tablet_manager.cc:1434] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:07.669770   759 tablet_bootstrap.cc:492] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1: Bootstrap starting.
I20260709 13:00:07.675416   758 tablet_bootstrap.cc:654] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.675180   760 raft_consensus.cc:359] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.675980   760 raft_consensus.cc:385] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.676470   760 raft_consensus.cc:740] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.677328   760 consensus_queue.cc:260] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.686645   759 tablet_bootstrap.cc:654] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.689950   760 ts_tablet_manager.cc:1434] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.004s
I20260709 13:00:07.691443   760 tablet_bootstrap.cc:492] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733: Bootstrap starting.
I20260709 13:00:07.690799   460 catalog_manager.cc:5697] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:07.694782   784 raft_consensus.cc:493] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.695799   784 raft_consensus.cc:515] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.703744   460 catalog_manager.cc:5697] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:07.707711   791 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.713546   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1" is_pre_election: true
I20260709 13:00:07.713546   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.716665   792 consensus_queue.cc:1048] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.717394   784 leader_election.cc:290] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
W20260709 13:00:07.716454   604 leader_election.cc:343] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.717656   566 raft_consensus.cc:2468] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 0.
I20260709 13:00:07.719240   605 leader_election.cc:304] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:07.720891   784 raft_consensus.cc:2804] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:07.721423   784 raft_consensus.cc:493] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.721784   784 raft_consensus.cc:3060] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.722656   758 tablet_bootstrap.cc:492] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4: No bootstrap required, opened a new log
I20260709 13:00:07.722612   760 tablet_bootstrap.cc:654] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:07.723873   758 ts_tablet_manager.cc:1403] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent bootstrapping tablet: real 0.065s	user 0.006s	sys 0.010s
I20260709 13:00:07.732849   758 raft_consensus.cc:359] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.733714   758 raft_consensus.cc:385] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.734019   758 raft_consensus.cc:740] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.735244   758 consensus_queue.cc:260] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.739203   758 ts_tablet_manager.cc:1434] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:00:07.754300   791 raft_consensus.cc:493] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.754922   791 raft_consensus.cc:515] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.755592   462 catalog_manager.cc:5697] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:07.760684   792 consensus_queue.cc:1048] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.767470   815 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.786360   791 leader_election.cc:290] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.794579   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ead3ba2d074aa38cf587085d937df0" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733" is_pre_election: true
I20260709 13:00:07.802785   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ead3ba2d074aa38cf587085d937df0" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" is_pre_election: true
I20260709 13:00:07.804824   641 raft_consensus.cc:2468] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 0.
I20260709 13:00:07.811440   784 raft_consensus.cc:515] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.817160   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:07.820096   759 tablet_bootstrap.cc:492] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1: No bootstrap required, opened a new log
W20260709 13:00:07.820595   679 leader_election.cc:343] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:07.820814   759 ts_tablet_manager.cc:1403] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1: Time spent bootstrapping tablet: real 0.151s	user 0.021s	sys 0.024s
I20260709 13:00:07.820463   565 raft_consensus.cc:3060] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.823279   678 leader_election.cc:304] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: b8d201fd04514f4083ba854d86117733
I20260709 13:00:07.827049   791 raft_consensus.cc:2804] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:07.828394   791 raft_consensus.cc:493] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.828881   791 raft_consensus.cc:3060] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.827997   759 raft_consensus.cc:359] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.830683   759 raft_consensus.cc:385] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.831158   759 raft_consensus.cc:740] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.832607   759 consensus_queue.cc:260] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.836269   759 ts_tablet_manager.cc:1434] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:00:07.837122   792 consensus_queue.cc:1048] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.843407   784 leader_election.cc:290] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:07.846307   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:07.847844   760 tablet_bootstrap.cc:492] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733: No bootstrap required, opened a new log
I20260709 13:00:07.849751   760 ts_tablet_manager.cc:1403] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733: Time spent bootstrapping tablet: real 0.159s	user 0.015s	sys 0.006s
I20260709 13:00:07.849900   715 raft_consensus.cc:3060] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.853605   760 raft_consensus.cc:359] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.855808   760 raft_consensus.cc:385] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:07.856541   760 raft_consensus.cc:740] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Initialized, Role: FOLLOWER
I20260709 13:00:07.857359   760 consensus_queue.cc:260] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.861604   760 ts_tablet_manager.cc:1434] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:00:07.869241   565 raft_consensus.cc:2468] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 1.
I20260709 13:00:07.871176   605 leader_election.cc:304] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:07.871131   792 consensus_queue.cc:1048] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:07.873194   784 raft_consensus.cc:2804] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.873823   784 raft_consensus.cc:697] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:07.875168   784 consensus_queue.cc:237] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.882041   792 consensus_queue.cc:1048] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.882589   791 raft_consensus.cc:515] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.885848   715 raft_consensus.cc:2468] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 1.
I20260709 13:00:07.906803   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ead3ba2d074aa38cf587085d937df0" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:07.907840   565 raft_consensus.cc:3060] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.908681   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ead3ba2d074aa38cf587085d937df0" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:07.909621   640 raft_consensus.cc:3060] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:07.911818   791 leader_election.cc:290] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:07.948196   640 raft_consensus.cc:2468] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:07.951457   678 leader_election.cc:304] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:07.953531   791 raft_consensus.cc:2804] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:07.954839   791 raft_consensus.cc:697] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:07.955838   791 consensus_queue.cc:237] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:07.962900   795 consensus_queue.cc:1048] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.965129   565 raft_consensus.cc:2468] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 1.
I20260709 13:00:07.971395   815 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:07.986688   460 catalog_manager.cc:5697] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.046346   765 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:08.097402   460 catalog_manager.cc:5697] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:08.342994   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
mode: GRACEFUL
new_leader_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:46852
I20260709 13:00:08.343748   714 raft_consensus.cc:606] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Received request to transfer leadership to TS b8d201fd04514f4083ba854d86117733
I20260709 13:00:08.350440   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
mode: GRACEFUL
new_leader_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:46852
I20260709 13:00:08.351313   714 raft_consensus.cc:606] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Received request to transfer leadership to TS b8d201fd04514f4083ba854d86117733
I20260709 13:00:08.362620   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
mode: GRACEFUL
new_leader_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:46852
I20260709 13:00:08.363361   714 raft_consensus.cc:606] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Received request to transfer leadership to TS b8d201fd04514f4083ba854d86117733
I20260709 13:00:08.380385   784 consensus_queue.cc:1048] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:08.382722   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:59588
I20260709 13:00:08.383435   566 raft_consensus.cc:606] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:08.394380   815 raft_consensus.cc:993] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: : Instructing follower b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:08.394954   815 raft_consensus.cc:1081] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Signalling peer b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:08.398617   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:59582
I20260709 13:00:08.399381   566 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.399796   566 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.401345   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:46852
I20260709 13:00:08.402042   714 raft_consensus.cc:606] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:08.413419   784 consensus_queue.cc:1048] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:08.427297   791 consensus_queue.cc:1048] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:08.435468   836 raft_consensus.cc:993] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:08.436069   836 raft_consensus.cc:1081] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:08.437453   564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:59588
I20260709 13:00:08.438235   564 raft_consensus.cc:606] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:08.438357   566 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.443406   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:08.444136   641 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.445405   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:08.446290   714 raft_consensus.cc:3055] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.446776   714 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:08.447930   714 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.450460   714 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.457638   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35812
I20260709 13:00:08.458398   639 raft_consensus.cc:493] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.458822   639 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.467998   566 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:08.477586   639 raft_consensus.cc:515] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.481144   641 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:08.481104   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:08.482141   565 raft_consensus.cc:3055] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.482481   565 raft_consensus.cc:740] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:08.483493   565 consensus_queue.cc:260] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.485388   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:08.486052   715 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.490511   529 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:08.492702   836 raft_consensus.cc:2804] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.492442   815 consensus_queue.cc:1048] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:08.494670   814 raft_consensus.cc:993] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: : Instructing follower b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:08.495342   814 raft_consensus.cc:1081] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Signalling peer b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:08.496239   836 raft_consensus.cc:697] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:08.497450   836 consensus_queue.cc:237] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.497982   715 raft_consensus.cc:2468] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:08.492480   565 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.501135   604 leader_election.cc:304] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:08.480225   639 leader_election.cc:290] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:08.501016   564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:59582
I20260709 13:00:08.503028   564 raft_consensus.cc:493] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.503515   564 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.505537   846 raft_consensus.cc:2804] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.506384   846 raft_consensus.cc:697] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:08.507735   846 consensus_queue.cc:237] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.521240   714 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:08.526571   565 raft_consensus.cc:2468] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:08.532328   564 raft_consensus.cc:515] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:08.533777   814 raft_consensus.cc:993] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:08.539149   765 raft_consensus.cc:1081] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:08.540092   564 leader_election.cc:290] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:08.541015   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:08.541316   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35802
I20260709 13:00:08.541712   639 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.541975   641 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.542418   641 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.542621   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:08.543322   714 raft_consensus.cc:3055] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.543655   714 raft_consensus.cc:740] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:08.544625   714 consensus_queue.cc:260] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:08.545936   714 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.551527   639 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:08.540776   460 catalog_manager.cc:5697] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 1 to 2, leader changed from b8d201fd04514f4083ba854d86117733 (127.0.107.193) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.555464   641 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.556915   529 leader_election.cc:304] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:08.558185   836 raft_consensus.cc:2804] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.558967   836 raft_consensus.cc:697] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:08.559480   714 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:08.560102   836 consensus_queue.cc:237] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:08.562668   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:08.563360   564 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.569437   641 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:08.570950   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:08.572139   714 raft_consensus.cc:3055] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.572566   714 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:08.577363   564 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:08.579442   605 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:08.580482   846 raft_consensus.cc:2804] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.581043   846 raft_consensus.cc:697] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:08.581920   846 consensus_queue.cc:237] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.580600   765 raft_consensus.cc:993] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: : Instructing follower b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:08.585753   765 raft_consensus.cc:1081] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Signalling peer b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:08.586547   460 catalog_manager.cc:5697] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 1 to 2, leader changed from 90658a877e1e4b709c134360c4b215d1 (127.0.107.195) to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.588213   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:59582
I20260709 13:00:08.588912   566 raft_consensus.cc:493] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.589342   566 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.587073   714 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.596457   462 catalog_manager.cc:5697] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 1 to 2, leader changed from 90658a877e1e4b709c134360c4b215d1 (127.0.107.195) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.597851   714 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.605381   566 raft_consensus.cc:515] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.607429   714 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:08.608480   566 leader_election.cc:290] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:08.609798   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:08.611073   715 raft_consensus.cc:3055] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.611474   715 raft_consensus.cc:740] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:08.610283   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:08.612799   641 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.613024   715 consensus_queue.cc:260] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.614372   715 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.624552   715 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:08.625948   529 leader_election.cc:304] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:08.627045   767 raft_consensus.cc:2804] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:08.627593   767 raft_consensus.cc:697] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:08.628554   767 consensus_queue.cc:237] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.636575   641 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:08.635838   460 catalog_manager.cc:5697] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 1 to 2, leader changed from 90658a877e1e4b709c134360c4b215d1 (127.0.107.195) to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:08.660379   460 catalog_manager.cc:5697] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 1 to 2, leader changed from 90658a877e1e4b709c134360c4b215d1 (127.0.107.195) to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:08.962085   767 raft_consensus.cc:993] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:08.962646   836 raft_consensus.cc:1081] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:08.964653   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35812
I20260709 13:00:08.965250   641 raft_consensus.cc:493] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:08.965636   641 raft_consensus.cc:3060] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.969580   715 raft_consensus.cc:1275] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:08.974511   641 raft_consensus.cc:515] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.975062   836 consensus_queue.cc:1048] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:08.977936   641 leader_election.cc:290] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:08.980177   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:08.980943   565 raft_consensus.cc:3055] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.981330   565 raft_consensus.cc:740] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:08.981367   566 raft_consensus.cc:1275] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:08.981590   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970388cd5b7c4b438eced336ec5d35c4" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:08.982367   714 raft_consensus.cc:3060] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.982373   565 consensus_queue.cc:260] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:08.983887   565 raft_consensus.cc:3060] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.986267   784 consensus_queue.cc:1048] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:08.997599   714 raft_consensus.cc:2468] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:08.998095   639 raft_consensus.cc:1275] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.005342   604 leader_election.cc:304] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:09.006943   784 raft_consensus.cc:2804] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:09.007519   784 raft_consensus.cc:697] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:09.008443   784 consensus_queue.cc:237] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:09.011417   565 raft_consensus.cc:2468] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:09.012843   836 consensus_queue.cc:1048] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.050125   566 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.056896   463 catalog_manager.cc:5697] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 1 to 2, leader changed from b8d201fd04514f4083ba854d86117733 (127.0.107.193) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:09.062877   641 raft_consensus.cc:1275] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.064702   714 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.067349   784 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.072649   836 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.075816   836 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.079628   715 raft_consensus.cc:1275] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.082726   846 consensus_queue.cc:1048] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.109104   641 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.125435   836 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.132699   713 raft_consensus.cc:1275] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.141989   713 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.144541   826 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.150219   836 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:00:09.493805   566 raft_consensus.cc:1275] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.495635   784 consensus_queue.cc:1048] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:09.506587   714 raft_consensus.cc:1275] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:09.508466   784 consensus_queue.cc:1048] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:11.564987   431 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:00:11.566217   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:11.592017   641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
mode: GRACEFUL
new_leader_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:35822
I20260709 13:00:11.592963   641 raft_consensus.cc:606] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Received request to transfer leadership to TS 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:11.607023   566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
mode: GRACEFUL
new_leader_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:59598
I20260709 13:00:11.607620   566 raft_consensus.cc:606] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Received request to transfer leadership to TS 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:11.609140   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:11.609714   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:11.858991   852 raft_consensus.cc:993] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: : Instructing follower 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:11.859604   874 raft_consensus.cc:1081] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Signalling peer 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:11.861577   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:46834
I20260709 13:00:11.862267   713 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:11.862619   713 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.869410   713 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:11.871703   713 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 3 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:11.873165   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:11.873355   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:11.873885   566 raft_consensus.cc:3055] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.874004   641 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.874243   566 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:11.875144   566 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:11.876546   566 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.883409   641 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 3.
I20260709 13:00:11.883359   566 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 3.
I20260709 13:00:11.885059   678 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:11.886008   870 raft_consensus.cc:2804] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.887501   870 raft_consensus.cc:697] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:11.889364   870 consensus_queue.cc:237] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:11.899163   463 catalog_manager.cc:5697] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 2 to 3, leader changed from b8d201fd04514f4083ba854d86117733 (127.0.107.193) to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:11.922883   881 raft_consensus.cc:993] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: : Instructing follower 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:11.923414   866 raft_consensus.cc:1081] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Signalling peer 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:11.925573   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:46850
I20260709 13:00:11.926258   713 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:11.926702   713 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.934110   713 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:11.936584   713 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 3 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:11.937832   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:11.938064   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:11.938478   641 raft_consensus.cc:3055] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.938855   641 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:11.938730   566 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.939658   641 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:11.940740   641 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.946012   566 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 3.
I20260709 13:00:11.947007   641 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 3.
I20260709 13:00:11.947602   679 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:11.948655   870 raft_consensus.cc:2804] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.949124   870 raft_consensus.cc:697] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:11.950209   870 consensus_queue.cc:237] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:11.960330   463 catalog_manager.cc:5697] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 2 to 3, leader changed from 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194) to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:12.379678   641 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.381671   870 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:12.388386   566 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.390149   870 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:12.415323   639 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.416105   566 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:12.417438   888 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:12.418375   875 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:12.610715   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:12.622830   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:59588
I20260709 13:00:12.623471   565 raft_consensus.cc:606] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:12.630304   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:59588
I20260709 13:00:12.631026   565 raft_consensus.cc:606] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:12.637588   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5fe58805431460c834064070707cbf6"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:59588
I20260709 13:00:12.638236   565 raft_consensus.cc:606] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:12.644127   713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:46852
I20260709 13:00:12.644996   713 raft_consensus.cc:606] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:12.656575   713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
mode: GRACEFUL
new_leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:46852
I20260709 13:00:12.657226   713 raft_consensus.cc:606] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Received request to transfer leadership to TS 6f9087c50833404182e2f6f4fdbf3eb4
I20260709 13:00:12.776276   874 raft_consensus.cc:993] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:12.777011   874 raft_consensus.cc:1081] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:12.779270   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5fe58805431460c834064070707cbf6"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35812
I20260709 13:00:12.780012   639 raft_consensus.cc:493] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.780483   639 raft_consensus.cc:3060] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.788888   639 raft_consensus.cc:515] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:12.791078   639 leader_election.cc:290] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:12.792098   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:12.792203   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5fe58805431460c834064070707cbf6" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:12.792928   565 raft_consensus.cc:3055] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.792974   713 raft_consensus.cc:3060] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.793292   565 raft_consensus.cc:740] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:12.794109   565 consensus_queue.cc:260] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:12.795188   565 raft_consensus.cc:3060] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.801246   713 raft_consensus.cc:2468] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:12.803777   604 leader_election.cc:304] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:12.805263   866 raft_consensus.cc:2804] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:12.807324   866 raft_consensus.cc:697] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:12.807044   565 raft_consensus.cc:2468] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 2.
I20260709 13:00:12.809118   866 consensus_queue.cc:237] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:12.861275   460 catalog_manager.cc:5697] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 1 to 2, leader changed from b8d201fd04514f4083ba854d86117733 (127.0.107.193) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:12.931660   891 raft_consensus.cc:993] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:12.932489   891 raft_consensus.cc:1081] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:12.935556   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35812
I20260709 13:00:12.937475   639 raft_consensus.cc:493] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.938148   639 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.939807   900 raft_consensus.cc:993] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:12.940479   888 raft_consensus.cc:1081] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:12.943164   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35802
I20260709 13:00:12.944051   641 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.944579   641 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.951308   639 raft_consensus.cc:515] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:12.955864   639 leader_election.cc:290] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 3 election: Requested vote from peers 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023), b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873)
I20260709 13:00:12.956089   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:12.957021   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a59cc4c26364d97964e31a9fdd0875a" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:12.957258   713 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.957705   565 raft_consensus.cc:3055] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.957182   641 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:12.958089   565 raft_consensus.cc:740] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:12.959342   565 consensus_queue.cc:260] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:12.965816   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:12.966233   565 raft_consensus.cc:3060] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.966768   566 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.968259   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:12.969904   714 raft_consensus.cc:3055] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.965895   641 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 4 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:12.970530   714 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:12.977571   713 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 3.
I20260709 13:00:12.979141   565 raft_consensus.cc:2468] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 3.
I20260709 13:00:12.979486   604 leader_election.cc:304] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, 90658a877e1e4b709c134360c4b215d1; no voters: 
I20260709 13:00:12.981029   866 raft_consensus.cc:2804] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.978711   714 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:12.981595   866 raft_consensus.cc:697] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:12.981402   566 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 4.
I20260709 13:00:12.982659   714 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.983155   866 consensus_queue.cc:237] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } }
I20260709 13:00:12.985109   605 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:12.986248   895 raft_consensus.cc:2804] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.992523   895 raft_consensus.cc:697] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:12.994467   714 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 4.
I20260709 13:00:12.994112   895 consensus_queue.cc:237] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.003298   460 catalog_manager.cc:5697] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 2 to 3, leader changed from b8d201fd04514f4083ba854d86117733 (127.0.107.193) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.008509   874 raft_consensus.cc:993] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:13.009238   891 raft_consensus.cc:1081] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:13.011502   888 raft_consensus.cc:993] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: : Instructing follower 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:13.011848   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35812
I20260709 13:00:13.012192   900 raft_consensus.cc:1081] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Signalling peer 6f9087c50833404182e2f6f4fdbf3eb4 to start an election
I20260709 13:00:13.012835   641 raft_consensus.cc:493] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:13.013345   641 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.015166   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
 from {username='slave'} at 127.0.0.1:35802
I20260709 13:00:13.015915   639 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:13.016396   639 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.022619   460 catalog_manager.cc:5697] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 3 to 4, leader changed from 90658a877e1e4b709c134360c4b215d1 (127.0.107.195) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.023224   641 raft_consensus.cc:515] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.026639   639 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.027732   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:13.031379   566 raft_consensus.cc:3055] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.031872   566 raft_consensus.cc:740] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:13.030103   641 leader_election.cc:290] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 3 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:13.033758   566 consensus_queue.cc:260] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.034925   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:13.034360   639 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 4 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:13.035708   565 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.036252   566 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.038980   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:13.039780   713 raft_consensus.cc:3055] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.040129   713 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:13.040731   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:13.041908   714 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.041350   713 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.043557   713 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.048198   566 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 3.
I20260709 13:00:13.051172   605 leader_election.cc:304] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:13.052423   895 raft_consensus.cc:2804] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.057519   895 raft_consensus.cc:697] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:13.058079   714 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 3.
I20260709 13:00:13.058931   895 consensus_queue.cc:237] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.061388   565 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 4.
I20260709 13:00:13.062846   713 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9087c50833404182e2f6f4fdbf3eb4 in term 4.
I20260709 13:00:13.062850   605 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f9087c50833404182e2f6f4fdbf3eb4, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:13.064414   895 raft_consensus.cc:2804] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.065682   895 raft_consensus.cc:697] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Becoming Leader. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:13.066814   895 consensus_queue.cc:237] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:13.070565   460 catalog_manager.cc:5697] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 2 to 3, leader changed from b8d201fd04514f4083ba854d86117733 (127.0.107.193) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.088665   460 catalog_manager.cc:5697] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 reported cstate change: term changed from 3 to 4, leader changed from 90658a877e1e4b709c134360c4b215d1 (127.0.107.195) to 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:13.223520   713 raft_consensus.cc:1275] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.225207   895 consensus_queue.cc:1048] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:13.241568   565 raft_consensus.cc:1275] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.243947   895 consensus_queue.cc:1048] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:13.431322   713 raft_consensus.cc:1275] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.433425   895 consensus_queue.cc:1048] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.452936   565 raft_consensus.cc:1275] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.456967   565 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.457115   895 consensus_queue.cc:1048] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.458750   895 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:13.476887   713 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.479789   910 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:13.496629   713 raft_consensus.cc:1275] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.499552   910 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:13.511725   566 raft_consensus.cc:1275] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.513939   901 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:13.528551   713 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.530797   901 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:13.545173   566 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Refusing update from remote peer 6f9087c50833404182e2f6f4fdbf3eb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:13.547562   866 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:15.670128   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:15.679163   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
mode: GRACEFUL
new_leader_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:35822
I20260709 13:00:15.680008   639 raft_consensus.cc:606] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Received request to transfer leadership to TS 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:15.683704   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
mode: GRACEFUL
new_leader_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:35822
I20260709 13:00:15.684315   639 raft_consensus.cc:606] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Received request to transfer leadership to TS 90658a877e1e4b709c134360c4b215d1
I20260709 13:00:15.687345   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
mode: GRACEFUL
new_leader_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:35822
I20260709 13:00:15.687917   639 raft_consensus.cc:606] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 LEADER]: Received request to transfer leadership to TS b8d201fd04514f4083ba854d86117733
I20260709 13:00:15.691087   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
mode: GRACEFUL
new_leader_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:35822
I20260709 13:00:15.691784   639 raft_consensus.cc:606] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Received request to transfer leadership to TS b8d201fd04514f4083ba854d86117733
I20260709 13:00:15.694859   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162"
dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
mode: GRACEFUL
new_leader_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:35822
I20260709 13:00:15.695398   639 raft_consensus.cc:606] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Received request to transfer leadership to TS b8d201fd04514f4083ba854d86117733
I20260709 13:00:15.696517   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:15.696991   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:15.820215   895 raft_consensus.cc:993] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: : Instructing follower b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:15.820907   918 raft_consensus.cc:1081] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Signalling peer b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:15.823163   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:59584
I20260709 13:00:15.823733   565 raft_consensus.cc:493] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.824055   565 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.831664   565 raft_consensus.cc:515] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.834311   565 leader_election.cc:290] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 3 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:15.835417   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:15.835734   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3d9a07d2a84147acf181a8aa0cee78" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:15.836160   639 raft_consensus.cc:3055] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:15.836622   639 raft_consensus.cc:740] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:15.836639   713 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.837754   639 consensus_queue.cc:260] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.838910   639 raft_consensus.cc:3060] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.842669   713 raft_consensus.cc:2468] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 3.
I20260709 13:00:15.844231   529 leader_election.cc:304] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:15.846004   919 raft_consensus.cc:2804] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.847113   639 raft_consensus.cc:2468] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 3.
I20260709 13:00:15.847353   919 raft_consensus.cc:697] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 3 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:15.849426   919 consensus_queue.cc:237] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.853545   895 raft_consensus.cc:993] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: : Instructing follower b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:15.854241   895 raft_consensus.cc:1081] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 LEADER]: Signalling peer b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:15.856738   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:59584
I20260709 13:00:15.857328   565 raft_consensus.cc:493] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.857718   565 raft_consensus.cc:3060] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.860953   460 catalog_manager.cc:5697] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 2 to 3, leader changed from 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194) to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.866267   565 raft_consensus.cc:515] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.868258   565 leader_election.cc:290] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:15.869769   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:15.870278   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b7e4b3af6e4950b2c622f5597c10bc" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:15.870651   639 raft_consensus.cc:3055] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:15.871104   713 raft_consensus.cc:3060] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.871109   639 raft_consensus.cc:740] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:15.872318   639 consensus_queue.cc:260] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.873850   639 raft_consensus.cc:3060] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.880098   713 raft_consensus.cc:2468] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:15.881737   529 leader_election.cc:304] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:15.882439   639 raft_consensus.cc:2468] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 2.
I20260709 13:00:15.882769   919 raft_consensus.cc:2804] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:15.883700   919 raft_consensus.cc:697] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:15.884843   919 consensus_queue.cc:237] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.894096   460 catalog_manager.cc:5697] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 1 to 2, leader changed from 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194) to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.936751   928 raft_consensus.cc:993] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: : Instructing follower 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:15.936800   895 raft_consensus.cc:993] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: : Instructing follower b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:15.937422   918 raft_consensus.cc:1081] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Signalling peer 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:15.937659   895 raft_consensus.cc:1081] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Signalling peer b8d201fd04514f4083ba854d86117733 to start an election
I20260709 13:00:15.939873   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162"
dest_uuid: "b8d201fd04514f4083ba854d86117733"
 from {username='slave'} at 127.0.0.1:59584
I20260709 13:00:15.939875   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "272779beaada449796221508dafcfa86"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:46850
I20260709 13:00:15.940860   713 raft_consensus.cc:493] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.940852   565 raft_consensus.cc:493] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:15.941653   565 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.941629   713 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.950114   713 raft_consensus.cc:515] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.950114   565 raft_consensus.cc:515] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.952741   565 leader_election.cc:290] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 4 election: Requested vote from peers 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591), 90658a877e1e4b709c134360c4b215d1 (127.0.107.195:33023)
I20260709 13:00:15.953224   713 leader_election.cc:290] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 5 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:15.954085   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:15.954806   639 raft_consensus.cc:3055] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:15.954735   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:15.955245   639 raft_consensus.cc:740] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:15.955329   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c4ef3bff14955abad790e2493d162" candidate_uuid: "b8d201fd04514f4083ba854d86117733" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "90658a877e1e4b709c134360c4b215d1"
I20260709 13:00:15.955472   640 raft_consensus.cc:3055] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:15.956079   713 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.956043   640 raft_consensus.cc:740] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:15.956393   639 consensus_queue.cc:260] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.955365   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272779beaada449796221508dafcfa86" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:15.957213   640 consensus_queue.cc:260] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.957878   565 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.957947   639 raft_consensus.cc:3060] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.958698   640 raft_consensus.cc:3060] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.964141   713 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 4.
I20260709 13:00:15.965705   529 leader_election.cc:304] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:15.966646   919 raft_consensus.cc:2804] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:15.967145   919 raft_consensus.cc:697] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 4 LEADER]: Becoming Leader. State: Replica: b8d201fd04514f4083ba854d86117733, State: Running, Role: LEADER
I20260709 13:00:15.965848   565 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 5.
I20260709 13:00:15.967866   639 raft_consensus.cc:2468] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8d201fd04514f4083ba854d86117733 in term 4.
I20260709 13:00:15.968225   640 raft_consensus.cc:2468] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 5.
I20260709 13:00:15.969429   679 leader_election.cc:304] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:15.970504   900 raft_consensus.cc:2804] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:15.968019   919 consensus_queue.cc:237] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.972223   900 raft_consensus.cc:697] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 5 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:15.973874   900 consensus_queue.cc:237] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:15.981670   460 catalog_manager.cc:5697] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 reported cstate change: term changed from 3 to 4, leader changed from 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194) to b8d201fd04514f4083ba854d86117733 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "b8d201fd04514f4083ba854d86117733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.983500   463 catalog_manager.cc:5697] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 4 to 5, leader changed from 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194) to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:16.134668   895 raft_consensus.cc:993] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: : Instructing follower 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:16.135262   895 raft_consensus.cc:1081] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Signalling peer 90658a877e1e4b709c134360c4b215d1 to start an election
I20260709 13:00:16.137319   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17"
dest_uuid: "90658a877e1e4b709c134360c4b215d1"
 from {username='slave'} at 127.0.0.1:46850
I20260709 13:00:16.137897   713 raft_consensus.cc:493] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:16.138175   713 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.145684   713 raft_consensus.cc:515] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:16.148885   713 leader_election.cc:290] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 5 election: Requested vote from peers b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873), 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194:43591)
I20260709 13:00:16.149808   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b8d201fd04514f4083ba854d86117733"
I20260709 13:00:16.150132   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d666ad56cd44ad6806f4138aec1db17" candidate_uuid: "90658a877e1e4b709c134360c4b215d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6f9087c50833404182e2f6f4fdbf3eb4"
I20260709 13:00:16.150563   565 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.150962   640 raft_consensus.cc:3055] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:16.151331   640 raft_consensus.cc:740] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6f9087c50833404182e2f6f4fdbf3eb4, State: Running, Role: LEADER
I20260709 13:00:16.152117   640 consensus_queue.cc:260] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:16.153241   640 raft_consensus.cc:3060] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.156131   565 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 5.
I20260709 13:00:16.157524   679 leader_election.cc:304] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90658a877e1e4b709c134360c4b215d1, b8d201fd04514f4083ba854d86117733; no voters: 
I20260709 13:00:16.158330   900 raft_consensus.cc:2804] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:16.158787   900 raft_consensus.cc:697] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 5 LEADER]: Becoming Leader. State: Replica: 90658a877e1e4b709c134360c4b215d1, State: Running, Role: LEADER
I20260709 13:00:16.159574   900 consensus_queue.cc:237] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } }
I20260709 13:00:16.160028   640 raft_consensus.cc:2468] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90658a877e1e4b709c134360c4b215d1 in term 5.
I20260709 13:00:16.168501   463 catalog_manager.cc:5697] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 reported cstate change: term changed from 4 to 5, leader changed from 6f9087c50833404182e2f6f4fdbf3eb4 (127.0.107.194) to 90658a877e1e4b709c134360c4b215d1 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "90658a877e1e4b709c134360c4b215d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:16.231007   640 raft_consensus.cc:1275] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.232942   919 consensus_queue.cc:1048] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:16.238973   713 raft_consensus.cc:1275] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:16.240919   919 consensus_queue.cc:1048] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:16.351603   713 raft_consensus.cc:1275] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:16.353195   920 consensus_queue.cc:1048] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:16.368013   640 raft_consensus.cc:1275] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:16.369863   920 consensus_queue.cc:1048] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:16.480612   640 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:16.482616   900 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:16.494033   565 raft_consensus.cc:1275] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:16.495797   929 consensus_queue.cc:1048] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:16.513870   640 raft_consensus.cc:1275] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:16.515615   920 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:16.526608   713 raft_consensus.cc:1275] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Refusing update from remote peer b8d201fd04514f4083ba854d86117733: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:16.529441   937 consensus_queue.cc:1048] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90658a877e1e4b709c134360c4b215d1" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:16.689675   565 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:16.691581   929 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d201fd04514f4083ba854d86117733" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:16.697868   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:16.704926   640 raft_consensus.cc:1275] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Refusing update from remote peer 90658a877e1e4b709c134360c4b215d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:16.707376   943 consensus_queue.cc:1048] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9087c50833404182e2f6f4fdbf3eb4" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:16.708671   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:00:16.760828   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:16.762106   431 tablet_replica.cc:333] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.763329   431 raft_consensus.cc:2243] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.764465   431 raft_consensus.cc:2272] T 39b7e4b3af6e4950b2c622f5597c10bc P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.769023   431 tablet_replica.cc:333] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.769775   431 raft_consensus.cc:2243] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.770493   431 raft_consensus.cc:2272] T 272779beaada449796221508dafcfa86 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.773775   431 tablet_replica.cc:333] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.774392   431 raft_consensus.cc:2243] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.775148   431 raft_consensus.cc:2272] T 3e3d9a07d2a84147acf181a8aa0cee78 P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.778440   431 tablet_replica.cc:333] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.779223   431 raft_consensus.cc:2243] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.779760   431 raft_consensus.cc:2272] T c5fe58805431460c834064070707cbf6 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.782464   431 tablet_replica.cc:333] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.783281   431 raft_consensus.cc:2243] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:16.784622   431 raft_consensus.cc:2272] T 7f1c4ef3bff14955abad790e2493d162 P b8d201fd04514f4083ba854d86117733 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.787447   431 tablet_replica.cc:333] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.788156   431 raft_consensus.cc:2243] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.789024   431 raft_consensus.cc:2272] T 970388cd5b7c4b438eced336ec5d35c4 P b8d201fd04514f4083ba854d86117733 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.792286   431 tablet_replica.cc:333] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.793417   431 raft_consensus.cc:2243] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.794299   431 raft_consensus.cc:2272] T 1d666ad56cd44ad6806f4138aec1db17 P b8d201fd04514f4083ba854d86117733 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.797178   431 tablet_replica.cc:333] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.797775   431 raft_consensus.cc:2243] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.798514   431 raft_consensus.cc:2272] T e8ead3ba2d074aa38cf587085d937df0 P b8d201fd04514f4083ba854d86117733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:16.800884   431 tablet_replica.cc:333] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733: stopping tablet replica
I20260709 13:00:16.802847   431 raft_consensus.cc:2243] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:16.804137   431 raft_consensus.cc:2272] T 6a59cc4c26364d97964e31a9fdd0875a P b8d201fd04514f4083ba854d86117733 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:17.005652   679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111)
I20260709 13:00:17.008976   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:00:17.013139   679 consensus_peers.cc:597] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:17.013401   605 consensus_peers.cc:597] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:17.033627   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:00:17.064143   605 consensus_peers.cc:597] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:17.077491   679 consensus_peers.cc:597] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:17.080814   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:17.082484   431 tablet_replica.cc:333] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
W20260709 13:00:17.082180   605 consensus_peers.cc:597] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:17.083675   431 raft_consensus.cc:2243] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.084128   431 raft_consensus.cc:2272] T 39b7e4b3af6e4950b2c622f5597c10bc P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.087100   431 tablet_replica.cc:333] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.087641   431 raft_consensus.cc:2243] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.088071   431 raft_consensus.cc:2272] T 1d666ad56cd44ad6806f4138aec1db17 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.092828   431 tablet_replica.cc:333] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.093477   431 raft_consensus.cc:2243] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.093977   431 raft_consensus.cc:2272] T 3e3d9a07d2a84147acf181a8aa0cee78 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.096858   431 tablet_replica.cc:333] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.097540   431 raft_consensus.cc:2243] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.098510   431 raft_consensus.cc:2272] T 970388cd5b7c4b438eced336ec5d35c4 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.100754   431 tablet_replica.cc:333] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.101915   431 raft_consensus.cc:2243] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.102653   431 raft_consensus.cc:2272] T 7f1c4ef3bff14955abad790e2493d162 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.105633   431 tablet_replica.cc:333] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.106647   431 raft_consensus.cc:2243] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.107298   431 raft_consensus.cc:2272] T e8ead3ba2d074aa38cf587085d937df0 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.109128   431 tablet_replica.cc:333] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.109634   431 raft_consensus.cc:2243] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.110770   431 raft_consensus.cc:2272] T 6a59cc4c26364d97964e31a9fdd0875a P 6f9087c50833404182e2f6f4fdbf3eb4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.113257   431 tablet_replica.cc:333] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.113927   431 raft_consensus.cc:2243] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.114886   431 raft_consensus.cc:2272] T c5fe58805431460c834064070707cbf6 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.117117   431 tablet_replica.cc:333] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4: stopping tablet replica
I20260709 13:00:17.117700   431 raft_consensus.cc:2243] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.118193   431 raft_consensus.cc:2272] T 272779beaada449796221508dafcfa86 P 6f9087c50833404182e2f6f4fdbf3eb4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.150979   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:00:17.172968   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:00:17.210060   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:17.210834   431 tablet_replica.cc:333] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.211697   431 raft_consensus.cc:2243] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.212283   431 raft_consensus.cc:2272] T 39b7e4b3af6e4950b2c622f5597c10bc P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.216039   431 tablet_replica.cc:333] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.217134   431 raft_consensus.cc:2243] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:00:17.217711   679 consensus_peers.cc:597] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 -> Peer b8d201fd04514f4083ba854d86117733 (127.0.107.193:36873): Couldn't send request to peer b8d201fd04514f4083ba854d86117733. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:17.218048   431 raft_consensus.cc:2272] T 3e3d9a07d2a84147acf181a8aa0cee78 P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.220990   431 tablet_replica.cc:333] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.221654   431 raft_consensus.cc:2243] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.222147   431 raft_consensus.cc:2272] T c5fe58805431460c834064070707cbf6 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.223906   431 tablet_replica.cc:333] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.224543   431 raft_consensus.cc:2243] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.225422   431 raft_consensus.cc:2272] T 1d666ad56cd44ad6806f4138aec1db17 P 90658a877e1e4b709c134360c4b215d1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.227569   431 tablet_replica.cc:333] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.228128   431 raft_consensus.cc:2243] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.228657   431 raft_consensus.cc:2272] T 7f1c4ef3bff14955abad790e2493d162 P 90658a877e1e4b709c134360c4b215d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.231266   431 tablet_replica.cc:333] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.231874   431 raft_consensus.cc:2243] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.232442   431 raft_consensus.cc:2272] T 970388cd5b7c4b438eced336ec5d35c4 P 90658a877e1e4b709c134360c4b215d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.234584   431 tablet_replica.cc:333] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.235203   431 raft_consensus.cc:2243] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.236272   431 raft_consensus.cc:2272] T 272779beaada449796221508dafcfa86 P 90658a877e1e4b709c134360c4b215d1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.238806   431 tablet_replica.cc:333] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.239405   431 raft_consensus.cc:2243] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.240582   431 raft_consensus.cc:2272] T e8ead3ba2d074aa38cf587085d937df0 P 90658a877e1e4b709c134360c4b215d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.243315   431 tablet_replica.cc:333] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1: stopping tablet replica
I20260709 13:00:17.243865   431 raft_consensus.cc:2243] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:17.244437   431 raft_consensus.cc:2272] T 6a59cc4c26364d97964e31a9fdd0875a P 90658a877e1e4b709c134360c4b215d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.274291   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:00:17.297035   431 master.cc:562] Master@127.0.107.254:33961 shutting down...
I20260709 13:00:17.458032   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:17.458807   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:17.459213   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P e557d7c6420746779cbef9af2411f9ba: stopping tablet replica
I20260709 13:00:17.483676   431 master.cc:584] Master@127.0.107.254:33961 shutdown complete.
6f9087c50833404182e2f6f4fdbf3eb4  4
90658a877e1e4b709c134360c4b215d1  1
b8d201fd04514f4083ba854d86117733  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14760 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:00:17.545079   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34809
I20260709 13:00:17.547138   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:17.555594   954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.557665   955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.560890   431 server_base.cc:1061] running on GCE node
W20260709 13:00:17.561280   957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.562698   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.562920   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.563050   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017563030 us; error 0 us; skew 500 ppm
I20260709 13:00:17.563582   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.567282   431 webserver.cc:533] Webserver started at http://127.0.107.254:34063/ using document root <none> and password file <none>
I20260709 13:00:17.567920   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.568203   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.568593   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.570132   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "908e08ffa4d7423286657d9b08c4a2fb"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-ftjc"
I20260709 13:00:17.576596   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:00:17.581314   962 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.582281   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:00:17.582795   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "908e08ffa4d7423286657d9b08c4a2fb"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-ftjc"
I20260709 13:00:17.583261   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.624398   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.626027   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.681697   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34809
I20260709 13:00:17.681751  1013 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34809 every 8 connection(s)
I20260709 13:00:17.687774  1014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.700569  1014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb: Bootstrap starting.
I20260709 13:00:17.707206  1014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:17.713064  1014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb: No bootstrap required, opened a new log
I20260709 13:00:17.715596  1014 raft_consensus.cc:359] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908e08ffa4d7423286657d9b08c4a2fb" member_type: VOTER }
I20260709 13:00:17.716048  1014 raft_consensus.cc:385] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:17.716390  1014 raft_consensus.cc:740] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908e08ffa4d7423286657d9b08c4a2fb, State: Initialized, Role: FOLLOWER
I20260709 13:00:17.717109  1014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908e08ffa4d7423286657d9b08c4a2fb" member_type: VOTER }
I20260709 13:00:17.717608  1014 raft_consensus.cc:399] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:17.717857  1014 raft_consensus.cc:493] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:17.718148  1014 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:17.723809  1014 raft_consensus.cc:515] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908e08ffa4d7423286657d9b08c4a2fb" member_type: VOTER }
I20260709 13:00:17.724654  1014 leader_election.cc:304] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 908e08ffa4d7423286657d9b08c4a2fb; no voters: 
I20260709 13:00:17.726152  1014 leader_election.cc:290] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:17.726836  1017 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:17.728614  1017 raft_consensus.cc:697] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 1 LEADER]: Becoming Leader. State: Replica: 908e08ffa4d7423286657d9b08c4a2fb, State: Running, Role: LEADER
I20260709 13:00:17.729346  1017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908e08ffa4d7423286657d9b08c4a2fb" member_type: VOTER }
I20260709 13:00:17.729991  1014 sys_catalog.cc:565] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:17.735448  1019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 908e08ffa4d7423286657d9b08c4a2fb. Latest consensus state: current_term: 1 leader_uuid: "908e08ffa4d7423286657d9b08c4a2fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908e08ffa4d7423286657d9b08c4a2fb" member_type: VOTER } }
I20260709 13:00:17.736527  1019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:17.737550  1018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "908e08ffa4d7423286657d9b08c4a2fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908e08ffa4d7423286657d9b08c4a2fb" member_type: VOTER } }
I20260709 13:00:17.738294  1018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:17.739054  1024 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:17.745486  1024 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:17.750773   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:17.758788  1024 catalog_manager.cc:1383] Generated new cluster ID: e245edb515284da19e4627a7b66669ad
I20260709 13:00:17.759141  1024 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:17.817735  1024 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:17.819481  1024 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:17.840746  1024 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb: Generated new TSK 0
I20260709 13:00:17.841506  1024 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:17.887847   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:17.895522  1035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.897192  1036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:17.902001  1038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.902729   431 server_base.cc:1061] running on GCE node
I20260709 13:00:17.903678   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.903874   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.904129   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017904108 us; error 0 us; skew 500 ppm
I20260709 13:00:17.904824   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.907778   431 webserver.cc:533] Webserver started at http://127.0.107.193:45819/ using document root <none> and password file <none>
I20260709 13:00:17.908499   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.908740   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.909075   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.910442   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "ca1e65a96e454f558f348160072c35d0"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-ftjc"
I20260709 13:00:17.916826   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:00:17.921198  1043 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.922423   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.003s
I20260709 13:00:17.922851   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "ca1e65a96e454f558f348160072c35d0"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-ftjc"
I20260709 13:00:17.923292   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.948935   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.951050   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.953066   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:17.956393   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:17.957015   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.001s	sys 0.000s
I20260709 13:00:17.957504   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:17.957793   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.013185   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46171
I20260709 13:00:18.013537  1105 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46171 every 8 connection(s)
I20260709 13:00:18.019960   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:18.028929  1111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:18.028843  1110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:18.035429  1113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:18.035871   431 server_base.cc:1061] running on GCE node
I20260709 13:00:18.037180   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:18.037395   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:18.037642   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602018037621 us; error 0 us; skew 500 ppm
I20260709 13:00:18.038316   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:18.041313   431 webserver.cc:533] Webserver started at http://127.0.107.194:46461/ using document root <none> and password file <none>
I20260709 13:00:18.041904   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:18.042126   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:18.042421   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:18.044015   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
format_stamp: "Formatted at 2026-07-09 13:00:18 on dist-test-slave-ftjc"
I20260709 13:00:18.051014   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:18.051489  1106 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34809
I20260709 13:00:18.051908  1106 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:18.052850  1106 heartbeater.cc:507] Master 127.0.107.254:34809 requested a full tablet report, sending...
I20260709 13:00:18.055466  1118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.055464   979 ts_manager.cc:194] Registered new tserver with Master: ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171)
I20260709 13:00:18.057091   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:18.057523   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
format_stamp: "Formatted at 2026-07-09 13:00:18 on dist-test-slave-ftjc"
I20260709 13:00:18.057875   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:18.058135   979 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54070
I20260709 13:00:18.079015   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:18.080757   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:18.082631   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:18.085659   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:18.085985   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.086359   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:18.086550   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.144208   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:44091
I20260709 13:00:18.144367  1180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:44091 every 8 connection(s)
I20260709 13:00:18.150599   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:18.160389  1184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:18.161831  1185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:18.164541  1187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:18.164844   431 server_base.cc:1061] running on GCE node
I20260709 13:00:18.166101   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:18.166391   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:18.166646   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602018166620 us; error 0 us; skew 500 ppm
I20260709 13:00:18.167315   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:18.167147  1181 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34809
I20260709 13:00:18.168198  1181 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:18.169325  1181 heartbeater.cc:507] Master 127.0.107.254:34809 requested a full tablet report, sending...
I20260709 13:00:18.171062   431 webserver.cc:533] Webserver started at http://127.0.107.195:46347/ using document root <none> and password file <none>
I20260709 13:00:18.171792   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:18.172079   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:18.172451   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:18.173230   979 ts_manager.cc:194] Registered new tserver with Master: 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:18.174088   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "58c9b0ef0380446991b7be408fe6b011"
format_stamp: "Formatted at 2026-07-09 13:00:18 on dist-test-slave-ftjc"
I20260709 13:00:18.175531   979 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54082
I20260709 13:00:18.182089   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:00:18.187181  1192 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.188195   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:00:18.188824   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "58c9b0ef0380446991b7be408fe6b011"
format_stamp: "Formatted at 2026-07-09 13:00:18 on dist-test-slave-ftjc"
I20260709 13:00:18.189136   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:18.202948   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:18.204329   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:18.206020   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:18.208789   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:18.209017   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.209316   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:18.209534   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:18.265785   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:33449
I20260709 13:00:18.265899  1254 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:33449 every 8 connection(s)
I20260709 13:00:18.284747  1255 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34809
I20260709 13:00:18.285254  1255 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:18.286202  1255 heartbeater.cc:507] Master 127.0.107.254:34809 requested a full tablet report, sending...
I20260709 13:00:18.288776   979 ts_manager.cc:194] Registered new tserver with Master: 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:18.289469   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019269901s
I20260709 13:00:18.290513   979 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54098
I20260709 13:00:18.321646   979 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54114:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:18.324939   979 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:18.424543  1146 tablet_service.cc:1511] Processing CreateTablet for tablet 36bedd9ba1194844af99715a6dfc1eb0 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:18.426337  1146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36bedd9ba1194844af99715a6dfc1eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.426608  1142 tablet_service.cc:1511] Processing CreateTablet for tablet aa5906670ff14a83abcb490f63361b01 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:18.428277  1142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5906670ff14a83abcb490f63361b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.429761  1145 tablet_service.cc:1511] Processing CreateTablet for tablet 75713d192a6b42399478f4f00e2a9c79 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:18.434028  1143 tablet_service.cc:1511] Processing CreateTablet for tablet 447eea4219634eb6a3aa17c9417f2462 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:18.434264  1141 tablet_service.cc:1511] Processing CreateTablet for tablet 4e5efa3edd19465a83308d4b5400fd70 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:18.434437  1220 tablet_service.cc:1511] Processing CreateTablet for tablet 36bedd9ba1194844af99715a6dfc1eb0 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:18.431763  1145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75713d192a6b42399478f4f00e2a9c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.446378  1220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36bedd9ba1194844af99715a6dfc1eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.447008  1143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447eea4219634eb6a3aa17c9417f2462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.450604  1214 tablet_service.cc:1511] Processing CreateTablet for tablet 909afa693e8b46e698b875b6a0a79c9a (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:18.452463  1214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909afa693e8b46e698b875b6a0a79c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.456048  1140 tablet_service.cc:1511] Processing CreateTablet for tablet 909afa693e8b46e698b875b6a0a79c9a (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:18.457943  1213 tablet_service.cc:1511] Processing CreateTablet for tablet 37bfb53e2f744bfda4e7f76f95c5aef3 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:18.458966  1139 tablet_service.cc:1511] Processing CreateTablet for tablet 37bfb53e2f744bfda4e7f76f95c5aef3 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:18.459673  1213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfb53e2f744bfda4e7f76f95c5aef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.436842  1218 tablet_service.cc:1511] Processing CreateTablet for tablet 98314e81f7c14579b10079f98293e8de (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:18.462050  1218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98314e81f7c14579b10079f98293e8de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.463593  1071 tablet_service.cc:1511] Processing CreateTablet for tablet 36bedd9ba1194844af99715a6dfc1eb0 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:18.437232  1219 tablet_service.cc:1511] Processing CreateTablet for tablet 75713d192a6b42399478f4f00e2a9c79 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:18.465312  1219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75713d192a6b42399478f4f00e2a9c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.465420  1071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36bedd9ba1194844af99715a6dfc1eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.439110  1217 tablet_service.cc:1511] Processing CreateTablet for tablet 447eea4219634eb6a3aa17c9417f2462 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:18.441748  1216 tablet_service.cc:1511] Processing CreateTablet for tablet aa5906670ff14a83abcb490f63361b01 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:18.443603  1215 tablet_service.cc:1511] Processing CreateTablet for tablet 4e5efa3edd19465a83308d4b5400fd70 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:18.443715  1144 tablet_service.cc:1511] Processing CreateTablet for tablet 98314e81f7c14579b10079f98293e8de (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:18.469457  1217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447eea4219634eb6a3aa17c9417f2462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.470233  1215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e5efa3edd19465a83308d4b5400fd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.473385  1216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5906670ff14a83abcb490f63361b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.473389  1140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909afa693e8b46e698b875b6a0a79c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.482597  1138 tablet_service.cc:1511] Processing CreateTablet for tablet c97aa01b06414cf4bb9dabdbd9cec5c7 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:18.484942  1212 tablet_service.cc:1511] Processing CreateTablet for tablet c97aa01b06414cf4bb9dabdbd9cec5c7 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:18.488060  1212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97aa01b06414cf4bb9dabdbd9cec5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.492977  1137 tablet_service.cc:1511] Processing CreateTablet for tablet 63de5ca351e64e4587621c643e5b0121 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:18.493120  1211 tablet_service.cc:1511] Processing CreateTablet for tablet 63de5ca351e64e4587621c643e5b0121 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:18.499410  1211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63de5ca351e64e4587621c643e5b0121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.484329  1138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97aa01b06414cf4bb9dabdbd9cec5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.508752  1141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e5efa3edd19465a83308d4b5400fd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.510644  1070 tablet_service.cc:1511] Processing CreateTablet for tablet 75713d192a6b42399478f4f00e2a9c79 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:18.511749  1067 tablet_service.cc:1511] Processing CreateTablet for tablet aa5906670ff14a83abcb490f63361b01 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:18.512581  1070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75713d192a6b42399478f4f00e2a9c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.512820  1139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfb53e2f744bfda4e7f76f95c5aef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.513654  1067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5906670ff14a83abcb490f63361b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.515467  1069 tablet_service.cc:1511] Processing CreateTablet for tablet 98314e81f7c14579b10079f98293e8de (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:18.516214  1144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98314e81f7c14579b10079f98293e8de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.517338  1069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98314e81f7c14579b10079f98293e8de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.519207  1137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63de5ca351e64e4587621c643e5b0121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.522097  1068 tablet_service.cc:1511] Processing CreateTablet for tablet 447eea4219634eb6a3aa17c9417f2462 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:18.523886  1068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447eea4219634eb6a3aa17c9417f2462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.536840  1066 tablet_service.cc:1511] Processing CreateTablet for tablet 4e5efa3edd19465a83308d4b5400fd70 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:18.539021  1066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e5efa3edd19465a83308d4b5400fd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.552770  1065 tablet_service.cc:1511] Processing CreateTablet for tablet 909afa693e8b46e698b875b6a0a79c9a (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:18.557229  1065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909afa693e8b46e698b875b6a0a79c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.569067  1064 tablet_service.cc:1511] Processing CreateTablet for tablet 37bfb53e2f744bfda4e7f76f95c5aef3 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:18.572052  1063 tablet_service.cc:1511] Processing CreateTablet for tablet c97aa01b06414cf4bb9dabdbd9cec5c7 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:18.605614  1064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37bfb53e2f744bfda4e7f76f95c5aef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.606377  1062 tablet_service.cc:1511] Processing CreateTablet for tablet 63de5ca351e64e4587621c643e5b0121 (DEFAULT_TABLE table=test-workload [id=e50dfbd8d3174018a160d69c934352ec]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:18.609903  1063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97aa01b06414cf4bb9dabdbd9cec5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.618141  1062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63de5ca351e64e4587621c643e5b0121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:18.623503  1269 tablet_bootstrap.cc:492] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:18.640545  1270 tablet_bootstrap.cc:492] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:18.659474  1270 tablet_bootstrap.cc:654] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.665578  1271 tablet_bootstrap.cc:492] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:18.688947  1269 tablet_bootstrap.cc:654] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.703348  1271 tablet_bootstrap.cc:654] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.706678  1269 tablet_bootstrap.cc:492] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:18.707630  1269 ts_tablet_manager.cc:1403] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.010s
I20260709 13:00:18.712481  1269 raft_consensus.cc:359] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.713835  1269 raft_consensus.cc:385] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.714305  1269 raft_consensus.cc:740] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.717015  1269 consensus_queue.cc:260] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.720073  1270 tablet_bootstrap.cc:492] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:18.720628  1270 ts_tablet_manager.cc:1403] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.016s
I20260709 13:00:18.721441  1271 tablet_bootstrap.cc:492] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:18.722158  1271 ts_tablet_manager.cc:1403] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.057s	user 0.006s	sys 0.015s
I20260709 13:00:18.723393  1270 raft_consensus.cc:359] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.724017  1270 raft_consensus.cc:385] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.724387  1270 raft_consensus.cc:740] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.725399  1255 heartbeater.cc:499] Master 127.0.107.254:34809 was elected leader, sending a full tablet report...
I20260709 13:00:18.725821  1269 ts_tablet_manager.cc:1434] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.012s
I20260709 13:00:18.725419  1270 consensus_queue.cc:260] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.726428  1271 raft_consensus.cc:359] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.726972  1269 tablet_bootstrap.cc:492] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:18.727259  1271 raft_consensus.cc:385] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.727609  1271 raft_consensus.cc:740] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.728694  1271 consensus_queue.cc:260] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.736819  1270 ts_tablet_manager.cc:1434] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.011s
I20260709 13:00:18.736703  1181 heartbeater.cc:499] Master 127.0.107.254:34809 was elected leader, sending a full tablet report...
I20260709 13:00:18.736943  1269 tablet_bootstrap.cc:654] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.743700  1270 tablet_bootstrap.cc:492] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
W20260709 13:00:18.744284  1027 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:18.750053  1106 heartbeater.cc:499] Master 127.0.107.254:34809 was elected leader, sending a full tablet report...
I20260709 13:00:18.751166  1271 ts_tablet_manager.cc:1434] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.029s	user 0.011s	sys 0.017s
I20260709 13:00:18.753021  1271 tablet_bootstrap.cc:492] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:18.761330  1270 tablet_bootstrap.cc:654] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.763922  1271 tablet_bootstrap.cc:654] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.776255  1270 tablet_bootstrap.cc:492] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:18.776834  1270 ts_tablet_manager.cc:1403] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.003s
I20260709 13:00:18.780887  1270 raft_consensus.cc:359] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.781858  1270 raft_consensus.cc:385] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.782194  1270 raft_consensus.cc:740] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.783434  1270 consensus_queue.cc:260] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.786427  1270 ts_tablet_manager.cc:1434] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:00:18.789201  1270 tablet_bootstrap.cc:492] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:18.796456  1269 tablet_bootstrap.cc:492] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:18.797113  1269 ts_tablet_manager.cc:1403] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.041s
I20260709 13:00:18.800771  1270 tablet_bootstrap.cc:654] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.808702  1269 raft_consensus.cc:359] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.809720  1269 raft_consensus.cc:385] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.810040  1269 raft_consensus.cc:740] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.811100  1269 consensus_queue.cc:260] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.814198  1269 ts_tablet_manager.cc:1434] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:00:18.815644  1269 tablet_bootstrap.cc:492] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:18.824368  1271 tablet_bootstrap.cc:492] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:18.824506  1269 tablet_bootstrap.cc:654] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.825049  1271 ts_tablet_manager.cc:1403] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.072s	user 0.032s	sys 0.029s
I20260709 13:00:18.829250  1271 raft_consensus.cc:359] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.830168  1271 raft_consensus.cc:385] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.830474  1277 raft_consensus.cc:493] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.830544  1271 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.831418  1277 raft_consensus.cc:515] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.831995  1271 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.835813  1270 tablet_bootstrap.cc:492] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:18.837244  1270 ts_tablet_manager.cc:1403] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.048s	user 0.023s	sys 0.012s
I20260709 13:00:18.837713  1269 tablet_bootstrap.cc:492] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:18.838315  1269 ts_tablet_manager.cc:1403] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 13:00:18.842041  1269 raft_consensus.cc:359] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.842494  1276 raft_consensus.cc:493] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.843099  1276 raft_consensus.cc:515] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.844391  1270 raft_consensus.cc:359] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.845534  1270 raft_consensus.cc:385] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.847008  1276 leader_election.cc:290] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:18.843598  1277 leader_election.cc:290] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:18.848675  1269 raft_consensus.cc:385] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.849315  1269 raft_consensus.cc:740] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.850363  1269 consensus_queue.cc:260] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.845981  1270 raft_consensus.cc:740] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.854583  1270 consensus_queue.cc:260] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.858484  1269 ts_tablet_manager.cc:1434] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 13:00:18.859679  1269 tablet_bootstrap.cc:492] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:18.867419  1271 ts_tablet_manager.cc:1434] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.042s	user 0.022s	sys 0.020s
I20260709 13:00:18.867425  1269 tablet_bootstrap.cc:654] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.868691  1271 tablet_bootstrap.cc:492] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:18.868139  1275 raft_consensus.cc:493] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.869467  1275 raft_consensus.cc:515] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.875669  1270 ts_tablet_manager.cc:1434] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.003s
I20260709 13:00:18.877724  1271 tablet_bootstrap.cc:654] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.891641  1275 leader_election.cc:290] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:18.932986  1283 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.934415  1283 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:18.893365  1270 tablet_bootstrap.cc:492] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:18.982789  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:18.983398  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:18.983768  1230 raft_consensus.cc:2468] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:18.985903  1122 leader_election.cc:304] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:18.987231  1276 raft_consensus.cc:2804] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.987768  1276 raft_consensus.cc:493] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.988263  1276 raft_consensus.cc:3060] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.989413  1283 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:18.993897  1156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
W20260709 13:00:19.003470  1122 leader_election.cc:343] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.004614  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.007928  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98314e81f7c14579b10079f98293e8de" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.003474  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98314e81f7c14579b10079f98293e8de" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.009392  1155 raft_consensus.cc:2468] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 0.
I20260709 13:00:19.008170  1276 raft_consensus.cc:515] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.012784  1156 raft_consensus.cc:2393] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58c9b0ef0380446991b7be408fe6b011 in current term 1: Already voted for candidate 4c2058117a3f4f608d6bbd90694cd4ea in this term.
I20260709 13:00:19.012617  1154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
W20260709 13:00:19.010517  1047 leader_election.cc:343] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.017678  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.000324  1270 tablet_bootstrap.cc:654] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:19.019451  1047 leader_election.cc:343] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.017972  1276 leader_election.cc:290] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
W20260709 13:00:19.020320  1047 leader_election.cc:343] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.021631  1047 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:19.022678  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.023514  1283 raft_consensus.cc:2749] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.023389  1230 raft_consensus.cc:3060] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.024598  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
W20260709 13:00:19.022673  1122 leader_election.cc:343] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.025219  1047 leader_election.cc:304] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:19.027942  1283 raft_consensus.cc:2804] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.028687  1283 raft_consensus.cc:493] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:19.028074  1196 leader_election.cc:343] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.029431  1283 raft_consensus.cc:3060] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.029438  1196 leader_election.cc:304] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.033721  1269 tablet_bootstrap.cc:492] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.034096  1271 tablet_bootstrap.cc:492] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.034381  1269 ts_tablet_manager.cc:1403] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.175s	user 0.038s	sys 0.090s
I20260709 13:00:19.034574  1271 ts_tablet_manager.cc:1403] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.166s	user 0.070s	sys 0.072s
I20260709 13:00:19.037357  1230 raft_consensus.cc:2468] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.039721  1271 raft_consensus.cc:359] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.038723  1269 raft_consensus.cc:359] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.040805  1275 raft_consensus.cc:2749] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.040675  1270 tablet_bootstrap.cc:492] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.041246  1271 raft_consensus.cc:385] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.041352  1269 raft_consensus.cc:385] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.042022  1122 leader_election.cc:304] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.044610  1269 raft_consensus.cc:740] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.044436  1271 raft_consensus.cc:740] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.042047  1283 raft_consensus.cc:515] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.045948  1276 raft_consensus.cc:2804] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.045670  1271 consensus_queue.cc:260] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.045670  1269 consensus_queue.cc:260] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.048568  1271 ts_tablet_manager.cc:1434] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:00:19.048741  1269 ts_tablet_manager.cc:1434] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:00:19.049424  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98314e81f7c14579b10079f98293e8de" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:19.049808  1271 tablet_bootstrap.cc:492] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.049808  1269 tablet_bootstrap.cc:492] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:19.050160  1155 raft_consensus.cc:3060] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.058019  1269 tablet_bootstrap.cc:654] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.058019  1271 tablet_bootstrap.cc:654] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.071893  1269 tablet_bootstrap.cc:492] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.072137  1155 raft_consensus.cc:2468] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 1.
I20260709 13:00:19.072963  1271 tablet_bootstrap.cc:492] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.073508  1271 ts_tablet_manager.cc:1403] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:00:19.073498  1047 leader_election.cc:304] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.075011  1277 raft_consensus.cc:2804] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.042466  1270 ts_tablet_manager.cc:1403] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.149s	user 0.017s	sys 0.002s
I20260709 13:00:19.072566  1269 ts_tablet_manager.cc:1403] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.006s
I20260709 13:00:19.080935  1275 raft_consensus.cc:493] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.082515  1275 raft_consensus.cc:515] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.083595  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98314e81f7c14579b10079f98293e8de" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.084108  1269 raft_consensus.cc:359] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.085011  1269 raft_consensus.cc:385] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.085379  1269 raft_consensus.cc:740] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.083452  1270 raft_consensus.cc:359] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
W20260709 13:00:19.085961  1047 leader_election.cc:343] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.088586  1270 raft_consensus.cc:385] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.089236  1270 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.090505  1270 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.091761  1275 leader_election.cc:290] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:19.092597  1277 raft_consensus.cc:697] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:19.080302  1271 raft_consensus.cc:359] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.093601  1271 raft_consensus.cc:385] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.093986  1271 raft_consensus.cc:740] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.095377  1270 ts_tablet_manager.cc:1434] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.006s
I20260709 13:00:19.087033  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63de5ca351e64e4587621c643e5b0121" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.087558  1283 leader_election.cc:290] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.087435  1276 raft_consensus.cc:697] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:19.095896  1271 consensus_queue.cc:260] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.096958  1270 tablet_bootstrap.cc:492] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:19.097189  1081 raft_consensus.cc:2468] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 0.
I20260709 13:00:19.093974  1277 consensus_queue.cc:237] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.086333  1269 consensus_queue.cc:260] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.096995  1276 consensus_queue.cc:237] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.102159  1269 ts_tablet_manager.cc:1434] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:00:19.099799  1196 leader_election.cc:304] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.105767  1275 raft_consensus.cc:2804] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.106484  1275 raft_consensus.cc:493] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.106896  1275 raft_consensus.cc:3060] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.107576  1271 ts_tablet_manager.cc:1434] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.004s
I20260709 13:00:19.108757  1271 tablet_bootstrap.cc:492] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.110131  1270 tablet_bootstrap.cc:654] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.115597  1275 raft_consensus.cc:515] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.123263  1269 tablet_bootstrap.cc:492] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:19.123654  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63de5ca351e64e4587621c643e5b0121" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.124397  1081 raft_consensus.cc:3060] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.125380  1270 tablet_bootstrap.cc:492] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.126540  1270 ts_tablet_manager.cc:1403] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.004s
I20260709 13:00:19.131914  1270 raft_consensus.cc:359] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.094902  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63de5ca351e64e4587621c643e5b0121" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.119024  1271 tablet_bootstrap.cc:654] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.134828  1296 raft_consensus.cc:493] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.135360  1296 raft_consensus.cc:515] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
W20260709 13:00:19.136232  1196 leader_election.cc:343] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.138182  1296 leader_election.cc:290] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.138475  1269 tablet_bootstrap.cc:654] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.141978  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.142825  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63de5ca351e64e4587621c643e5b0121" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
W20260709 13:00:19.144302  1196 leader_election.cc:343] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:19.144604  1122 leader_election.cc:343] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.134199  1270 raft_consensus.cc:385] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.145879  1270 raft_consensus.cc:740] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.146653  1275 leader_election.cc:290] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:19.147322  1275 raft_consensus.cc:493] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.147882  1275 raft_consensus.cc:515] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.147419  1270 consensus_queue.cc:260] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.151647  1270 ts_tablet_manager.cc:1434] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.005s
I20260709 13:00:19.155695  1270 tablet_bootstrap.cc:492] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:19.156587  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.157823  1271 tablet_bootstrap.cc:492] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.158432  1271 ts_tablet_manager.cc:1403] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.011s
I20260709 13:00:19.156915   978 catalog_manager.cc:5697] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:19.159533  1196 leader_election.cc:343] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.153493  1081 raft_consensus.cc:2468] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 1.
I20260709 13:00:19.160274   977 catalog_manager.cc:5697] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.161448  1196 leader_election.cc:304] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea
I20260709 13:00:19.162308  1271 raft_consensus.cc:359] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.163098  1304 raft_consensus.cc:2804] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.164736  1270 tablet_bootstrap.cc:654] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.165975  1275 leader_election.cc:290] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:19.163169  1271 raft_consensus.cc:385] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.166872  1271 raft_consensus.cc:740] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.167073  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.168157  1271 consensus_queue.cc:260] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
W20260709 13:00:19.170920  1196 leader_election.cc:343] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.171692  1196 leader_election.cc:304] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.172777  1271 ts_tablet_manager.cc:1434] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.003s
I20260709 13:00:19.173521  1275 raft_consensus.cc:2749] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.174460  1271 tablet_bootstrap.cc:492] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.179762  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.180917  1230 raft_consensus.cc:2468] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.182696  1122 leader_election.cc:304] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.184244  1276 raft_consensus.cc:2804] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.184504  1271 tablet_bootstrap.cc:654] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.184778  1276 raft_consensus.cc:493] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.185163  1276 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.186669  1304 raft_consensus.cc:697] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 1 LEADER]: Becoming Leader. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:19.187913  1304 consensus_queue.cc:237] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.196628  1269 tablet_bootstrap.cc:492] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.197270  1269 ts_tablet_manager.cc:1403] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.007s
I20260709 13:00:19.199383  1270 tablet_bootstrap.cc:492] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.200021  1270 ts_tablet_manager.cc:1403] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.014s
I20260709 13:00:19.201097  1269 raft_consensus.cc:359] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.201973  1269 raft_consensus.cc:385] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.195760  1306 raft_consensus.cc:493] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.202410  1269 raft_consensus.cc:740] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.202755  1306 raft_consensus.cc:515] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.202998  1276 raft_consensus.cc:515] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.206959  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.207265  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.207394  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.208823  1306 leader_election.cc:290] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
W20260709 13:00:19.209326  1196 leader_election.cc:343] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.212271  1270 raft_consensus.cc:359] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.213302  1270 raft_consensus.cc:385] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.213658  1270 raft_consensus.cc:740] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
W20260709 13:00:19.221450  1122 leader_election.cc:343] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.223634  1269 consensus_queue.cc:260] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
W20260709 13:00:19.224656  1196 leader_election.cc:343] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.225351  1196 leader_election.cc:304] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.226467  1304 raft_consensus.cc:2749] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.229737  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.221982  1270 consensus_queue.cc:260] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.231519  1276 leader_election.cc:290] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.235363  1269 ts_tablet_manager.cc:1434] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.001s
I20260709 13:00:19.237983  1270 ts_tablet_manager.cc:1434] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.006s
I20260709 13:00:19.238093  1269 tablet_bootstrap.cc:492] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:19.232018  1230 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.239617  1270 tablet_bootstrap.cc:492] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:19.237664   979 catalog_manager.cc:5697] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58c9b0ef0380446991b7be408fe6b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.242853  1271 tablet_bootstrap.cc:492] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.243517  1271 ts_tablet_manager.cc:1403] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.026s
I20260709 13:00:19.247136  1271 raft_consensus.cc:359] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.247972  1271 raft_consensus.cc:385] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.248286  1271 raft_consensus.cc:740] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.249125  1270 tablet_bootstrap.cc:654] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.249236  1271 consensus_queue.cc:260] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.253469  1271 ts_tablet_manager.cc:1434] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:00:19.255231  1271 tablet_bootstrap.cc:492] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.259364  1269 tablet_bootstrap.cc:654] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.264882  1271 tablet_bootstrap.cc:654] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.275686  1269 tablet_bootstrap.cc:492] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.276757  1269 ts_tablet_manager.cc:1403] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.002s
I20260709 13:00:19.281524  1269 raft_consensus.cc:359] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.282888  1230 raft_consensus.cc:2468] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.283430  1269 raft_consensus.cc:385] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.290069  1269 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.291612  1269 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.293944  1122 leader_election.cc:304] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.296242  1269 ts_tablet_manager.cc:1434] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 13:00:19.297091  1276 raft_consensus.cc:2804] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.297681  1269 tablet_bootstrap.cc:492] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:19.297668  1276 raft_consensus.cc:697] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:19.302735  1276 consensus_queue.cc:237] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.309849  1271 tablet_bootstrap.cc:492] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.310649  1271 ts_tablet_manager.cc:1403] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.056s	user 0.027s	sys 0.023s
I20260709 13:00:19.311612  1270 tablet_bootstrap.cc:492] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.312170  1270 ts_tablet_manager.cc:1403] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.073s	user 0.037s	sys 0.009s
I20260709 13:00:19.314823  1296 raft_consensus.cc:493] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.315431  1296 raft_consensus.cc:515] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.315629  1270 raft_consensus.cc:359] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.317785  1270 raft_consensus.cc:385] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.317997  1296 leader_election.cc:290] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.318202  1270 raft_consensus.cc:740] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.319340  1270 consensus_queue.cc:260] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.320603  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.319466  1271 raft_consensus.cc:359] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
W20260709 13:00:19.322492  1122 leader_election.cc:343] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.322769  1271 raft_consensus.cc:385] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.323194  1271 raft_consensus.cc:740] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.324052  1271 consensus_queue.cc:260] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.324676  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.325628  1230 raft_consensus.cc:2468] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.326669  1271 ts_tablet_manager.cc:1434] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:00:19.327031  1122 leader_election.cc:304] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.327770  1271 tablet_bootstrap.cc:492] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.328419  1296 raft_consensus.cc:2804] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.328886  1296 raft_consensus.cc:493] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.329313  1296 raft_consensus.cc:3060] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.331939  1270 ts_tablet_manager.cc:1434] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:00:19.333649  1270 tablet_bootstrap.cc:492] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:19.335515   978 catalog_manager.cc:5697] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.344029  1271 tablet_bootstrap.cc:654] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.344125  1269 tablet_bootstrap.cc:654] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.344067  1296 raft_consensus.cc:515] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.345688  1270 tablet_bootstrap.cc:654] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.351619  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.352151  1271 tablet_bootstrap.cc:492] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.352692  1271 ts_tablet_manager.cc:1403] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.005s
W20260709 13:00:19.353412  1122 leader_election.cc:343] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.355967  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.355818  1271 raft_consensus.cc:359] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.356710  1271 raft_consensus.cc:385] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.356678  1230 raft_consensus.cc:3060] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.357226  1271 raft_consensus.cc:740] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.358248  1270 tablet_bootstrap.cc:492] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.358088  1271 consensus_queue.cc:260] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.358987  1270 ts_tablet_manager.cc:1403] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.000s
I20260709 13:00:19.361409  1271 ts_tablet_manager.cc:1434] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:00:19.362602  1271 tablet_bootstrap.cc:492] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.362660  1270 raft_consensus.cc:359] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.363574  1270 raft_consensus.cc:385] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.363902  1270 raft_consensus.cc:740] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.355985  1269 tablet_bootstrap.cc:492] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.365051  1269 ts_tablet_manager.cc:1403] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.001s
I20260709 13:00:19.364846  1270 consensus_queue.cc:260] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.367035  1277 raft_consensus.cc:493] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.367687  1270 ts_tablet_manager.cc:1434] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:00:19.367586  1277 raft_consensus.cc:515] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.368166  1269 raft_consensus.cc:359] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.368793  1270 tablet_bootstrap.cc:492] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:19.368916  1269 raft_consensus.cc:385] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.369279  1269 raft_consensus.cc:740] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.369997  1277 leader_election.cc:290] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.370041  1296 leader_election.cc:290] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.372798  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.373272  1230 raft_consensus.cc:2468] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
W20260709 13:00:19.374547  1047 leader_election.cc:343] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.374660  1122 leader_election.cc:304] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.376024  1296 raft_consensus.cc:2804] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.376271  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.376641  1296 raft_consensus.cc:697] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:19.377034  1229 raft_consensus.cc:2468] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 0.
I20260709 13:00:19.377601  1296 consensus_queue.cc:237] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.378768  1047 leader_election.cc:304] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea
I20260709 13:00:19.379302  1270 tablet_bootstrap.cc:654] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.380160  1269 consensus_queue.cc:260] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.383311  1269 ts_tablet_manager.cc:1434] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:00:19.384802  1269 tablet_bootstrap.cc:492] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:19.393424  1277 raft_consensus.cc:2804] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.396068  1283 raft_consensus.cc:493] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.400656  1277 raft_consensus.cc:493] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.401110  1277 raft_consensus.cc:3060] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.400823  1283 raft_consensus.cc:515] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.403293  1283 leader_election.cc:290] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.396432  1269 tablet_bootstrap.cc:654] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.399513   978 catalog_manager.cc:5697] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.404821  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.405580  1155 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 0.
I20260709 13:00:19.410992  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.411840  1230 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 0.
I20260709 13:00:19.413044  1277 raft_consensus.cc:515] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.414111  1047 leader_election.cc:304] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.415207  1283 raft_consensus.cc:2804] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.415664  1283 raft_consensus.cc:493] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.416021  1283 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.416898  1277 leader_election.cc:290] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.421059  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.422004  1230 raft_consensus.cc:3060] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.422031  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
W20260709 13:00:19.423817  1047 leader_election.cc:343] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.425010  1271 tablet_bootstrap.cc:654] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.425769  1270 tablet_bootstrap.cc:492] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.426343  1270 ts_tablet_manager.cc:1403] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.016s
I20260709 13:00:19.429692  1283 raft_consensus.cc:515] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.429847  1270 raft_consensus.cc:359] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.430794  1270 raft_consensus.cc:385] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.431056  1230 raft_consensus.cc:2468] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 1.
I20260709 13:00:19.431248  1270 raft_consensus.cc:740] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.433086  1283 leader_election.cc:290] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.432942  1047 leader_election.cc:304] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea
I20260709 13:00:19.433966  1277 raft_consensus.cc:2804] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.434541  1269 tablet_bootstrap.cc:492] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.435138  1269 ts_tablet_manager.cc:1403] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.003s
I20260709 13:00:19.435312  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:19.435963  1155 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.436738  1271 tablet_bootstrap.cc:492] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.437219  1271 ts_tablet_manager.cc:1403] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.003s
I20260709 13:00:19.434556  1277 raft_consensus.cc:697] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:19.438212  1270 consensus_queue.cc:260] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.438501  1277 consensus_queue.cc:237] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.440743  1270 ts_tablet_manager.cc:1434] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:00:19.441762  1270 tablet_bootstrap.cc:492] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: Bootstrap starting.
I20260709 13:00:19.442420  1271 raft_consensus.cc:359] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.443099  1271 raft_consensus.cc:385] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.443486  1271 raft_consensus.cc:740] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.444298  1271 consensus_queue.cc:260] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.447062  1271 ts_tablet_manager.cc:1434] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:00:19.448182  1271 tablet_bootstrap.cc:492] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0: Bootstrap starting.
I20260709 13:00:19.448801  1270 tablet_bootstrap.cc:654] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.452656  1269 raft_consensus.cc:359] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.453547  1269 raft_consensus.cc:385] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.453869  1269 raft_consensus.cc:740] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.454346  1283 raft_consensus.cc:493] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.454782  1269 consensus_queue.cc:260] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.454908  1283 raft_consensus.cc:515] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.455480  1271 tablet_bootstrap.cc:654] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.457088  1269 ts_tablet_manager.cc:1434] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.002s
I20260709 13:00:19.457309  1283 leader_election.cc:290] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.458266  1269 tablet_bootstrap.cc:492] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011: Bootstrap starting.
I20260709 13:00:19.460062  1156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.462327  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.463094  1230 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.462347  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
W20260709 13:00:19.464376  1047 leader_election.cc:343] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.465771  1269 tablet_bootstrap.cc:654] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:19.466545  1047 leader_election.cc:343] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.467043  1047 leader_election.cc:304] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:19.468827  1283 raft_consensus.cc:2749] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.472028  1155 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 1.
I20260709 13:00:19.474226  1047 leader_election.cc:304] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.475739  1283 raft_consensus.cc:2804] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.477280  1283 raft_consensus.cc:697] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:19.478641  1283 consensus_queue.cc:237] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.485512  1270 tablet_bootstrap.cc:492] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: No bootstrap required, opened a new log
I20260709 13:00:19.486107  1270 ts_tablet_manager.cc:1403] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.007s
I20260709 13:00:19.487167  1271 tablet_bootstrap.cc:492] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0: No bootstrap required, opened a new log
I20260709 13:00:19.487881  1271 ts_tablet_manager.cc:1403] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.005s
I20260709 13:00:19.488325  1270 raft_consensus.cc:359] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.489008  1270 raft_consensus.cc:385] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.489300  1270 raft_consensus.cc:740] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.490059  1270 consensus_queue.cc:260] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.495440  1271 raft_consensus.cc:359] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.496662  1271 raft_consensus.cc:385] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.497229  1271 raft_consensus.cc:740] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.498536  1270 ts_tablet_manager.cc:1434] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.002s
I20260709 13:00:19.498158  1271 consensus_queue.cc:260] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.502141  1271 ts_tablet_manager.cc:1434] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.002s
I20260709 13:00:19.505980  1230 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 1.
I20260709 13:00:19.507061  1269 tablet_bootstrap.cc:492] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011: No bootstrap required, opened a new log
I20260709 13:00:19.506408   978 catalog_manager.cc:5697] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.507701  1269 ts_tablet_manager.cc:1403] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.002s
I20260709 13:00:19.511111  1269 raft_consensus.cc:359] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.512029  1269 raft_consensus.cc:385] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.512555  1269 raft_consensus.cc:740] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.513538  1269 consensus_queue.cc:260] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.517347  1269 ts_tablet_manager.cc:1434] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:00:19.528798  1296 raft_consensus.cc:493] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.529476  1296 raft_consensus.cc:515] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.530931   979 catalog_manager.cc:5697] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.532472  1296 leader_election.cc:290] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449), ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171)
I20260709 13:00:19.533947  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.534013  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.535117  1230 raft_consensus.cc:2468] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.535709  1080 raft_consensus.cc:2468] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.536463  1122 leader_election.cc:304] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:19.537408  1296 raft_consensus.cc:2804] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.537925  1296 raft_consensus.cc:493] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.538372  1296 raft_consensus.cc:3060] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.541226  1276 raft_consensus.cc:493] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.541733  1276 raft_consensus.cc:515] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.543833  1276 leader_election.cc:290] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.544889  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.545130  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.545547  1080 raft_consensus.cc:2468] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.546025  1230 raft_consensus.cc:2468] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.546962  1122 leader_election.cc:304] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.548246  1276 raft_consensus.cc:2804] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.547901  1296 raft_consensus.cc:515] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.548731  1276 raft_consensus.cc:493] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.549171  1276 raft_consensus.cc:3060] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.550670  1296 leader_election.cc:290] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Requested vote from peers 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449), ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171)
I20260709 13:00:19.551874  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.552163  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.552713  1230 raft_consensus.cc:3060] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.552788  1080 raft_consensus.cc:3060] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.553484  1296 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.554297  1296 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.556951  1296 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.557967  1276 raft_consensus.cc:515] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.560935  1276 leader_election.cc:290] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.562417  1080 raft_consensus.cc:2468] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.562680  1283 consensus_queue.cc:1048] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.564800  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.565023  1230 raft_consensus.cc:2468] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.565565  1081 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.568128  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.568128  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.568399  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.568897  1122 leader_election.cc:304] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.569080  1081 raft_consensus.cc:3060] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.569363  1229 raft_consensus.cc:3060] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.571441  1276 raft_consensus.cc:2804] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.571727  1122 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.572105  1230 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 0.
I20260709 13:00:19.573382  1276 raft_consensus.cc:697] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:19.574533  1296 raft_consensus.cc:2804] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.575042  1296 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.575623  1296 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.574544  1276 consensus_queue.cc:237] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:19.581442  1081 raft_consensus.cc:2468] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.582398  1229 raft_consensus.cc:2468] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.583220  1122 leader_election.cc:304] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:19.585548  1332 raft_consensus.cc:2804] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.590602  1296 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.598151  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:19.598937  1081 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.605144  1332 raft_consensus.cc:697] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:19.605942  1080 raft_consensus.cc:3060] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.606266  1332 consensus_queue.cc:237] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.612499  1229 raft_consensus.cc:3060] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.617009   979 catalog_manager.cc:5697] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.618889  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:19.619498  1230 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.618054  1296 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.620692  1334 consensus_queue.cc:1048] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.626303  1304 consensus_queue.cc:1048] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.631595  1156 raft_consensus.cc:3060] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.634675  1230 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.636456  1122 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:19.637548  1334 raft_consensus.cc:2804] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.638105  1334 raft_consensus.cc:697] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:19.639057  1334 consensus_queue.cc:237] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.639807  1283 consensus_queue.cc:1048] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.649376  1283 raft_consensus.cc:493] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.650079  1283 raft_consensus.cc:515] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.653195  1283 leader_election.cc:290] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.653512   979 catalog_manager.cc:5697] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.655321  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.657837  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97aa01b06414cf4bb9dabdbd9cec5c7" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.658996  1230 raft_consensus.cc:2393] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca1e65a96e454f558f348160072c35d0 in current term 1: Already voted for candidate 4c2058117a3f4f608d6bbd90694cd4ea in this term.
I20260709 13:00:19.654322  1296 consensus_queue.cc:1048] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.662366  1081 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 1.
I20260709 13:00:19.664860  1047 leader_election.cc:304] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:19.671571  1283 raft_consensus.cc:3060] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.672698  1296 raft_consensus.cc:493] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.673497  1306 consensus_queue.cc:1048] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.674131  1296 raft_consensus.cc:515] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.677549  1296 leader_election.cc:290] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.681720  1081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63de5ca351e64e4587621c643e5b0121" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.682719  1230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63de5ca351e64e4587621c643e5b0121" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.684804  1122 leader_election.cc:304] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea; no voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.691053  1296 raft_consensus.cc:2749] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.688722   979 catalog_manager.cc:5697] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.715996  1277 raft_consensus.cc:493] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.716638  1283 raft_consensus.cc:2749] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.716702  1277 raft_consensus.cc:515] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.725201  1277 leader_election.cc:290] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.727562  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" is_pre_election: true
I20260709 13:00:19.728078  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.729574  1228 raft_consensus.cc:2393] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca1e65a96e454f558f348160072c35d0 in current term 1: Already voted for candidate 4c2058117a3f4f608d6bbd90694cd4ea in this term.
I20260709 13:00:19.731405  1047 leader_election.cc:304] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:19.733808  1277 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.744122  1277 raft_consensus.cc:2749] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.772284  1296 consensus_queue.cc:1048] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.782829  1276 consensus_queue.cc:1048] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.789938  1276 consensus_queue.cc:1048] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.818279  1335 consensus_queue.cc:1048] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.847100  1156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:54636
I20260709 13:00:19.847896  1156 raft_consensus.cc:606] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.862453  1156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:54636
I20260709 13:00:19.863317  1156 raft_consensus.cc:606] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.930473  1335 raft_consensus.cc:493] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.930945  1335 raft_consensus.cc:515] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:19.932991  1335 leader_election.cc:290] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:19.934008  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1e65a96e454f558f348160072c35d0" is_pre_election: true
I20260709 13:00:19.934427  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58c9b0ef0380446991b7be408fe6b011" is_pre_election: true
I20260709 13:00:19.935491  1228 raft_consensus.cc:2393] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4c2058117a3f4f608d6bbd90694cd4ea in current term 1: Already voted for candidate ca1e65a96e454f558f348160072c35d0 in this term.
I20260709 13:00:19.936961  1122 leader_election.cc:304] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea; no voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0
I20260709 13:00:19.937934  1335 raft_consensus.cc:3060] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.944331  1335 raft_consensus.cc:2749] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.946949  1277 consensus_queue.cc:1048] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.960156  1277 consensus_queue.cc:1048] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.972934  1277 consensus_queue.cc:1048] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.989555  1283 consensus_queue.cc:1048] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.999194  1335 consensus_queue.cc:1048] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.014144  1334 consensus_queue.cc:1048] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:20.029853  1276 raft_consensus.cc:993] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:20.030544  1276 raft_consensus.cc:1081] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:20.032734  1080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46098
I20260709 13:00:20.033336  1080 raft_consensus.cc:493] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.033748  1080 raft_consensus.cc:3060] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.042927  1080 raft_consensus.cc:515] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:20.044816  1276 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.047066  1276 consensus_queue.cc:1048] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.047343  1080 leader_election.cc:290] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:20.047255  1156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:20.047963  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36bedd9ba1194844af99715a6dfc1eb0" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:20.048509  1156 raft_consensus.cc:3055] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.048684  1228 raft_consensus.cc:3060] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.048969  1156 raft_consensus.cc:740] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:20.050037  1156 consensus_queue.cc:260] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:20.053347  1156 raft_consensus.cc:3060] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.064810  1276 consensus_queue.cc:1048] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:20.068369  1228 raft_consensus.cc:2468] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:20.073457  1047 leader_election.cc:304] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:20.075045  1300 raft_consensus.cc:2804] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.074826  1156 raft_consensus.cc:2468] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:20.075747  1300 raft_consensus.cc:697] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:20.077740  1300 consensus_queue.cc:237] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } }
I20260709 13:00:20.077780  1276 raft_consensus.cc:993] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:20.083599  1332 raft_consensus.cc:1081] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:20.086669  1079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46098
I20260709 13:00:20.087389  1079 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.087860  1079 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.093665  1332 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:20.099291   977 catalog_manager.cc:5697] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 1 to 2, leader changed from 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.118592  1079 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:20.123678  1156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:20.124528  1156 raft_consensus.cc:3055] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.124928  1156 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:20.126188  1156 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:20.127265  1156 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.127744  1079 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:20.130362  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:20.131496  1229 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.139573  1156 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:20.141162  1047 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:20.142052  1229 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:20.142365  1300 raft_consensus.cc:2804] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.143031  1300 raft_consensus.cc:697] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:20.143971  1300 consensus_queue.cc:237] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:20.154172   978 catalog_manager.cc:5697] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 1 to 2, leader changed from 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.525583  1156 raft_consensus.cc:1275] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.527155  1283 consensus_queue.cc:1048] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.537375  1229 raft_consensus.cc:1275] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.539166  1277 consensus_queue.cc:1048] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.699733  1156 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.701475  1283 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.712229  1229 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.714445  1300 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:23.035701   431 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:00:23.037009   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:23.070284  1080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
mode: GRACEFUL
new_leader_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:00:23.071738  1080 raft_consensus.cc:606] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Received request to transfer leadership to TS 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:23.075574  1080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
mode: GRACEFUL
new_leader_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:00:23.076314  1080 raft_consensus.cc:606] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Received request to transfer leadership to TS 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:23.078012   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:23.078792   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:23.095026  1277 raft_consensus.cc:993] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: : Instructing follower 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:23.096550  1366 raft_consensus.cc:1081] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Signalling peer 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:23.099550  1229 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:44312
I20260709 13:00:23.100623  1229 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:23.101089  1229 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.116947  1229 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:23.120078  1229 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 3 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:23.121409  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:23.121907  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:23.122443  1080 raft_consensus.cc:3055] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:23.122778  1155 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.123407  1080 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:23.124660  1080 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:23.127923  1080 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.134850  1155 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 3.
I20260709 13:00:23.136842  1196 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:23.136972  1080 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 3.
I20260709 13:00:23.138227  1347 raft_consensus.cc:2804] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:23.139828  1347 raft_consensus.cc:697] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 LEADER]: Becoming Leader. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:23.141892  1347 consensus_queue.cc:237] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:23.152964   978 catalog_manager.cc:5697] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 reported cstate change: term changed from 2 to 3, leader changed from ca1e65a96e454f558f348160072c35d0 (127.0.107.193) to 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58c9b0ef0380446991b7be408fe6b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.505419  1366 raft_consensus.cc:993] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: : Instructing follower 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:23.506000  1277 raft_consensus.cc:1081] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Signalling peer 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:23.508565  1229 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79"
dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:44312
I20260709 13:00:23.509320  1229 raft_consensus.cc:493] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:23.510087  1229 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:23.552807  1229 raft_consensus.cc:515] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:23.555922  1229 leader_election.cc:290] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 2 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:23.557354  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:23.557564  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:23.558107  1080 raft_consensus.cc:3055] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:23.558264  1155 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:23.558552  1080 raft_consensus.cc:740] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:23.559589  1080 consensus_queue.cc:260] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:23.561185  1080 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:23.605612  1079 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:23.607331  1347 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:23.621680  1156 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:23.623809  1347 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:23.658823  1155 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 2.
I20260709 13:00:23.658823  1080 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 2.
I20260709 13:00:23.660638  1196 leader_election.cc:304] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:23.661707  1347 raft_consensus.cc:2804] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:23.662997  1347 raft_consensus.cc:697] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Becoming Leader. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:23.664278  1347 consensus_queue.cc:237] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:23.675093   978 catalog_manager.cc:5697] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 reported cstate change: term changed from 1 to 2, leader changed from ca1e65a96e454f558f348160072c35d0 (127.0.107.193) to 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58c9b0ef0380446991b7be408fe6b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:24.080260   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:24.110167  1229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79"
dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:34340
I20260709 13:00:24.110956  1229 raft_consensus.cc:606] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:24.121804  1229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:34340
I20260709 13:00:24.122634  1229 raft_consensus.cc:606] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:24.131487  1080 raft_consensus.cc:1275] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.133432  1347 consensus_queue.cc:1048] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:24.133958  1155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:54636
I20260709 13:00:24.134752  1155 raft_consensus.cc:606] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:24.143265  1155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:54636
I20260709 13:00:24.144537  1155 raft_consensus.cc:606] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:24.144531  1371 raft_consensus.cc:993] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.146579  1371 raft_consensus.cc:1081] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.148748  1079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46112
I20260709 13:00:24.149533  1079 raft_consensus.cc:493] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.150103  1079 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.154451  1155 raft_consensus.cc:1275] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.158286  1155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
mode: GRACEFUL
new_leader_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:54636
I20260709 13:00:24.159320  1155 raft_consensus.cc:606] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Received request to transfer leadership to TS ca1e65a96e454f558f348160072c35d0
I20260709 13:00:24.161703  1371 consensus_queue.cc:1048] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:24.165930  1079 raft_consensus.cc:515] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.168442  1080 raft_consensus.cc:1240] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Rejecting Update request from peer 58c9b0ef0380446991b7be408fe6b011 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:24.171746  1154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:24.173105  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75713d192a6b42399478f4f00e2a9c79" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:24.171813  1371 consensus_queue.cc:1059] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:00:24.175077  1079 leader_election.cc:290] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 3 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:24.177955  1229 raft_consensus.cc:3055] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:24.178597  1229 raft_consensus.cc:740] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:24.180099  1229 consensus_queue.cc:260] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.185240  1229 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.236545  1368 raft_consensus.cc:993] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.237226  1396 raft_consensus.cc:1081] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.239372  1079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46098
I20260709 13:00:24.240216  1079 raft_consensus.cc:493] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.240819  1079 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.269821  1229 raft_consensus.cc:2468] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 3.
I20260709 13:00:24.269861  1079 raft_consensus.cc:515] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
W20260709 13:00:24.271530  1393 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:00:24.272946  1079 leader_election.cc:290] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:24.273077  1047 leader_election.cc:304] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 4c2058117a3f4f608d6bbd90694cd4ea
I20260709 13:00:24.275207  1277 raft_consensus.cc:2804] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:24.275900  1277 raft_consensus.cc:697] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 3 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:24.277437  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:24.277870  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:24.278079  1228 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.277624  1277 consensus_queue.cc:237] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.278544  1155 raft_consensus.cc:3055] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:24.278959  1155 raft_consensus.cc:740] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:24.279917  1155 consensus_queue.cc:260] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.281467  1155 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.289599   978 catalog_manager.cc:5697] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 2 to 3, leader changed from 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.304489  1368 raft_consensus.cc:993] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.304980  1396 raft_consensus.cc:1081] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.306990  1079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46098
I20260709 13:00:24.307472  1079 raft_consensus.cc:493] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.307754  1079 raft_consensus.cc:3060] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.375849  1228 raft_consensus.cc:2468] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:24.375849  1155 raft_consensus.cc:2468] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:24.375988  1079 raft_consensus.cc:515] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.378262  1047 leader_election.cc:304] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:24.379074  1079 leader_election.cc:290] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:24.379987  1277 raft_consensus.cc:2804] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:24.380939  1277 raft_consensus.cc:697] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:24.382674  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:24.382215  1277 consensus_queue.cc:237] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.382792  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909afa693e8b46e698b875b6a0a79c9a" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:24.383261  1228 raft_consensus.cc:3060] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.383446  1155 raft_consensus.cc:3055] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:24.384083  1155 raft_consensus.cc:740] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:24.385089  1155 consensus_queue.cc:260] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.386543  1155 raft_consensus.cc:3060] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.395504   978 catalog_manager.cc:5697] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 1 to 2, leader changed from 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.443812  1368 raft_consensus.cc:993] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.444517  1396 raft_consensus.cc:1081] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.447247  1079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46098
I20260709 13:00:24.448038  1079 raft_consensus.cc:493] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.448786  1079 raft_consensus.cc:3060] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.482743  1155 raft_consensus.cc:2468] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:24.482743  1228 raft_consensus.cc:2468] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:24.482792  1079 raft_consensus.cc:515] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.485142  1047 leader_election.cc:304] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:24.486135  1079 leader_election.cc:290] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:24.488062  1277 raft_consensus.cc:2804] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:24.488250  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:24.488168  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37bfb53e2f744bfda4e7f76f95c5aef3" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:24.488719  1277 raft_consensus.cc:697] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:24.489229  1229 raft_consensus.cc:3060] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.489202  1155 raft_consensus.cc:3055] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:24.489992  1155 raft_consensus.cc:740] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:24.490650  1277 consensus_queue.cc:237] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.491235  1155 consensus_queue.cc:260] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.492684  1155 raft_consensus.cc:3060] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:24.503364   978 catalog_manager.cc:5697] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 1 to 2, leader changed from 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.589478  1155 raft_consensus.cc:2468] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:24.589478  1229 raft_consensus.cc:2468] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 2.
I20260709 13:00:24.591486  1047 leader_election.cc:304] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:24.592449  1277 raft_consensus.cc:2804] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:24.592983  1277 raft_consensus.cc:697] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:24.593961  1277 consensus_queue.cc:237] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.602187   978 catalog_manager.cc:5697] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 1 to 2, leader changed from 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.641433  1371 raft_consensus.cc:993] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: : Instructing follower ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.642110  1393 raft_consensus.cc:1081] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 LEADER]: Signalling peer ca1e65a96e454f558f348160072c35d0 to start an election
I20260709 13:00:24.644505  1079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
 from {username='slave'} at 127.0.0.1:46112
I20260709 13:00:24.645233  1079 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.645728  1079 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.697013  1079 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.699565  1079 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 4 election: Requested vote from peers 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:24.701077  1155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:24.702113  1229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "ca1e65a96e454f558f348160072c35d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:24.702435  1155 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.702680  1229 raft_consensus.cc:3055] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:24.703293  1229 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:24.704093  1229 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.705365  1229 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.780732  1228 raft_consensus.cc:1275] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.781401  1154 raft_consensus.cc:3060] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.782728  1277 consensus_queue.cc:1048] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:24.798873  1156 raft_consensus.cc:1275] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.800648  1406 consensus_queue.cc:1048] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:24.804774  1229 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 4.
I20260709 13:00:24.806057  1047 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58c9b0ef0380446991b7be408fe6b011, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:24.807304  1397 raft_consensus.cc:2804] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:24.808080  1397 raft_consensus.cc:697] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 LEADER]: Becoming Leader. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:24.809221  1397 consensus_queue.cc:237] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:24.811566  1228 raft_consensus.cc:1275] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.813716  1406 consensus_queue.cc:1048] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:24.816336  1155 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1e65a96e454f558f348160072c35d0 in term 4.
I20260709 13:00:24.820022   978 catalog_manager.cc:5697] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 reported cstate change: term changed from 3 to 4, leader changed from 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195) to ca1e65a96e454f558f348160072c35d0 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "ca1e65a96e454f558f348160072c35d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.882079  1228 raft_consensus.cc:1275] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.884474  1397 consensus_queue.cc:1048] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:24.893287  1154 raft_consensus.cc:1275] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.896278  1397 consensus_queue.cc:1048] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:00:24.903698  1154 raft_consensus.cc:1275] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.906332  1397 consensus_queue.cc:1048] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:24.994374  1228 raft_consensus.cc:1275] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:24.996385  1412 consensus_queue.cc:1048] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:25.008540  1154 raft_consensus.cc:1275] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:25.011021  1277 consensus_queue.cc:1048] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:25.289865  1154 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 4 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:25.291505  1277 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:25.302138  1228 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 4 FOLLOWER]: Refusing update from remote peer ca1e65a96e454f558f348160072c35d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:25.304281  1277 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:27.184917   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:27.194557  1079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
mode: GRACEFUL
new_leader_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:00:27.195220  1079 raft_consensus.cc:606] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 LEADER]: Received request to transfer leadership to TS 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:27.198378  1079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
mode: GRACEFUL
new_leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:00:27.199136  1079 raft_consensus.cc:606] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Received request to transfer leadership to TS 4c2058117a3f4f608d6bbd90694cd4ea
I20260709 13:00:27.202469  1079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98314e81f7c14579b10079f98293e8de"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
mode: GRACEFUL
new_leader_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:00:27.203078  1079 raft_consensus.cc:606] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Received request to transfer leadership to TS 58c9b0ef0380446991b7be408fe6b011
I20260709 13:00:27.206463  1079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01"
dest_uuid: "ca1e65a96e454f558f348160072c35d0"
mode: GRACEFUL
new_leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:00:27.207185  1079 raft_consensus.cc:606] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Received request to transfer leadership to TS 4c2058117a3f4f608d6bbd90694cd4ea
I20260709 13:00:27.208503   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:27.209072   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:27.335911  1406 raft_consensus.cc:993] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: : Instructing follower 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:27.336525  1420 raft_consensus.cc:1081] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Signalling peer 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:27.338263  1228 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98314e81f7c14579b10079f98293e8de"
dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:44312
I20260709 13:00:27.338922  1228 raft_consensus.cc:493] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.339248  1228 raft_consensus.cc:3060] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.343899  1406 raft_consensus.cc:993] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: : Instructing follower 4c2058117a3f4f608d6bbd90694cd4ea to start an election
I20260709 13:00:27.344795  1420 raft_consensus.cc:1081] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Signalling peer 4c2058117a3f4f608d6bbd90694cd4ea to start an election
I20260709 13:00:27.347962  1154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
 from {username='slave'} at 127.0.0.1:54626
I20260709 13:00:27.348201  1228 raft_consensus.cc:515] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.348753  1154 raft_consensus.cc:493] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.349489  1154 raft_consensus.cc:3060] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.351287  1228 leader_election.cc:290] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 2 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:27.352815  1079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98314e81f7c14579b10079f98293e8de" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:27.352998  1156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98314e81f7c14579b10079f98293e8de" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:27.353673  1156 raft_consensus.cc:3060] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.353628  1079 raft_consensus.cc:3055] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:27.354326  1079 raft_consensus.cc:740] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:27.355435  1079 consensus_queue.cc:260] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.357045  1079 raft_consensus.cc:3060] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.358825  1154 raft_consensus.cc:515] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.360946  1156 raft_consensus.cc:2468] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 2.
I20260709 13:00:27.361547  1154 leader_election.cc:290] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 2 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:27.362499  1196 leader_election.cc:304] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:27.362815  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:27.362830  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5906670ff14a83abcb490f63361b01" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:27.363507  1419 raft_consensus.cc:2804] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:27.363950  1080 raft_consensus.cc:3055] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:27.364564  1080 raft_consensus.cc:740] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:27.365374  1419 raft_consensus.cc:697] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Becoming Leader. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:27.365716  1080 consensus_queue.cc:260] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.366420  1419 consensus_queue.cc:237] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.367098  1080 raft_consensus.cc:3060] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.368254  1079 raft_consensus.cc:2468] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 2.
I20260709 13:00:27.368793  1228 raft_consensus.cc:3060] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.378046  1080 raft_consensus.cc:2468] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 2.
I20260709 13:00:27.378079  1228 raft_consensus.cc:2468] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 2.
I20260709 13:00:27.377802   977 catalog_manager.cc:5697] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 reported cstate change: term changed from 1 to 2, leader changed from ca1e65a96e454f558f348160072c35d0 (127.0.107.193) to 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58c9b0ef0380446991b7be408fe6b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.379606  1122 leader_election.cc:304] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, ca1e65a96e454f558f348160072c35d0; no voters: 
I20260709 13:00:27.380527  1396 raft_consensus.cc:2804] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:27.381855  1396 raft_consensus.cc:697] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:27.382932  1396 consensus_queue.cc:237] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.395448   978 catalog_manager.cc:5697] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 1 to 2, leader changed from ca1e65a96e454f558f348160072c35d0 (127.0.107.193) to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 2 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.415997  1420 raft_consensus.cc:993] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: : Instructing follower 4c2058117a3f4f608d6bbd90694cd4ea to start an election
I20260709 13:00:27.416626  1406 raft_consensus.cc:1081] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Signalling peer 4c2058117a3f4f608d6bbd90694cd4ea to start an election
I20260709 13:00:27.418498  1156 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70"
dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
 from {username='slave'} at 127.0.0.1:54626
I20260709 13:00:27.419014  1156 raft_consensus.cc:493] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.419291  1156 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.425344  1156 raft_consensus.cc:515] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.427723  1156 leader_election.cc:290] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 3 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195:33449)
I20260709 13:00:27.429028  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:27.429482  1228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e5efa3edd19465a83308d4b5400fd70" candidate_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
I20260709 13:00:27.429679  1080 raft_consensus.cc:3055] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:27.430099  1228 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.430167  1080 raft_consensus.cc:740] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:27.431207  1080 consensus_queue.cc:260] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.432331  1080 raft_consensus.cc:3060] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.437382  1228 raft_consensus.cc:2468] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 3.
I20260709 13:00:27.438885  1122 leader_election.cc:304] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:27.439571  1080 raft_consensus.cc:2468] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2058117a3f4f608d6bbd90694cd4ea in term 3.
I20260709 13:00:27.440080  1396 raft_consensus.cc:2804] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:27.440662  1396 raft_consensus.cc:697] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 LEADER]: Becoming Leader. State: Replica: 4c2058117a3f4f608d6bbd90694cd4ea, State: Running, Role: LEADER
I20260709 13:00:27.441566  1396 consensus_queue.cc:237] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.450289   978 catalog_manager.cc:5697] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea reported cstate change: term changed from 2 to 3, leader changed from ca1e65a96e454f558f348160072c35d0 (127.0.107.193) to 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194). New cstate: current_term: 3 leader_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.594498  1420 raft_consensus.cc:993] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: : Instructing follower 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:27.595043  1406 raft_consensus.cc:1081] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 LEADER]: Signalling peer 58c9b0ef0380446991b7be408fe6b011 to start an election
I20260709 13:00:27.596830  1228 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462"
dest_uuid: "58c9b0ef0380446991b7be408fe6b011"
 from {username='slave'} at 127.0.0.1:44312
I20260709 13:00:27.597391  1228 raft_consensus.cc:493] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.597723  1228 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.603667  1228 raft_consensus.cc:515] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.605919  1228 leader_election.cc:290] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 5 election: Requested vote from peers ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171), 4c2058117a3f4f608d6bbd90694cd4ea (127.0.107.194:44091)
I20260709 13:00:27.606976  1080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ca1e65a96e454f558f348160072c35d0"
I20260709 13:00:27.607270  1156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447eea4219634eb6a3aa17c9417f2462" candidate_uuid: "58c9b0ef0380446991b7be408fe6b011" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4c2058117a3f4f608d6bbd90694cd4ea"
I20260709 13:00:27.607571  1080 raft_consensus.cc:3055] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.607824  1156 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.607947  1080 raft_consensus.cc:740] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca1e65a96e454f558f348160072c35d0, State: Running, Role: LEADER
I20260709 13:00:27.608912  1080 consensus_queue.cc:260] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.609959  1080 raft_consensus.cc:3060] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.614120  1156 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 5.
I20260709 13:00:27.615510  1196 leader_election.cc:304] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2058117a3f4f608d6bbd90694cd4ea, 58c9b0ef0380446991b7be408fe6b011; no voters: 
I20260709 13:00:27.616118  1080 raft_consensus.cc:2468] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58c9b0ef0380446991b7be408fe6b011 in term 5.
I20260709 13:00:27.616464  1419 raft_consensus.cc:2804] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.617004  1419 raft_consensus.cc:697] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 5 LEADER]: Becoming Leader. State: Replica: 58c9b0ef0380446991b7be408fe6b011, State: Running, Role: LEADER
I20260709 13:00:27.617949  1419 consensus_queue.cc:237] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } }
I20260709 13:00:27.626883   978 catalog_manager.cc:5697] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 reported cstate change: term changed from 4 to 5, leader changed from ca1e65a96e454f558f348160072c35d0 (127.0.107.193) to 58c9b0ef0380446991b7be408fe6b011 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "58c9b0ef0380446991b7be408fe6b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.844478  1080 raft_consensus.cc:1275] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:27.846362  1419 consensus_queue.cc:1048] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:27.854723  1079 raft_consensus.cc:1275] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Refusing update from remote peer 4c2058117a3f4f608d6bbd90694cd4ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:27.856891  1156 raft_consensus.cc:1275] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:27.856833  1396 consensus_queue.cc:1048] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:27.861480  1419 consensus_queue.cc:1048] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:27.895550  1228 raft_consensus.cc:1275] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Refusing update from remote peer 4c2058117a3f4f608d6bbd90694cd4ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:27.897190  1396 consensus_queue.cc:1048] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:27.910778  1080 raft_consensus.cc:1275] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Refusing update from remote peer 4c2058117a3f4f608d6bbd90694cd4ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:27.913180  1396 consensus_queue.cc:1048] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:27.940693  1228 raft_consensus.cc:1275] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Refusing update from remote peer 4c2058117a3f4f608d6bbd90694cd4ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:27.942682  1423 consensus_queue.cc:1048] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c9b0ef0380446991b7be408fe6b011" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:28.045176  1080 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 5 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:28.046942  1419 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1e65a96e454f558f348160072c35d0" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:28.060863  1156 raft_consensus.cc:1275] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 5 FOLLOWER]: Refusing update from remote peer 58c9b0ef0380446991b7be408fe6b011: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:28.063161  1421 consensus_queue.cc:1048] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2058117a3f4f608d6bbd90694cd4ea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:28.210011   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:28.218621   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:00:28.258985   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:28.260234   431 tablet_replica.cc:333] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.261253   431 raft_consensus.cc:2243] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.261866   431 raft_consensus.cc:2272] T c97aa01b06414cf4bb9dabdbd9cec5c7 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.263881   431 tablet_replica.cc:333] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.264518   431 raft_consensus.cc:2243] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.265260   431 raft_consensus.cc:2272] T 909afa693e8b46e698b875b6a0a79c9a P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.267352   431 tablet_replica.cc:333] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.268055   431 raft_consensus.cc:2243] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.269021   431 raft_consensus.cc:2272] T 37bfb53e2f744bfda4e7f76f95c5aef3 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.271569   431 tablet_replica.cc:333] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.272387   431 raft_consensus.cc:2243] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.272989   431 raft_consensus.cc:2272] T 4e5efa3edd19465a83308d4b5400fd70 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.275766   431 tablet_replica.cc:333] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.276664   431 raft_consensus.cc:2243] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.277573   431 raft_consensus.cc:2272] T 75713d192a6b42399478f4f00e2a9c79 P ca1e65a96e454f558f348160072c35d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.280207   431 tablet_replica.cc:333] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.280900   431 raft_consensus.cc:2243] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.281410   431 raft_consensus.cc:2272] T 98314e81f7c14579b10079f98293e8de P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.285055   431 tablet_replica.cc:333] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.285923   431 raft_consensus.cc:2243] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.286516   431 raft_consensus.cc:2272] T 63de5ca351e64e4587621c643e5b0121 P ca1e65a96e454f558f348160072c35d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.289448   431 tablet_replica.cc:333] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.290736   431 raft_consensus.cc:2243] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.291766   431 raft_consensus.cc:2272] T aa5906670ff14a83abcb490f63361b01 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:28.292984  1122 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
I20260709 13:00:28.295387   431 tablet_replica.cc:333] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.296640   431 raft_consensus.cc:2243] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.297279   431 raft_consensus.cc:2272] T 447eea4219634eb6a3aa17c9417f2462 P ca1e65a96e454f558f348160072c35d0 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:28.297312  1122 consensus_peers.cc:597] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea -> Peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Couldn't send request to peer ca1e65a96e454f558f348160072c35d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.299899   431 tablet_replica.cc:333] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0: stopping tablet replica
I20260709 13:00:28.300695   431 raft_consensus.cc:2243] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.301863   431 raft_consensus.cc:2272] T 36bedd9ba1194844af99715a6dfc1eb0 P ca1e65a96e454f558f348160072c35d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.336663   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:00:28.360900   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:00:28.379487  1196 consensus_peers.cc:597] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 -> Peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Couldn't send request to peer ca1e65a96e454f558f348160072c35d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.398188   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:28.399230   431 tablet_replica.cc:333] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.399962   431 raft_consensus.cc:2243] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:00:28.400563  1196 consensus_peers.cc:597] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 -> Peer ca1e65a96e454f558f348160072c35d0 (127.0.107.193:46171): Couldn't send request to peer ca1e65a96e454f558f348160072c35d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.400771   431 raft_consensus.cc:2272] T 36bedd9ba1194844af99715a6dfc1eb0 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.403190   431 tablet_replica.cc:333] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.403813   431 raft_consensus.cc:2243] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.404970   431 raft_consensus.cc:2272] T aa5906670ff14a83abcb490f63361b01 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.408169   431 tablet_replica.cc:333] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.409570   431 raft_consensus.cc:2243] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.410370   431 raft_consensus.cc:2272] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.413025   431 tablet_replica.cc:333] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.413730   431 raft_consensus.cc:2243] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.414638   431 raft_consensus.cc:2272] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.416874   431 tablet_replica.cc:333] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.417510   431 raft_consensus.cc:2243] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.418139   431 raft_consensus.cc:2272] T 447eea4219634eb6a3aa17c9417f2462 P 4c2058117a3f4f608d6bbd90694cd4ea [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.421227   431 tablet_replica.cc:333] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.421981   431 raft_consensus.cc:2243] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.422837   431 raft_consensus.cc:2272] T 75713d192a6b42399478f4f00e2a9c79 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.425743   431 tablet_replica.cc:333] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.426546   431 raft_consensus.cc:2243] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.427683   431 raft_consensus.cc:2272] T 4e5efa3edd19465a83308d4b5400fd70 P 4c2058117a3f4f608d6bbd90694cd4ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.430531   431 tablet_replica.cc:333] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.431383   431 raft_consensus.cc:2243] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.431999   431 raft_consensus.cc:2272] T 909afa693e8b46e698b875b6a0a79c9a P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.434235   431 tablet_replica.cc:333] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.434796   431 raft_consensus.cc:2243] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.435276   431 raft_consensus.cc:2272] T 63de5ca351e64e4587621c643e5b0121 P 4c2058117a3f4f608d6bbd90694cd4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.437337   431 tablet_replica.cc:333] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea: stopping tablet replica
I20260709 13:00:28.437868   431 raft_consensus.cc:2243] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.438344   431 raft_consensus.cc:2272] T 98314e81f7c14579b10079f98293e8de P 4c2058117a3f4f608d6bbd90694cd4ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.473932   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:00:28.496862   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:00:28.532902   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:28.533855   431 tablet_replica.cc:333] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.534675   431 raft_consensus.cc:2243] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.535256   431 raft_consensus.cc:2272] T 36bedd9ba1194844af99715a6dfc1eb0 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.537505   431 tablet_replica.cc:333] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.538175   431 raft_consensus.cc:2243] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.539605   431 raft_consensus.cc:2272] T 447eea4219634eb6a3aa17c9417f2462 P 58c9b0ef0380446991b7be408fe6b011 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.544064   431 tablet_replica.cc:333] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.545331   431 raft_consensus.cc:2243] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.545972   431 raft_consensus.cc:2272] T aa5906670ff14a83abcb490f63361b01 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.549624   431 tablet_replica.cc:333] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.550540   431 raft_consensus.cc:2243] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.551177   431 raft_consensus.cc:2272] T 75713d192a6b42399478f4f00e2a9c79 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.552989   431 tablet_replica.cc:333] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.553524   431 raft_consensus.cc:2243] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.553925   431 raft_consensus.cc:2272] T 4e5efa3edd19465a83308d4b5400fd70 P 58c9b0ef0380446991b7be408fe6b011 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.556955   431 tablet_replica.cc:333] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.558038   431 raft_consensus.cc:2243] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.558705   431 raft_consensus.cc:2272] T c97aa01b06414cf4bb9dabdbd9cec5c7 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.560642   431 tablet_replica.cc:333] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.561108   431 raft_consensus.cc:2243] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.561458   431 raft_consensus.cc:2272] T 37bfb53e2f744bfda4e7f76f95c5aef3 P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.563297   431 tablet_replica.cc:333] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.563848   431 raft_consensus.cc:2243] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.564893   431 raft_consensus.cc:2272] T 98314e81f7c14579b10079f98293e8de P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.567492   431 tablet_replica.cc:333] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.568145   431 raft_consensus.cc:2243] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.569213   431 raft_consensus.cc:2272] T 63de5ca351e64e4587621c643e5b0121 P 58c9b0ef0380446991b7be408fe6b011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.571275   431 tablet_replica.cc:333] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011: stopping tablet replica
I20260709 13:00:28.571843   431 raft_consensus.cc:2243] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.572417   431 raft_consensus.cc:2272] T 909afa693e8b46e698b875b6a0a79c9a P 58c9b0ef0380446991b7be408fe6b011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.608616   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:00:28.634358   431 master.cc:562] Master@127.0.107.254:34809 shutting down...
I20260709 13:00:28.670725   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.671603   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.672042   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 908e08ffa4d7423286657d9b08c4a2fb: stopping tablet replica
I20260709 13:00:28.697367   431 master.cc:584] Master@127.0.107.254:34809 shutdown complete.
4c2058117a3f4f608d6bbd90694cd4ea  4
58c9b0ef0380446991b7be408fe6b011  1
ca1e65a96e454f558f348160072c35d0  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (11203 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:00:28.747877   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:32927
I20260709 13:00:28.749385   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:28.756753  1449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:28.756773  1450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:28.758211   431 server_base.cc:1061] running on GCE node
W20260709 13:00:28.759605  1452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:28.761089   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:28.761317   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:28.761528   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602028761508 us; error 0 us; skew 500 ppm
I20260709 13:00:28.762181   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:28.765301   431 webserver.cc:533] Webserver started at http://127.0.107.254:43215/ using document root <none> and password file <none>
I20260709 13:00:28.765899   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:28.766089   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:28.766455   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:28.768229   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "d61e0c3f41244926bf144dca0f9acf61"
format_stamp: "Formatted at 2026-07-09 13:00:28 on dist-test-slave-ftjc"
I20260709 13:00:28.774578   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:00:28.779043  1457 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:28.780489   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:28.780823   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "d61e0c3f41244926bf144dca0f9acf61"
format_stamp: "Formatted at 2026-07-09 13:00:28 on dist-test-slave-ftjc"
I20260709 13:00:28.781122   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:28.813097   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:28.814839   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:28.870623   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:32927
I20260709 13:00:28.870822  1508 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:32927 every 8 connection(s)
I20260709 13:00:28.875892  1509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:28.892006  1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Bootstrap starting.
I20260709 13:00:28.898321  1509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:28.904210  1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: No bootstrap required, opened a new log
I20260709 13:00:28.907063  1509 raft_consensus.cc:359] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61e0c3f41244926bf144dca0f9acf61" member_type: VOTER }
I20260709 13:00:28.907580  1509 raft_consensus.cc:385] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:28.907761  1509 raft_consensus.cc:740] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61e0c3f41244926bf144dca0f9acf61, State: Initialized, Role: FOLLOWER
I20260709 13:00:28.908561  1509 consensus_queue.cc:260] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61e0c3f41244926bf144dca0f9acf61" member_type: VOTER }
I20260709 13:00:28.909054  1509 raft_consensus.cc:399] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:28.909277  1509 raft_consensus.cc:493] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:28.909514  1509 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:28.916199  1509 raft_consensus.cc:515] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61e0c3f41244926bf144dca0f9acf61" member_type: VOTER }
I20260709 13:00:28.917331  1509 leader_election.cc:304] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d61e0c3f41244926bf144dca0f9acf61; no voters: 
I20260709 13:00:28.918931  1509 leader_election.cc:290] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:28.919514  1512 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:28.921120  1512 raft_consensus.cc:697] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 1 LEADER]: Becoming Leader. State: Replica: d61e0c3f41244926bf144dca0f9acf61, State: Running, Role: LEADER
I20260709 13:00:28.922101  1512 consensus_queue.cc:237] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61e0c3f41244926bf144dca0f9acf61" member_type: VOTER }
I20260709 13:00:28.922773  1509 sys_catalog.cc:565] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:28.927511  1513 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d61e0c3f41244926bf144dca0f9acf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61e0c3f41244926bf144dca0f9acf61" member_type: VOTER } }
I20260709 13:00:28.927862  1514 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d61e0c3f41244926bf144dca0f9acf61. Latest consensus state: current_term: 1 leader_uuid: "d61e0c3f41244926bf144dca0f9acf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61e0c3f41244926bf144dca0f9acf61" member_type: VOTER } }
I20260709 13:00:28.928323  1513 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:28.928404  1514 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:28.932226  1519 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:28.939759  1519 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:28.945008   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:28.952003  1519 catalog_manager.cc:1383] Generated new cluster ID: 28d478a65eec43ac970589e95d2e1719
I20260709 13:00:28.952428  1519 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:28.971905  1519 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:28.973577  1519 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:28.999614  1519 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Generated new TSK 0
I20260709 13:00:29.000483  1519 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:29.013746   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:29.021327  1530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.022677  1531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.024317  1533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.024461   431 server_base.cc:1061] running on GCE node
I20260709 13:00:29.025671   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:29.025903   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:29.026168   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602029026143 us; error 0 us; skew 500 ppm
I20260709 13:00:29.026849   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:29.030087   431 webserver.cc:533] Webserver started at http://127.0.107.193:39837/ using document root <none> and password file <none>
I20260709 13:00:29.030714   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:29.030947   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:29.031303   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:29.032789   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "4f737473248e4c059fa74af3b8e1f77e"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-ftjc"
I20260709 13:00:29.039563   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:29.044220  1538 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.045290   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:29.045702   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "4f737473248e4c059fa74af3b8e1f77e"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-ftjc"
I20260709 13:00:29.046108   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:29.063017   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:29.064848   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:29.066649   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:29.069957   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:29.070325   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.070621   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:29.070837   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.130407   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:45275
I20260709 13:00:29.130681  1600 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:45275 every 8 connection(s)
I20260709 13:00:29.136080   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:29.146046  1605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.147743  1606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.153065   431 server_base.cc:1061] running on GCE node
W20260709 13:00:29.153128  1608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.154780   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:29.155087   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:29.155323   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602029155297 us; error 0 us; skew 500 ppm
I20260709 13:00:29.156067   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:29.158372  1601 heartbeater.cc:344] Connected to a master server at 127.0.107.254:32927
I20260709 13:00:29.158931  1601 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:29.159924  1601 heartbeater.cc:507] Master 127.0.107.254:32927 requested a full tablet report, sending...
I20260709 13:00:29.160655   431 webserver.cc:533] Webserver started at http://127.0.107.194:34251/ using document root <none> and password file <none>
I20260709 13:00:29.161513   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:29.161748   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:29.162104   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:29.163350  1474 ts_manager.cc:194] Registered new tserver with Master: 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:29.164023   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "cd6d377efe514bc58ae49c3f4a107232"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-ftjc"
I20260709 13:00:29.166407  1474 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41284
I20260709 13:00:29.171523   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:00:29.177016  1613 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.178153   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:00:29.178596   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "cd6d377efe514bc58ae49c3f4a107232"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-ftjc"
I20260709 13:00:29.178931   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:29.202346   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:29.203990   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:29.205824   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:29.208940   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:29.209229   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.209538   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:29.209746   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.263877   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43325
I20260709 13:00:29.264039  1675 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43325 every 8 connection(s)
I20260709 13:00:29.269991   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:29.278247  1679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.280108  1680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.281733   431 server_base.cc:1061] running on GCE node
W20260709 13:00:29.284179  1683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.285795   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:29.286154   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:29.286372   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602029286352 us; error 0 us; skew 500 ppm
I20260709 13:00:29.286931  1676 heartbeater.cc:344] Connected to a master server at 127.0.107.254:32927
I20260709 13:00:29.287189   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:29.287364  1676 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:29.288400  1676 heartbeater.cc:507] Master 127.0.107.254:32927 requested a full tablet report, sending...
I20260709 13:00:29.290720   431 webserver.cc:533] Webserver started at http://127.0.107.195:40723/ using document root <none> and password file <none>
I20260709 13:00:29.291651   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:29.291803  1474 ts_manager.cc:194] Registered new tserver with Master: cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:29.291937   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:29.293156   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:29.294324  1474 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41288
I20260709 13:00:29.295276   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "58cdb682fbf54792a702a7a455f265a2"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-ftjc"
I20260709 13:00:29.301378   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260709 13:00:29.306602  1687 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.307967   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:00:29.308413   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "58cdb682fbf54792a702a7a455f265a2"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-ftjc"
I20260709 13:00:29.308789   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:29.342860   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:29.344907   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:29.346889   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:29.350153   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:29.350656   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:00:29.351162   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:29.351384   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.418824   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:44281
I20260709 13:00:29.418987  1749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:44281 every 8 connection(s)
I20260709 13:00:29.438649  1750 heartbeater.cc:344] Connected to a master server at 127.0.107.254:32927
I20260709 13:00:29.439185  1750 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:29.440441  1750 heartbeater.cc:507] Master 127.0.107.254:32927 requested a full tablet report, sending...
I20260709 13:00:29.443260  1474 ts_manager.cc:194] Registered new tserver with Master: 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:29.444216   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020811503s
I20260709 13:00:29.445298  1474 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41296
I20260709 13:00:29.479538  1474 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41300:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:29.487331  1474 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:29.727896  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 2c501b3d8f86404e991cec1222bd0825 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:29.730041  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c501b3d8f86404e991cec1222bd0825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.730147  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 62836cb7c4a548b5a6c44c89782a958c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:29.731905  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62836cb7c4a548b5a6c44c89782a958c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.731753  1565 tablet_service.cc:1511] Processing CreateTablet for tablet 7778fc5cf1c746c4a4a974883a271ce0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:29.733479  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7778fc5cf1c746c4a4a974883a271ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.738513  1564 tablet_service.cc:1511] Processing CreateTablet for tablet 53c9e58595214defa013274c5193de1a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:29.740504  1564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c9e58595214defa013274c5193de1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.745579  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 2c501b3d8f86404e991cec1222bd0825 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:29.747325  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c501b3d8f86404e991cec1222bd0825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.760669  1640 tablet_service.cc:1511] Processing CreateTablet for tablet 7778fc5cf1c746c4a4a974883a271ce0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:29.762591  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7778fc5cf1c746c4a4a974883a271ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.763288  1562 tablet_service.cc:1511] Processing CreateTablet for tablet d5ac82307bd14706ab03f881a279f3e5 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:29.767499  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ac82307bd14706ab03f881a279f3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.768800  1639 tablet_service.cc:1511] Processing CreateTablet for tablet 53c9e58595214defa013274c5193de1a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:29.770612  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c9e58595214defa013274c5193de1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.774214  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 42b29e0e5e884027ab75497ae8e020bf (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:29.779209  1768 tablet_bootstrap.cc:492] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:29.778999  1564 tablet_service.cc:1511] Processing CreateTablet for tablet 27e7f73e79344731aa22e748b700ffd9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:29.777519  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42b29e0e5e884027ab75497ae8e020bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.779829  1560 tablet_service.cc:1511] Processing CreateTablet for tablet dde38eb3bd9341aa90d41a6f21bb29a1 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:29.793965  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde38eb3bd9341aa90d41a6f21bb29a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.796093  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 5938eec8e53f4d08b26ec3011f1cdbfc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:29.798346  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5938eec8e53f4d08b26ec3011f1cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.784667  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 57592deb11fc4830921f01c2a3c32c4a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:29.801247  1564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e7f73e79344731aa22e748b700ffd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.803496  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57592deb11fc4830921f01c2a3c32c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.804723  1557 tablet_service.cc:1511] Processing CreateTablet for tablet fb93f4d4caa2474bb5d42f9940f0dcdc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:29.807534  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb93f4d4caa2474bb5d42f9940f0dcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.814330  1638 tablet_service.cc:1511] Processing CreateTablet for tablet 62836cb7c4a548b5a6c44c89782a958c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:29.816777  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62836cb7c4a548b5a6c44c89782a958c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.817505  1566 tablet_service.cc:1511] Processing CreateTablet for tablet f6d77c909fe243b197ef61ea6b271908 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:29.822211  1637 tablet_service.cc:1511] Processing CreateTablet for tablet d5ac82307bd14706ab03f881a279f3e5 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:29.825538  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ac82307bd14706ab03f881a279f3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.825947  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 27e7f73e79344731aa22e748b700ffd9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:29.827623  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e7f73e79344731aa22e748b700ffd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.836984  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d77c909fe243b197ef61ea6b271908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.838486  1564 tablet_service.cc:1511] Processing CreateTablet for tablet 1d22dc39228b4001961709e74ef6bc9c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:29.838768  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 42b29e0e5e884027ab75497ae8e020bf (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:29.840514  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42b29e0e5e884027ab75497ae8e020bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.841403  1768 tablet_bootstrap.cc:654] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.842000  1634 tablet_service.cc:1511] Processing CreateTablet for tablet dde38eb3bd9341aa90d41a6f21bb29a1 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:29.843653  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde38eb3bd9341aa90d41a6f21bb29a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.849248  1564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d22dc39228b4001961709e74ef6bc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.866643  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 2c501b3d8f86404e991cec1222bd0825 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:29.866930  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 57592deb11fc4830921f01c2a3c32c4a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:29.868624  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57592deb11fc4830921f01c2a3c32c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.849846  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 6bd1d3ea2c984a999cdd42423ebec16f (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:29.856781  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 57592deb11fc4830921f01c2a3c32c4a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:29.869139  1713 tablet_service.cc:1511] Processing CreateTablet for tablet 53c9e58595214defa013274c5193de1a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:29.870362  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57592deb11fc4830921f01c2a3c32c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.871058  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c9e58595214defa013274c5193de1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.858078  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 62836cb7c4a548b5a6c44c89782a958c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:29.861951  1710 tablet_service.cc:1511] Processing CreateTablet for tablet 27e7f73e79344731aa22e748b700ffd9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:29.864162  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 42b29e0e5e884027ab75497ae8e020bf (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:29.859863  1708 tablet_service.cc:1511] Processing CreateTablet for tablet dde38eb3bd9341aa90d41a6f21bb29a1 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:29.859858  1711 tablet_service.cc:1511] Processing CreateTablet for tablet d5ac82307bd14706ab03f881a279f3e5 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:29.865422  1714 tablet_service.cc:1511] Processing CreateTablet for tablet 7778fc5cf1c746c4a4a974883a271ce0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:29.873478  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c501b3d8f86404e991cec1222bd0825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.879230  1632 tablet_service.cc:1511] Processing CreateTablet for tablet f6d77c909fe243b197ef61ea6b271908 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:29.881074  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde38eb3bd9341aa90d41a6f21bb29a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.881989  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ac82307bd14706ab03f881a279f3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.883932  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62836cb7c4a548b5a6c44c89782a958c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.884744  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e7f73e79344731aa22e748b700ffd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.885908  1706 tablet_service.cc:1511] Processing CreateTablet for tablet f6d77c909fe243b197ef61ea6b271908 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:29.887701  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d77c909fe243b197ef61ea6b271908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.888943  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42b29e0e5e884027ab75497ae8e020bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.893787  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d77c909fe243b197ef61ea6b271908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.895350  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7778fc5cf1c746c4a4a974883a271ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.900434  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 44cb1d5977df435cac936d86c5bd1880 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:29.902256  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44cb1d5977df435cac936d86c5bd1880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.924829  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd1d3ea2c984a999cdd42423ebec16f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:29.942679  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:29.957809  1565 tablet_service.cc:1511] Processing CreateTablet for tablet 81bce6c3bf1547e4b8d33586cafe5f6c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:29.983987  1769 tablet_bootstrap.cc:492] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:29.985050  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bce6c3bf1547e4b8d33586cafe5f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.015842  1769 tablet_bootstrap.cc:654] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.016849  1637 tablet_service.cc:1511] Processing CreateTablet for tablet fb93f4d4caa2474bb5d42f9940f0dcdc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:30.016958  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 5938eec8e53f4d08b26ec3011f1cdbfc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:30.018558  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5938eec8e53f4d08b26ec3011f1cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.021855  1559 tablet_service.cc:1511] Processing CreateTablet for tablet fac0a563dc584b4fbe82f01f362fa966 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:30.037010  1768 tablet_bootstrap.cc:492] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:30.037675  1768 ts_tablet_manager.cc:1403] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.259s	user 0.008s	sys 0.015s
I20260709 13:00:30.044763  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0a563dc584b4fbe82f01f362fa966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.045768  1558 tablet_service.cc:1511] Processing CreateTablet for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:30.047425  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b837a5ec9a0d4f2a8635dedf45d51aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.068876  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 5938eec8e53f4d08b26ec3011f1cdbfc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:30.070799  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5938eec8e53f4d08b26ec3011f1cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.073625  1768 raft_consensus.cc:359] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.074612  1768 raft_consensus.cc:385] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.074960  1768 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.080533  1639 tablet_service.cc:1511] Processing CreateTablet for tablet 6bd1d3ea2c984a999cdd42423ebec16f (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:30.082185  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd1d3ea2c984a999cdd42423ebec16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.072110  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 1d22dc39228b4001961709e74ef6bc9c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:30.083386  1708 tablet_service.cc:1511] Processing CreateTablet for tablet fb93f4d4caa2474bb5d42f9940f0dcdc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:30.085317  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb93f4d4caa2474bb5d42f9940f0dcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.092700  1768 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.094425  1557 tablet_service.cc:1511] Processing CreateTablet for tablet f048890019da43548110ab19fce7a878 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:30.094475  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d22dc39228b4001961709e74ef6bc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.098258  1640 tablet_service.cc:1511] Processing CreateTablet for tablet fac0a563dc584b4fbe82f01f362fa966 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:30.104626  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 1d22dc39228b4001961709e74ef6bc9c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:30.119786  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 6bd1d3ea2c984a999cdd42423ebec16f (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:30.121842  1713 tablet_service.cc:1511] Processing CreateTablet for tablet 81bce6c3bf1547e4b8d33586cafe5f6c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:30.100181  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0a563dc584b4fbe82f01f362fa966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.125042  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f048890019da43548110ab19fce7a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.107745  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 01c1afc8c26047b0b3d8c006f9df5a30 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:30.128837  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 44cb1d5977df435cac936d86c5bd1880 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:30.108160  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 44cb1d5977df435cac936d86c5bd1880 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:30.139675  1638 tablet_service.cc:1511] Processing CreateTablet for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:30.148543  1771 tablet_bootstrap.cc:492] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:30.155691  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 08c9dfcb3286481dbbb396ccf74ada24 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:30.161741  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c1afc8c26047b0b3d8c006f9df5a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.163246  1715 tablet_service.cc:1511] Processing CreateTablet for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:30.169873  1769 tablet_bootstrap.cc:492] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:30.170384  1601 heartbeater.cc:499] Master 127.0.107.254:32927 was elected leader, sending a full tablet report...
I20260709 13:00:30.170559  1769 ts_tablet_manager.cc:1403] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.187s	user 0.021s	sys 0.018s
I20260709 13:00:30.176586  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c9dfcb3286481dbbb396ccf74ada24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.134548  1706 tablet_service.cc:1511] Processing CreateTablet for tablet fac0a563dc584b4fbe82f01f362fa966 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:30.182910  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44cb1d5977df435cac936d86c5bd1880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.188926  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b837a5ec9a0d4f2a8635dedf45d51aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.187633  1632 tablet_service.cc:1511] Processing CreateTablet for tablet f048890019da43548110ab19fce7a878 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:30.127743  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 81bce6c3bf1547e4b8d33586cafe5f6c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:30.194589  1769 raft_consensus.cc:359] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.140656  1560 tablet_service.cc:1511] Processing CreateTablet for tablet c55c47030a294c17ad5561abe541b993 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:30.202131  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 414132da4f974e5c912e5d5dfa077654 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:30.202874  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb93f4d4caa2474bb5d42f9940f0dcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.211588  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d22dc39228b4001961709e74ef6bc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.215025  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd1d3ea2c984a999cdd42423ebec16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.217700  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44cb1d5977df435cac936d86c5bd1880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.218686  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b837a5ec9a0d4f2a8635dedf45d51aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.205771  1769 raft_consensus.cc:385] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.227677  1768 ts_tablet_manager.cc:1434] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.190s	user 0.029s	sys 0.008s
I20260709 13:00:30.232376  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0a563dc584b4fbe82f01f362fa966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.232409  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bce6c3bf1547e4b8d33586cafe5f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.228009  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 01c1afc8c26047b0b3d8c006f9df5a30 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:30.228937  1769 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.242167  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414132da4f974e5c912e5d5dfa077654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.250869  1768 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:30.254106  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bce6c3bf1547e4b8d33586cafe5f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.260634  1565 tablet_service.cc:1511] Processing CreateTablet for tablet 54bb54b92d594aa7a2d8f8402e20c0ca (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:30.263958  1641 tablet_service.cc:1511] Processing CreateTablet for tablet c55c47030a294c17ad5561abe541b993 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:30.264300  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 01c1afc8c26047b0b3d8c006f9df5a30 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:30.260712  1564 tablet_service.cc:1511] Processing CreateTablet for tablet 48efc21b583b4e7fa81ec24726402281 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:30.265928  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c1afc8c26047b0b3d8c006f9df5a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.267431  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 54db7417c3014ba3948e6ce733254ba3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:30.281148  1566 tablet_service.cc:1511] Processing CreateTablet for tablet e69630519d3c48eca350b04183abca32 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:30.281795  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c55c47030a294c17ad5561abe541b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.292625  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69630519d3c48eca350b04183abca32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.293475  1771 tablet_bootstrap.cc:654] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.284588  1769 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.277652  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f048890019da43548110ab19fce7a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.296134  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54bb54b92d594aa7a2d8f8402e20c0ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.298118  1676 heartbeater.cc:499] Master 127.0.107.254:32927 was elected leader, sending a full tablet report...
I20260709 13:00:30.225170  1714 tablet_service.cc:1511] Processing CreateTablet for tablet f048890019da43548110ab19fce7a878 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:30.304632  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54db7417c3014ba3948e6ce733254ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.305136  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f048890019da43548110ab19fce7a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.309131  1564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48efc21b583b4e7fa81ec24726402281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.314143  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 08c9dfcb3286481dbbb396ccf74ada24 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:30.316140  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c55c47030a294c17ad5561abe541b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.320930  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c1afc8c26047b0b3d8c006f9df5a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.289263  1710 tablet_service.cc:1511] Processing CreateTablet for tablet c55c47030a294c17ad5561abe541b993 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:30.329546  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c55c47030a294c17ad5561abe541b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.342646  1557 tablet_service.cc:1511] Processing CreateTablet for tablet fd4d3a0e164248a5be4b2d19407551af (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:30.350569  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c9dfcb3286481dbbb396ccf74ada24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.369128  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd4d3a0e164248a5be4b2d19407551af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.374789  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 26917aa1f55f48918fedd4960446f38c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:30.376402  1769 ts_tablet_manager.cc:1434] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.205s	user 0.012s	sys 0.007s
I20260709 13:00:30.369022  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 87baee349cdd48d88642bf37176b1b26 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:30.382505  1769 tablet_bootstrap.cc:492] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:30.397603  1771 tablet_bootstrap.cc:492] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:30.398233  1771 ts_tablet_manager.cc:1403] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.250s	user 0.009s	sys 0.009s
I20260709 13:00:30.405072  1713 tablet_service.cc:1511] Processing CreateTablet for tablet 414132da4f974e5c912e5d5dfa077654 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:30.435051  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414132da4f974e5c912e5d5dfa077654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.435206  1769 tablet_bootstrap.cc:654] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.438071  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 48efc21b583b4e7fa81ec24726402281 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:30.457804  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48efc21b583b4e7fa81ec24726402281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.453749  1750 heartbeater.cc:499] Master 127.0.107.254:32927 was elected leader, sending a full tablet report...
I20260709 13:00:30.463694  1768 tablet_bootstrap.cc:654] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.443297  1639 tablet_service.cc:1511] Processing CreateTablet for tablet 414132da4f974e5c912e5d5dfa077654 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:30.417366  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26917aa1f55f48918fedd4960446f38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.409159  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 48efc21b583b4e7fa81ec24726402281 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:30.407989  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 9c347beda1704867ba2582a7a34d255b (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:30.462311  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 54db7417c3014ba3948e6ce733254ba3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:30.458966  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 54bb54b92d594aa7a2d8f8402e20c0ca (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:30.468745  1774 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.400297  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 08c9dfcb3286481dbbb396ccf74ada24 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:30.401816  1771 raft_consensus.cc:359] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.479984  1771 raft_consensus.cc:385] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.480417  1771 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.481511  1771 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.474400  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87baee349cdd48d88642bf37176b1b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.474601  1773 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.485484  1773 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.488812  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c347beda1704867ba2582a7a34d255b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.475776  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48efc21b583b4e7fa81ec24726402281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.480870  1774 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.488911  1773 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:30.487686  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 740ac262ab184918992607d66446b8f3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:30.506191  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740ac262ab184918992607d66446b8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.508929  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54bb54b92d594aa7a2d8f8402e20c0ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.511687  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54db7417c3014ba3948e6ce733254ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.493795  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 54db7417c3014ba3948e6ce733254ba3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:30.533370  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c9dfcb3286481dbbb396ccf74ada24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.523604  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414132da4f974e5c912e5d5dfa077654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.518462  1640 tablet_service.cc:1511] Processing CreateTablet for tablet 54bb54b92d594aa7a2d8f8402e20c0ca (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:30.566725  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54db7417c3014ba3948e6ce733254ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.542621  1713 tablet_service.cc:1511] Processing CreateTablet for tablet e69630519d3c48eca350b04183abca32 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:30.574033  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69630519d3c48eca350b04183abca32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.582223  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 87baee349cdd48d88642bf37176b1b26 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:30.589010  1634 tablet_service.cc:1511] Processing CreateTablet for tablet e69630519d3c48eca350b04183abca32 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:30.597618  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54bb54b92d594aa7a2d8f8402e20c0ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.633212  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 87baee349cdd48d88642bf37176b1b26 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:30.636718  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87baee349cdd48d88642bf37176b1b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.604918  1774 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:30.655251  1771 ts_tablet_manager.cc:1434] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.257s	user 0.016s	sys 0.007s
I20260709 13:00:30.656224  1769 tablet_bootstrap.cc:492] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:30.670437  1769 ts_tablet_manager.cc:1403] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.293s	user 0.015s	sys 0.009s
I20260709 13:00:30.690172  1769 raft_consensus.cc:359] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.691124  1769 raft_consensus.cc:385] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.691488  1769 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.693038  1771 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:30.701860  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 26917aa1f55f48918fedd4960446f38c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:30.704725  1714 tablet_service.cc:1511] Processing CreateTablet for tablet fd4d3a0e164248a5be4b2d19407551af (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:30.720974  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87baee349cdd48d88642bf37176b1b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.713529  1769 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.737507  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 26917aa1f55f48918fedd4960446f38c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:30.738775  1769 ts_tablet_manager.cc:1434] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.002s
I20260709 13:00:30.740746  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69630519d3c48eca350b04183abca32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.747068  1768 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:30.746904  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 38bef56447d743db91bd1f59274db995 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:30.747764  1768 ts_tablet_manager.cc:1403] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.497s	user 0.009s	sys 0.013s
I20260709 13:00:30.749814  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 30619dafd05d4e36a03ff8f500c3a38c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:30.731788  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 9c347beda1704867ba2582a7a34d255b (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:30.753726  1564 tablet_service.cc:1511] Processing CreateTablet for tablet 8014e1ef10d84a9f9c4d676b56c5105d (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:30.755541  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26917aa1f55f48918fedd4960446f38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.751889  1768 raft_consensus.cc:359] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.757172  1768 raft_consensus.cc:385] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.757768  1768 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.759379  1768 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.767215  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bef56447d743db91bd1f59274db995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.754226  1638 tablet_service.cc:1511] Processing CreateTablet for tablet fd4d3a0e164248a5be4b2d19407551af (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:30.782917  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd4d3a0e164248a5be4b2d19407551af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.782757  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 740ac262ab184918992607d66446b8f3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:30.789314  1564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8014e1ef10d84a9f9c4d676b56c5105d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.784615  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c347beda1704867ba2582a7a34d255b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.791661  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30619dafd05d4e36a03ff8f500c3a38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.792759  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26917aa1f55f48918fedd4960446f38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.786377  1769 tablet_bootstrap.cc:492] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:30.796885  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740ac262ab184918992607d66446b8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.797497  1771 tablet_bootstrap.cc:654] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.799219  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd4d3a0e164248a5be4b2d19407551af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.811652  1710 tablet_service.cc:1511] Processing CreateTablet for tablet 9c347beda1704867ba2582a7a34d255b (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:30.813158  1769 tablet_bootstrap.cc:654] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.813886  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c347beda1704867ba2582a7a34d255b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.838519  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:30.839442  1576 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:30.841300  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:30.842567  1774 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.843016  1774 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.843470  1774 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.848785  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:30.834714  1768 ts_tablet_manager.cc:1434] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.003s
I20260709 13:00:30.858418  1768 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:30.858771  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 38bef56447d743db91bd1f59274db995 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:30.863745  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 9273554b0f40499d928621ee1bccc9d8 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:30.873624  1774 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.875268  1651 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:30.877622  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9273554b0f40499d928621ee1bccc9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.864933  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bef56447d743db91bd1f59274db995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.885937  1774 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:30.893348  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:30.900825  1576 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.917627  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 8014e1ef10d84a9f9c4d676b56c5105d (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:30.919317  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8014e1ef10d84a9f9c4d676b56c5105d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.924446  1558 tablet_service.cc:1511] Processing CreateTablet for tablet d7e1643d2c2f4a96a57a8bae29e7f19a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:30.925997  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e1643d2c2f4a96a57a8bae29e7f19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.927498  1713 tablet_service.cc:1511] Processing CreateTablet for tablet 30619dafd05d4e36a03ff8f500c3a38c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:30.929055  1771 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:30.929738  1771 ts_tablet_manager.cc:1403] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.237s	user 0.016s	sys 0.005s
I20260709 13:00:30.929386  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30619dafd05d4e36a03ff8f500c3a38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.936285  1771 raft_consensus.cc:359] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.937278  1771 raft_consensus.cc:385] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.937016  1561 tablet_service.cc:1511] Processing CreateTablet for tablet 1be748f6ba29434fbee8b14cedc1d192 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:30.937636  1771 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.938638  1561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1be748f6ba29434fbee8b14cedc1d192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.939184  1778 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.939572  1711 tablet_service.cc:1511] Processing CreateTablet for tablet 740ac262ab184918992607d66446b8f3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:30.939846  1778 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.941468  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740ac262ab184918992607d66446b8f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:30.946139  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:30.947549  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 9273554b0f40499d928621ee1bccc9d8 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:30.950268  1778 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:30.958741  1771 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.966902  1771 ts_tablet_manager.cc:1434] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.004s
I20260709 13:00:30.976086  1715 tablet_service.cc:1511] Processing CreateTablet for tablet d7e1643d2c2f4a96a57a8bae29e7f19a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:30.976981  1576 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:30.977764  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:30.978639  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:30.978865  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:30.979907  1774 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.980195  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:30.980525  1774 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:30.981592  1773 raft_consensus.cc:2764] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:00:30.981539  1774 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:30.968154  1771 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:30.952032  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9273554b0f40499d928621ee1bccc9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:30.999382  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 38bef56447d743db91bd1f59274db995 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:31.001148  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bef56447d743db91bd1f59274db995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.001771  1769 tablet_bootstrap.cc:492] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:31.002346  1769 ts_tablet_manager.cc:1403] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.216s	user 0.021s	sys 0.014s
I20260709 13:00:31.002461  1778 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.003026  1778 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.005545  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 7011d2943a2040ccb1a34122c0fd3ac3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:31.007197  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7011d2943a2040ccb1a34122c0fd3ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.007496  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 30619dafd05d4e36a03ff8f500c3a38c (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:31.008496  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 1acf73e529394004842219637cff7cfc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:31.008929  1778 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:31.009002  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30619dafd05d4e36a03ff8f500c3a38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.009913  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1acf73e529394004842219637cff7cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.010059  1769 raft_consensus.cc:359] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.010881  1769 raft_consensus.cc:385] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:31.011462  1773 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.011950  1773 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.014068  1710 tablet_service.cc:1511] Processing CreateTablet for tablet 7011d2943a2040ccb1a34122c0fd3ac3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:31.016151  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 1be748f6ba29434fbee8b14cedc1d192 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:31.019587  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 8014e1ef10d84a9f9c4d676b56c5105d (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:31.024959  1769 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:31.027704  1776 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.028285  1776 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.030946  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e1643d2c2f4a96a57a8bae29e7f19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.035349  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1be748f6ba29434fbee8b14cedc1d192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.053665  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:31.054294  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:31.054618  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:31.059705  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8014e1ef10d84a9f9c4d676b56c5105d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.065294  1773 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:31.065459  1639 tablet_service.cc:1511] Processing CreateTablet for tablet d7e1643d2c2f4a96a57a8bae29e7f19a (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:31.066148  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:31.066987  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e1643d2c2f4a96a57a8bae29e7f19a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:31.068511  1539 leader_election.cc:343] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.049309  1769 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.070914  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7011d2943a2040ccb1a34122c0fd3ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.030200  1640 tablet_service.cc:1511] Processing CreateTablet for tablet 9273554b0f40499d928621ee1bccc9d8 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:31.096982  1792 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.097672  1792 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.104980  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 996f59fd8f2340a49403b3e9ef4c8e86 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:31.105614  1565 tablet_service.cc:1511] Processing CreateTablet for tablet e8efe0ac5e5e45d8acfa1dfe55172859 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:31.107157  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 996f59fd8f2340a49403b3e9ef4c8e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.107851  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8efe0ac5e5e45d8acfa1dfe55172859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.083302  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:31.112435  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 20085fd5893f43378bf1c59b14c351c0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:31.114562  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20085fd5893f43378bf1c59b14c351c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.118984  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:31.119443  1539 leader_election.cc:343] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.119908  1539 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:00:31.120639  1616 leader_election.cc:343] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.121142  1773 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.122253  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:31.124456  1614 leader_election.cc:343] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.124219  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 1be748f6ba29434fbee8b14cedc1d192 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:31.124943  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:31.113000  1776 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:31.126685  1776 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.128175  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:31.130651  1690 leader_election.cc:343] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.131395  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 7011d2943a2040ccb1a34122c0fd3ac3 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:31.131772  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:31.132879  1576 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:31.125955  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1be748f6ba29434fbee8b14cedc1d192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.137984  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9273554b0f40499d928621ee1bccc9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.160929  1768 tablet_bootstrap.cc:654] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.164369  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7011d2943a2040ccb1a34122c0fd3ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.173193  1714 tablet_service.cc:1511] Processing CreateTablet for tablet 1acf73e529394004842219637cff7cfc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:31.175845  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1acf73e529394004842219637cff7cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.178275  1771 tablet_bootstrap.cc:654] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.232089  1769 ts_tablet_manager.cc:1434] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.229s	user 0.011s	sys 0.008s
I20260709 13:00:31.233515  1769 tablet_bootstrap.cc:492] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:31.256552  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 1acf73e529394004842219637cff7cfc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:31.273907  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:31.277849  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 9bb9de7d85a2414bb920f63ec26f8686 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:00:31.279022  1616 leader_election.cc:343] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.279491  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb9de7d85a2414bb920f63ec26f8686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.280296  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:31.282037  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:31.288810  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:31.305014  1778 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:31.305274  1688 leader_election.cc:343] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.305863  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:31.307078  1792 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:31.309808  1564 tablet_service.cc:1511] Processing CreateTablet for tablet 78d12cc2bc7b4654acd41e6c45eb9753 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:31.311750  1564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d12cc2bc7b4654acd41e6c45eb9753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.313344  1563 tablet_service.cc:1511] Processing CreateTablet for tablet b8513b00817042619ae0bf0c8da58904 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:31.314730  1641 tablet_service.cc:1511] Processing CreateTablet for tablet 9bb9de7d85a2414bb920f63ec26f8686 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:31.315457  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8513b00817042619ae0bf0c8da58904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.303723  1473 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.331691  1778 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.337600  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1acf73e529394004842219637cff7cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.349895  1639 tablet_service.cc:1511] Processing CreateTablet for tablet 996f59fd8f2340a49403b3e9ef4c8e86 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:31.355706  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 9bb9de7d85a2414bb920f63ec26f8686 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:31.364434  1768 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:31.365232  1768 ts_tablet_manager.cc:1403] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.507s	user 0.020s	sys 0.017s
I20260709 13:00:31.366226  1641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb9de7d85a2414bb920f63ec26f8686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.371683  1768 raft_consensus.cc:359] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:31.373723  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 151611us
I20260709 13:00:31.376762  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 996f59fd8f2340a49403b3e9ef4c8e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.341432  1778 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.346907  1769 tablet_bootstrap.cc:654] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.393224  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb9de7d85a2414bb920f63ec26f8686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.385177  1768 raft_consensus.cc:385] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:31.399387  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94161.5us
I20260709 13:00:31.401978  1565 tablet_service.cc:1511] Processing CreateTablet for tablet 4d5536a0ca5d48e4b98e0084e0aefb70 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:31.403645  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d5536a0ca5d48e4b98e0084e0aefb70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.412431  1638 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49746 (request call id 28) took 1091 ms. Trace:
I20260709 13:00:31.413264  1638 rpcz_store.cc:276] 0709 13:00:30.321286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.613363 (+292077us) service_pool.cc:224] Handling call
0709 13:00:30.788736 (+175373us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:00:30.788844 (+   108us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:00:31.053281 (+264437us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:31.360800 (+307519us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:31.411267 (+ 50467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:31.415568  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:31.417047  1566 tablet_service.cc:1511] Processing CreateTablet for tablet ba20622bf78343c5bf6ac194af441026 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:31.420553  1771 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:31.421196  1771 ts_tablet_manager.cc:1403] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.453s	user 0.024s	sys 0.022s
I20260709 13:00:31.424052  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 20085fd5893f43378bf1c59b14c351c0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:31.424373  1713 tablet_service.cc:1511] Processing CreateTablet for tablet e8efe0ac5e5e45d8acfa1dfe55172859 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:31.424600  1709 tablet_service.cc:1511] Processing CreateTablet for tablet b8513b00817042619ae0bf0c8da58904 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:31.425714  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20085fd5893f43378bf1c59b14c351c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:31.426146  1614 leader_election.cc:343] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.426575  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8513b00817042619ae0bf0c8da58904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.426718  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:31.425186  1771 raft_consensus.cc:359] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.428769  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba20622bf78343c5bf6ac194af441026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.427676  1640 tablet_service.cc:1511] Processing CreateTablet for tablet b8513b00817042619ae0bf0c8da58904 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:31.442425  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8513b00817042619ae0bf0c8da58904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.438243  1637 tablet_service.cc:1511] Processing CreateTablet for tablet 20085fd5893f43378bf1c59b14c351c0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:31.439005  1794 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.429013  1771 raft_consensus.cc:385] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:31.440099  1713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8efe0ac5e5e45d8acfa1dfe55172859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.445302  1637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20085fd5893f43378bf1c59b14c351c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.451340  1557 tablet_service.cc:1511] Processing CreateTablet for tablet c47fb1f3610c496ea3c5542634768fa6 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:31.445108  1771 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:31.453720  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47fb1f3610c496ea3c5542634768fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.399000  1768 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:31.456467  1768 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.459240  1794 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:31.460599  1638 tablet_service.cc:1511] Processing CreateTablet for tablet e8efe0ac5e5e45d8acfa1dfe55172859 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:31.462227  1638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8efe0ac5e5e45d8acfa1dfe55172859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.459245  1768 ts_tablet_manager.cc:1434] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.000s
I20260709 13:00:31.466588  1768 tablet_bootstrap.cc:492] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:31.472720  1771 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.473697  1635 tablet_service.cc:1511] Processing CreateTablet for tablet ba20622bf78343c5bf6ac194af441026 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:31.475594  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba20622bf78343c5bf6ac194af441026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.476648  1771 ts_tablet_manager.cc:1434] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.001s
I20260709 13:00:31.487876  1768 tablet_bootstrap.cc:654] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.493127  1771 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:31.503273  1636 tablet_service.cc:1511] Processing CreateTablet for tablet 78d12cc2bc7b4654acd41e6c45eb9753 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:31.504899  1636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d12cc2bc7b4654acd41e6c45eb9753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.510907  1707 tablet_service.cc:1511] Processing CreateTablet for tablet ba20622bf78343c5bf6ac194af441026 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:31.514097  1714 tablet_service.cc:1511] Processing CreateTablet for tablet 78d12cc2bc7b4654acd41e6c45eb9753 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:31.516875  1797 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.517637  1797 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.477339  1712 tablet_service.cc:1511] Processing CreateTablet for tablet 996f59fd8f2340a49403b3e9ef4c8e86 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:31.527544  1710 tablet_service.cc:1511] Processing CreateTablet for tablet 4d5536a0ca5d48e4b98e0084e0aefb70 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:31.549428  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 996f59fd8f2340a49403b3e9ef4c8e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.561873  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:31.562716  1575 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:31.554812  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d12cc2bc7b4654acd41e6c45eb9753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.551798  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 4d5536a0ca5d48e4b98e0084e0aefb70 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:31.574155  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d5536a0ca5d48e4b98e0084e0aefb70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.574810  1562 tablet_service.cc:1511] Processing CreateTablet for tablet de2b34490a3844168645e5b940ba01a9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:31.576663  1690 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:31.577952  1711 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55254 (request call id 31) took 1022 ms. Trace:
I20260709 13:00:31.576330  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2b34490a3844168645e5b940ba01a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.578296  1778 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:31.579087  1778 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.578541  1711 rpcz_store.cc:276] 0709 13:00:30.555879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.847982 (+292103us) service_pool.cc:224] Handling call
0709 13:00:30.940774 (+ 92792us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:00:30.940916 (+   142us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:00:31.255299 (+314383us) tablet_metadata.cc:665] Metadata flushed
0709 13:00:31.577319 (+322020us) tablet_replica.cc:206] Creating consensus instance
0709 13:00:31.577893 (+   574us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:31.579628  1778 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:31.582513  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba20622bf78343c5bf6ac194af441026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.588876  1797 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:31.592110  1706 tablet_service.cc:1511] Processing CreateTablet for tablet c47fb1f3610c496ea3c5542634768fa6 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:31.592126  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:31.598295  1688 leader_election.cc:343] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.600696  1794 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:31.608793  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47fb1f3610c496ea3c5542634768fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.612675  1710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d5536a0ca5d48e4b98e0084e0aefb70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.614598  1639 tablet_service.cc:1511] Processing CreateTablet for tablet c47fb1f3610c496ea3c5542634768fa6 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:31.616533  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47fb1f3610c496ea3c5542634768fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.619474  1773 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.620050  1773 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.622546  1773 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:31.624776  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:31.629865  1724 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:31.629781  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:31.640188  1634 tablet_service.cc:1511] Processing CreateTablet for tablet de2b34490a3844168645e5b940ba01a9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:00:31.643134  1539 leader_election.cc:343] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.643810  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2b34490a3844168645e5b940ba01a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.651160  1769 tablet_bootstrap.cc:492] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:31.652683  1769 ts_tablet_manager.cc:1403] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.419s	user 0.018s	sys 0.024s
I20260709 13:00:31.653200  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:31.654652  1773 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:31.655133  1773 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.655548  1773 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:31.661026  1711 tablet_service.cc:1511] Processing CreateTablet for tablet de2b34490a3844168645e5b940ba01a9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:31.663575  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2b34490a3844168645e5b940ba01a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.667096  1773 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.667469  1557 tablet_service.cc:1511] Processing CreateTablet for tablet f3646214b1a14514955e904e05fadcf1 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:31.669170  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3646214b1a14514955e904e05fadcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.675096  1768 tablet_bootstrap.cc:492] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:31.675853  1768 ts_tablet_manager.cc:1403] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.210s	user 0.025s	sys 0.009s
I20260709 13:00:31.655428  1778 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.699036  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:00:31.700764  1539 leader_election.cc:343] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.703665  1632 tablet_service.cc:1511] Processing CreateTablet for tablet f3646214b1a14514955e904e05fadcf1 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:31.705406  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3646214b1a14514955e904e05fadcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.710253  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:31.710472  1773 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:31.711257  1724 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:31.715988  1768 raft_consensus.cc:359] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.717276  1768 raft_consensus.cc:385] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:31.717738  1768 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:31.719043  1768 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.729712  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:31.730499  1575 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:31.749161  1566 tablet_service.cc:1511] Processing CreateTablet for tablet 4391f9a46a9542a7bf726b05393b7609 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:31.757758  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4391f9a46a9542a7bf726b05393b7609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.758258  1724 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:31.761416  1635 tablet_service.cc:1511] Processing CreateTablet for tablet 4391f9a46a9542a7bf726b05393b7609 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:31.763116  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4391f9a46a9542a7bf726b05393b7609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.765151  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:31.766130  1773 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:31.724295  1769 raft_consensus.cc:359] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.776476  1769 raft_consensus.cc:385] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:31.776857  1769 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:31.777871  1769 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.781713  1771 tablet_bootstrap.cc:654] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.783113  1768 ts_tablet_manager.cc:1434] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.107s	user 0.016s	sys 0.004s
I20260709 13:00:31.784534  1768 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:31.787976  1565 tablet_service.cc:1511] Processing CreateTablet for tablet 363173567dd940e99e459f02148a42e0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:31.791436  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363173567dd940e99e459f02148a42e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.793941  1560 tablet_service.cc:1511] Processing CreateTablet for tablet 6834f8da7764493a94dc534e4471b3dc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:31.797024  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 363173567dd940e99e459f02148a42e0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:31.798218  1560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834f8da7764493a94dc534e4471b3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.800838  1778 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:31.802292  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:31.812463  1769 ts_tablet_manager.cc:1434] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.127s	user 0.004s	sys 0.001s
W20260709 13:00:31.827796  1688 leader_election.cc:343] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.837435  1639 tablet_service.cc:1511] Processing CreateTablet for tablet 6834f8da7764493a94dc534e4471b3dc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:31.849997  1773 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:31.851132  1773 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.855911  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 363173567dd940e99e459f02148a42e0 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:31.856885  1575 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:31.858543  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363173567dd940e99e459f02148a42e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.860442  1690 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:31.869339  1769 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:31.870226  1715 tablet_service.cc:1511] Processing CreateTablet for tablet 4391f9a46a9542a7bf726b05393b7609 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:31.871898  1715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4391f9a46a9542a7bf726b05393b7609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.872740  1778 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:31.873345  1778 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:31.874713  1778 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.878002  1714 tablet_service.cc:1511] Processing CreateTablet for tablet 6834f8da7764493a94dc534e4471b3dc (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:31.879703  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834f8da7764493a94dc534e4471b3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.886749  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363173567dd940e99e459f02148a42e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.887586  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834f8da7764493a94dc534e4471b3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.888792  1709 tablet_service.cc:1511] Processing CreateTablet for tablet f3646214b1a14514955e904e05fadcf1 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:31.890431  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3646214b1a14514955e904e05fadcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.902482  1809 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.903117  1809 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:31.909459  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:31.912544  1539 leader_election.cc:343] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.915863  1562 tablet_service.cc:1511] Processing CreateTablet for tablet 93ef45e97cf24863a576b705ec8e07bd (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:31.920387  1562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ef45e97cf24863a576b705ec8e07bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.920872  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:31.925058  1539 leader_election.cc:343] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:31.925969  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:31.927121  1811 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.928331  1809 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:31.935542  1768 tablet_bootstrap.cc:654] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.937153  1769 tablet_bootstrap.cc:654] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:31.971419  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 93ef45e97cf24863a576b705ec8e07bd (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:31.973528  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ef45e97cf24863a576b705ec8e07bd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:31.978945  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:31.979203  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 93ef45e97cf24863a576b705ec8e07bd (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:31.981724  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ef45e97cf24863a576b705ec8e07bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:31.953742  1792 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.993892  1792 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.003026  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.016868  1563 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa278fc63cf4f6e8ca5de6992849e6e (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:32.018225  1792 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:32.020438  1616 leader_election.cc:343] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.020941  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.031562  1563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa278fc63cf4f6e8ca5de6992849e6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:32.022636  1614 leader_election.cc:343] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.037062  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.038817  1792 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.044535  1639 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa278fc63cf4f6e8ca5de6992849e6e (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:32.045190  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa278fc63cf4f6e8ca5de6992849e6e (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:32.046972  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa278fc63cf4f6e8ca5de6992849e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.046207  1639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa278fc63cf4f6e8ca5de6992849e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.046135  1474 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:32.057639  1565 tablet_service.cc:1511] Processing CreateTablet for tablet fb755eb990bc40778167e79183a40b84 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:32.058525  1771 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:32.059326  1771 ts_tablet_manager.cc:1403] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.567s	user 0.028s	sys 0.022s
I20260709 13:00:32.059569  1565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb755eb990bc40778167e79183a40b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.065049  1566 tablet_service.cc:1511] Processing CreateTablet for tablet f9396e14142a4f6cb8733d8b08ac73b9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:32.066922  1566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9396e14142a4f6cb8733d8b08ac73b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.074375  1632 tablet_service.cc:1511] Processing CreateTablet for tablet fb755eb990bc40778167e79183a40b84 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:32.079015  1711 tablet_service.cc:1511] Processing CreateTablet for tablet fb755eb990bc40778167e79183a40b84 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:32.063714  1771 raft_consensus.cc:359] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.081746  1471 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:32.085769  1711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb755eb990bc40778167e79183a40b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.087059  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb755eb990bc40778167e79183a40b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.094303  1557 tablet_service.cc:1511] Processing CreateTablet for tablet da3ba11d664442d7b28bc849c61fdcdf (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:32.108469  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da3ba11d664442d7b28bc849c61fdcdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.110456  1769 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.110253  1714 tablet_service.cc:1511] Processing CreateTablet for tablet f9396e14142a4f6cb8733d8b08ac73b9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:32.123430  1714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9396e14142a4f6cb8733d8b08ac73b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.127643  1712 tablet_service.cc:1511] Processing CreateTablet for tablet da3ba11d664442d7b28bc849c61fdcdf (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:32.128912  1769 ts_tablet_manager.cc:1403] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.260s	user 0.012s	sys 0.029s
I20260709 13:00:32.113328  1635 tablet_service.cc:1511] Processing CreateTablet for tablet f9396e14142a4f6cb8733d8b08ac73b9 (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:32.089851  1771 raft_consensus.cc:385] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.113328  1640 tablet_service.cc:1511] Processing CreateTablet for tablet da3ba11d664442d7b28bc849c61fdcdf (DEFAULT_TABLE table=test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:32.129540  1712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da3ba11d664442d7b28bc849c61fdcdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.129761  1771 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.130971  1771 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.138227  1768 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.139088  1768 ts_tablet_manager.cc:1403] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.355s	user 0.015s	sys 0.018s
I20260709 13:00:32.136693  1769 raft_consensus.cc:359] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.135382  1771 ts_tablet_manager.cc:1434] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.006s
I20260709 13:00:32.156926  1635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9396e14142a4f6cb8733d8b08ac73b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.147238  1769 raft_consensus.cc:385] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.180126  1640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da3ba11d664442d7b28bc849c61fdcdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:32.174122  1769 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.181488  1769 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.185411  1769 ts_tablet_manager.cc:1434] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.002s
I20260709 13:00:32.187739  1769 tablet_bootstrap.cc:492] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.144328  1768 raft_consensus.cc:359] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.201393  1768 raft_consensus.cc:385] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.201812  1768 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.202778  1768 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.205713  1768 ts_tablet_manager.cc:1434] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.066s	user 0.012s	sys 0.000s
I20260709 13:00:32.206426  1769 tablet_bootstrap.cc:654] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.220286  1768 tablet_bootstrap.cc:492] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.214951  1771 tablet_bootstrap.cc:492] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:32.248294  1768 tablet_bootstrap.cc:654] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.262897  1771 tablet_bootstrap.cc:654] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.273417  1768 tablet_bootstrap.cc:492] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.274672  1768 ts_tablet_manager.cc:1403] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.003s
I20260709 13:00:32.278935  1769 tablet_bootstrap.cc:492] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.279102  1768 raft_consensus.cc:359] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.279579  1769 ts_tablet_manager.cc:1403] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.092s	user 0.003s	sys 0.040s
I20260709 13:00:32.281821  1768 raft_consensus.cc:385] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.282584  1768 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.284118  1768 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.285713  1769 raft_consensus.cc:359] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.286538  1769 raft_consensus.cc:385] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.287396  1778 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.287985  1778 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.290285  1768 ts_tablet_manager.cc:1434] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.004s
I20260709 13:00:32.290545  1778 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:32.293027  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.293884  1576 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:32.295464  1768 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.295668  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:32.286953  1769 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.299180  1769 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.295464  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:00:32.302884  1688 leader_election.cc:343] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.304797  1769 ts_tablet_manager.cc:1434] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:00:32.305774  1773 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.308672  1773 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.309165  1778 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:32.309623  1778 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.310019  1778 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.310138  1769 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.311471  1773 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.315876  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:32.316748  1650 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:32.318709  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.319722  1769 tablet_bootstrap.cc:654] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.325572  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:32.326757  1539 leader_election.cc:343] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.329212  1778 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.307868  1771 tablet_bootstrap.cc:492] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:32.332962  1809 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:32.335198  1771 ts_tablet_manager.cc:1403] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.120s	user 0.006s	sys 0.014s
I20260709 13:00:32.335448  1778 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:32.336069  1768 tablet_bootstrap.cc:654] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.335889  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:32.337014  1809 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.339926  1809 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.342569  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:32.343355  1576 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.360592  1771 raft_consensus.cc:359] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.361898  1809 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.364439  1809 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.367771  1769 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.368108  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:32.368582  1769 ts_tablet_manager.cc:1403] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.020s
I20260709 13:00:32.369079  1650 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.373024  1769 raft_consensus.cc:359] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.374208  1769 raft_consensus.cc:385] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.374576  1769 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.374647  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:32.375878  1769 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:32.378865  1539 leader_election.cc:343] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.379451  1769 ts_tablet_manager.cc:1434] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
W20260709 13:00:32.384402  1688 leader_election.cc:343] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.389461  1651 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.392549  1771 raft_consensus.cc:385] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.393837  1771 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.395506  1809 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:32.400903  1797 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:32.409054  1771 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.412930  1769 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.427026  1769 tablet_bootstrap.cc:654] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.448019  1797 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:32.451244  1539 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:32.473704  1771 ts_tablet_manager.cc:1434] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.135s	user 0.010s	sys 0.002s
I20260709 13:00:32.475409  1771 tablet_bootstrap.cc:492] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:32.482393  1576 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:32.490782  1650 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:32.493445  1768 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.495196  1768 ts_tablet_manager.cc:1403] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.204s	user 0.042s	sys 0.082s
I20260709 13:00:32.514123  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:32.510965  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.522528  1769 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.523165  1769 ts_tablet_manager.cc:1403] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.111s	user 0.018s	sys 0.009s
I20260709 13:00:32.524765  1768 raft_consensus.cc:359] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.525882  1768 raft_consensus.cc:385] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.526404  1768 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.527834  1768 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.529013  1797 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:32.529103  1809 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:32.529691  1797 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:32.542569  1769 raft_consensus.cc:359] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.543431  1769 raft_consensus.cc:385] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.543744  1769 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.547047  1769 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.551859  1797 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.566232  1769 ts_tablet_manager.cc:1434] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:00:32.565519  1792 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:32.570488  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60798.5us
I20260709 13:00:32.565603  1809 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:32.573227  1778 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.573751  1809 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.569608  1792 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.580518  1792 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.581372  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.581954  1769 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.583057  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.589838  1768 ts_tablet_manager.cc:1434] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.084s	user 0.005s	sys 0.005s
I20260709 13:00:32.591356  1768 tablet_bootstrap.cc:492] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.573830  1778 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.601478  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.597249  1792 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.605929  1792 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.606679  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.608601  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:32.605311  1778 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:00:32.610488  1688 leader_election.cc:343] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:32.611371  1690 leader_election.cc:343] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.611150  1792 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.614784  1792 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.613112  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:32.621105  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.622951  1576 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:32.625123  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:32.616816  1828 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.628053  1836 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:32.634562  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.633570  1836 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.635596  1836 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:32.636855  1614 leader_election.cc:343] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.646622  1768 tablet_bootstrap.cc:654] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.639752  1771 tablet_bootstrap.cc:654] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.653345  1474 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:32.657560  1768 tablet_bootstrap.cc:492] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.658820  1768 ts_tablet_manager.cc:1403] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.008s
I20260709 13:00:32.660100  1769 tablet_bootstrap.cc:654] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.660995  1471 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:32.662932  1768 raft_consensus.cc:359] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.663678  1768 raft_consensus.cc:385] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.664012  1768 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.665186  1768 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.668424  1768 ts_tablet_manager.cc:1434] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:00:32.669556  1768 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.670359  1771 tablet_bootstrap.cc:492] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:32.671082  1771 ts_tablet_manager.cc:1403] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.196s	user 0.007s	sys 0.017s
I20260709 13:00:32.675505  1771 raft_consensus.cc:359] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.677150  1771 raft_consensus.cc:385] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.680470  1769 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.680542  1771 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.681124  1769 ts_tablet_manager.cc:1403] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.099s	user 0.006s	sys 0.014s
I20260709 13:00:32.677966  1836 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.681691  1771 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.679586  1768 tablet_bootstrap.cc:654] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.685148  1836 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.689919  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:32.690764  1771 ts_tablet_manager.cc:1434] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.010s
I20260709 13:00:32.689751  1769 raft_consensus.cc:359] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.690913  1576 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.691339  1769 raft_consensus.cc:385] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.691922  1769 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.692142  1771 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:32.692976  1769 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.696671  1769 ts_tablet_manager.cc:1434] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:00:32.699764  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:32.701792  1614 leader_election.cc:343] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.702885  1576 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:32.704432  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.706050  1839 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:32.707675  1836 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.708518  1836 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.711717  1769 tablet_bootstrap.cc:492] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.719043  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.720422  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:32.721556  1616 leader_election.cc:343] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.717239  1839 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:00:32.723018  1614 leader_election.cc:343] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.723044  1839 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.723763  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.725049  1836 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.726329  1836 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.731149  1768 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.731797  1768 ts_tablet_manager.cc:1403] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.017s
I20260709 13:00:32.732136  1771 tablet_bootstrap.cc:654] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.735155  1768 raft_consensus.cc:359] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.736152  1768 raft_consensus.cc:385] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.736495  1768 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.737260  1768 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.739107  1768 ts_tablet_manager.cc:1434] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:32.742740  1769 tablet_bootstrap.cc:654] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.745185  1768 tablet_bootstrap.cc:492] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.747637  1833 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.748263  1833 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.755453  1768 tablet_bootstrap.cc:654] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.756014  1833 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.756922  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:32.758797  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:32.759097  1539 leader_election.cc:343] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.759770  1725 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:32.762218  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:32.763664  1769 tablet_bootstrap.cc:492] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.763733  1809 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:32.764425  1769 ts_tablet_manager.cc:1403] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.009s
I20260709 13:00:32.764393  1809 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.764899  1809 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.766458  1833 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.767019  1833 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.767845  1769 raft_consensus.cc:359] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.768754  1769 raft_consensus.cc:385] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.769097  1769 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.769878  1769 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.763200  1471 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:32.772842  1769 ts_tablet_manager.cc:1434] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:00:32.774117  1769 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.762182  1771 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:32.781885  1771 ts_tablet_manager.cc:1403] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.012s
I20260709 13:00:32.786022  1769 tablet_bootstrap.cc:654] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.786022  1833 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.787783  1768 tablet_bootstrap.cc:492] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.788450  1768 ts_tablet_manager.cc:1403] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.022s
I20260709 13:00:32.788503  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:32.788923  1771 raft_consensus.cc:359] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.789983  1771 raft_consensus.cc:385] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.790388  1771 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.791482  1839 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.792133  1771 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.793033  1839 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.795789  1771 ts_tablet_manager.cc:1434] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:00:32.792695  1768 raft_consensus.cc:359] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.797899  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.799158  1768 raft_consensus.cc:385] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.799641  1768 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
W20260709 13:00:32.800498  1616 leader_election.cc:343] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.800606  1768 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.801601  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:32.803334  1614 leader_election.cc:343] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.803962  1768 ts_tablet_manager.cc:1434] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:00:32.803813  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.797981  1839 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.805564  1836 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.806746  1768 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.806984  1771 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:32.815440  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.816593  1768 tablet_bootstrap.cc:654] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.816871  1836 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.817442  1836 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.819804  1836 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.823251  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.825120  1576 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:00:32.823251  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.829773  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:32.831477  1836 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:32.836254  1539 leader_election.cc:343] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:32.837137  1539 leader_election.cc:343] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:32.837596  1539 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:32.837946  1768 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.838567  1833 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.839963  1768 ts_tablet_manager.cc:1403] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.011s
I20260709 13:00:32.843072  1809 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.844445  1768 raft_consensus.cc:359] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.845664  1768 raft_consensus.cc:385] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.846040  1768 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.847114  1809 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.847517  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:32.848238  1649 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.848657  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:32.849284  1725 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.851351  1836 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:32.856300  1771 tablet_bootstrap.cc:654] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.848831  1768 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.859082  1768 ts_tablet_manager.cc:1434] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.003s
I20260709 13:00:32.859514  1649 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:32.861166  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:32.862319  1833 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:32.862824  1833 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:32.863889  1769 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.863700  1833 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.864578  1769 ts_tablet_manager.cc:1403] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.091s	user 0.029s	sys 0.036s
I20260709 13:00:32.871074  1768 tablet_bootstrap.cc:492] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.872277  1725 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:32.883761  1768 tablet_bootstrap.cc:654] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.883805  1769 raft_consensus.cc:359] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.885326  1769 raft_consensus.cc:385] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.885792  1769 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.887039  1769 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.888085  1471 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:32.905776  1769 ts_tablet_manager.cc:1434] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.041s	user 0.010s	sys 0.000s
I20260709 13:00:32.920826  1769 tablet_bootstrap.cc:492] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.930661  1769 tablet_bootstrap.cc:654] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.939533  1771 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:32.939998  1768 tablet_bootstrap.cc:492] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:32.940177  1771 ts_tablet_manager.cc:1403] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.133s	user 0.041s	sys 0.046s
I20260709 13:00:32.940639  1769 tablet_bootstrap.cc:492] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:32.940732  1768 ts_tablet_manager.cc:1403] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.070s	user 0.038s	sys 0.021s
I20260709 13:00:32.941217  1769 ts_tablet_manager.cc:1403] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 13:00:32.943533  1771 raft_consensus.cc:359] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.943601  1768 raft_consensus.cc:359] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.944003  1769 raft_consensus.cc:359] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.944592  1771 raft_consensus.cc:385] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.944638  1768 raft_consensus.cc:385] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.944815  1769 raft_consensus.cc:385] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:32.945063  1768 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.945036  1771 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.945273  1769 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:32.946125  1769 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.946092  1768 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.946005  1771 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.948835  1771 ts_tablet_manager.cc:1434] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:00:32.950213  1771 tablet_bootstrap.cc:492] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:32.950306  1768 ts_tablet_manager.cc:1434] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:00:32.948835  1769 ts_tablet_manager.cc:1434] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:32.951790  1769 tablet_bootstrap.cc:492] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:32.953518  1768 tablet_bootstrap.cc:492] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:32.958606  1836 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.959336  1836 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:32.963492  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:32.964119  1769 tablet_bootstrap.cc:654] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.964303  1576 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:32.965974  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:32.967406  1839 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:32.967947  1839 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.968389  1839 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:32.970121  1836 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:32.972437  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:32.973379  1771 tablet_bootstrap.cc:654] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:32.973225  1725 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:32.977296  1649 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:32.998246  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:32.998504  1833 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:33.005198  1768 tablet_bootstrap.cc:654] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.015700  1833 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.009797  1839 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.017532  1833 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.019896  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:33.020889  1576 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.022132  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:33.023269  1651 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.024829  1809 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.025580  1809 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.030151  1809 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.035497  1839 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.034667  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:33.036355  1724 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.036639  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.038695  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:33.033803  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.045065  1833 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.048601  1769 tablet_bootstrap.cc:492] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.049204  1769 ts_tablet_manager.cc:1403] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.098s	user 0.024s	sys 0.019s
I20260709 13:00:33.049489  1724 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:33.051985  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.054240  1836 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.055531  1836 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:33.057147  1836 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.058965  1769 raft_consensus.cc:359] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.059839  1769 raft_consensus.cc:385] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.060195  1769 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.062374  1769 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.066179  1769 ts_tablet_manager.cc:1434] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.002s
I20260709 13:00:33.066099  1768 tablet_bootstrap.cc:492] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.067283  1771 tablet_bootstrap.cc:492] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.067978  1769 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.068740  1771 ts_tablet_manager.cc:1403] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.119s	user 0.029s	sys 0.039s
W20260709 13:00:33.108167  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71254.4us
I20260709 13:00:33.109552  1576 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:33.067970  1768 ts_tablet_manager.cc:1403] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.003s
W20260709 13:00:33.113413  1539 leader_election.cc:343] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.125689  1773 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:33.126536  1773 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.127076  1773 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.164544  1836 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.190572  1768 raft_consensus.cc:359] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.192662  1838 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.193873  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.165853  1836 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.218770  1724 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.204696  1778 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:00:33.222771  1828 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:33.216902  1768 raft_consensus.cc:385] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.216701  1838 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.227895  1768 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.232363  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:33.232184  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:33.234208  1838 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:33.233645  1649 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:33.234639  1575 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:33.227443  1836 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:33.282976  1614 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:33.240732  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.258394  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:33.281651  1771 raft_consensus.cc:359] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.292989  1839 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:33.293915  1771 raft_consensus.cc:385] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.294409  1771 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.265138  1768 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.297794  1768 ts_tablet_manager.cc:1434] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.178s	user 0.013s	sys 0.004s
W20260709 13:00:33.303013  1616 leader_election.cc:343] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.305177  1768 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
W20260709 13:00:33.305068  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66047.9us
W20260709 13:00:33.237903  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 115750us
I20260709 13:00:33.309370  1471 catalog_manager.cc:5697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:33.313665  1828 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:33.315097  1828 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.315868  1828 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.321622  1773 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.322861  1771 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.324761  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:33.327169  1614 leader_election.cc:343] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.335582  1769 tablet_bootstrap.cc:654] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:33.329995  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 93851.4us
I20260709 13:00:33.339082  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.340951  1836 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:33.344775  1771 ts_tablet_manager.cc:1434] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.275s	user 0.012s	sys 0.000s
I20260709 13:00:33.330431  1773 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:33.338512  1539 leader_election.cc:343] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.348201  1768 tablet_bootstrap.cc:654] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.357846  1828 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.349776  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:33.362486  1866 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:33.363440  1771 tablet_bootstrap.cc:492] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.365494  1724 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.374636  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:33.374971  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:33.375563  1651 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.375748  1650 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.378065  1828 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:33.392737  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:33.396299  1771 tablet_bootstrap.cc:654] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.396752  1575 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.402611  1650 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:33.417150  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.418457  1838 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.419435  1838 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:33.420676  1838 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.422415  1769 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.423414  1769 ts_tablet_manager.cc:1403] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.356s	user 0.007s	sys 0.037s
I20260709 13:00:33.427003  1769 raft_consensus.cc:359] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.427927  1769 raft_consensus.cc:385] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.431833  1771 tablet_bootstrap.cc:492] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.432858  1771 ts_tablet_manager.cc:1403] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.000s
I20260709 13:00:33.428267  1769 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.433892  1769 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.438164  1575 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:33.445943  1768 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.446274  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:33.446700  1768 ts_tablet_manager.cc:1403] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.142s	user 0.027s	sys 0.026s
I20260709 13:00:33.433930  1651 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:33.448132  1724 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.451474  1771 raft_consensus.cc:359] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.455798  1771 raft_consensus.cc:385] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.449445  1769 ts_tablet_manager.cc:1434] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.007s
I20260709 13:00:33.456279  1771 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.457412  1771 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.457799  1773 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:33.458415  1866 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:33.454888  1768 raft_consensus.cc:359] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.459527  1768 raft_consensus.cc:385] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.459853  1768 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.459954  1771 ts_tablet_manager.cc:1434] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 13:00:33.460737  1768 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.461853  1771 tablet_bootstrap.cc:492] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.455940  1472 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:33.463754  1845 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.464322  1845 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.457799  1769 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.467830  1845 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.475701  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:33.475960  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.476835  1575 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:33.477499  1768 ts_tablet_manager.cc:1434] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.003s
W20260709 13:00:33.478451  1614 leader_election.cc:343] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.480185  1616 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.480852  1769 tablet_bootstrap.cc:654] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.481200  1724 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:33.484438  1845 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:33.486081  1768 tablet_bootstrap.cc:492] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:33.486322  1771 tablet_bootstrap.cc:654] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.487103  1845 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.487839  1845 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.491983  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.499594  1864 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.500217  1864 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:33.501360  1864 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.496706  1866 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:33.535229  1866 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.535995  1866 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.535396  1836 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.539710  1836 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.540606  1866 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.542810  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:33.543924  1649 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.548535  1769 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.549067  1769 ts_tablet_manager.cc:1403] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.092s	user 0.018s	sys 0.012s
I20260709 13:00:33.551832  1769 raft_consensus.cc:359] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.552949  1769 raft_consensus.cc:385] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.553292  1769 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.553879  1845 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.554067  1769 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.556370  1845 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.556815  1769 ts_tablet_manager.cc:1434] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:33.558458  1769 tablet_bootstrap.cc:492] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.560438  1836 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.567229  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:33.567672  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:33.568416  1576 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.568796  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:33.573690  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:33.575752  1616 leader_election.cc:343] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:33.576705  1614 leader_election.cc:343] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.578276  1864 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:33.582762  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.585219  1768 tablet_bootstrap.cc:654] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.578775  1864 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.588165  1864 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:33.588780  1614 leader_election.cc:343] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.589314  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.590638  1836 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:33.578712  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.597469  1722 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.601212  1471 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:33.604007  1576 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:33.606426  1616 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.609630  1836 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.618291  1769 tablet_bootstrap.cc:654] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.619278  1836 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:33.621519  1771 tablet_bootstrap.cc:492] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.622061  1771 ts_tablet_manager.cc:1403] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.161s	user 0.041s	sys 0.014s
I20260709 13:00:33.620656  1836 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.614845  1864 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.631234  1839 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:33.636991  1771 raft_consensus.cc:359] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.638438  1771 raft_consensus.cc:385] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.638855  1771 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.639672  1771 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.642160  1771 ts_tablet_manager.cc:1434] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:00:33.643347  1769 tablet_bootstrap.cc:492] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.644222  1769 ts_tablet_manager.cc:1403] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.005s
I20260709 13:00:33.647919  1771 tablet_bootstrap.cc:492] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.648798  1769 raft_consensus.cc:359] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.649797  1769 raft_consensus.cc:385] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.651638  1768 tablet_bootstrap.cc:492] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.652212  1768 ts_tablet_manager.cc:1403] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.166s	user 0.017s	sys 0.014s
I20260709 13:00:33.652002  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:33.653592  1651 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.651253  1864 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.661646  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:33.661971  1768 raft_consensus.cc:359] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.662439  1723 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.662695  1768 raft_consensus.cc:385] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.663060  1768 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.664059  1768 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.666759  1768 ts_tablet_manager.cc:1434] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:00:33.667987  1768 tablet_bootstrap.cc:492] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:33.669833  1839 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:33.676333  1771 tablet_bootstrap.cc:654] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.681954  1768 tablet_bootstrap.cc:654] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.650247  1769 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.687314  1471 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:33.692248  1769 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.688316  1651 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:33.694324  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.695281  1769 ts_tablet_manager.cc:1434] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.005s
I20260709 13:00:33.695393  1864 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.696049  1864 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:33.696698  1769 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.697139  1864 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.703426  1723 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:33.709504  1771 tablet_bootstrap.cc:492] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.710083  1771 ts_tablet_manager.cc:1403] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.000s
I20260709 13:00:33.714727  1771 raft_consensus.cc:359] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.715598  1771 raft_consensus.cc:385] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.715917  1771 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.717075  1771 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.719841  1769 tablet_bootstrap.cc:654] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.719803  1771 ts_tablet_manager.cc:1434] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:00:33.728245  1768 tablet_bootstrap.cc:492] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.730813  1836 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.731642  1836 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.735697  1836 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.736235  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:33.737074  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.730854  1771 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
W20260709 13:00:33.738322  1616 leader_election.cc:343] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:33.738873  1614 leader_election.cc:343] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.739336  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.740195  1836 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:33.728945  1768 ts_tablet_manager.cc:1403] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.003s
I20260709 13:00:33.740581  1769 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.741159  1769 ts_tablet_manager.cc:1403] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.010s
I20260709 13:00:33.743754  1768 raft_consensus.cc:359] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.744391  1723 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.744681  1768 raft_consensus.cc:385] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.744989  1768 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.745780  1768 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.744036  1769 raft_consensus.cc:359] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.747391  1769 raft_consensus.cc:385] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.747828  1769 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.748586  1864 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.748889  1769 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.749188  1864 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.749769  1768 ts_tablet_manager.cc:1434] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:00:33.751479  1771 tablet_bootstrap.cc:654] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.755559  1472 catalog_manager.cc:5697] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:33.756898  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:33.758745  1539 leader_election.cc:343] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.759464  1864 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.753543  1768 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:33.753273  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:33.760752  1651 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.751772  1769 ts_tablet_manager.cc:1434] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:00:33.762588  1769 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.762420  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.771236  1864 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:33.775709  1864 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.776443  1864 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.779784  1768 tablet_bootstrap.cc:654] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.773105  1769 tablet_bootstrap.cc:654] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.783717  1771 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.784305  1771 ts_tablet_manager.cc:1403] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.054s	user 0.028s	sys 0.010s
I20260709 13:00:33.791389  1771 raft_consensus.cc:359] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.792204  1771 raft_consensus.cc:385] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.792546  1771 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.793751  1771 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.796995  1771 ts_tablet_manager.cc:1434] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:00:33.797079  1864 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.798725  1771 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.800052  1864 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.801869  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:33.802033  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:33.802974  1651 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:33.808946  1539 leader_election.cc:343] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.816108  1836 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.816774  1836 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.819077  1836 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.821692  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:33.822093  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:33.823585  1616 leader_election.cc:343] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:33.824640  1614 leader_election.cc:343] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:33.825084  1651 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:33.825165  1614 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.826491  1836 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:33.826547  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:33.827455  1771 tablet_bootstrap.cc:654] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.827960  1864 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.828797  1864 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:33.829808  1864 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.830988  1769 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.830732  1768 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.831789  1769 ts_tablet_manager.cc:1403] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.069s	user 0.023s	sys 0.013s
I20260709 13:00:33.831912  1768 ts_tablet_manager.cc:1403] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.000s
I20260709 13:00:33.835796  1768 raft_consensus.cc:359] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.836885  1768 raft_consensus.cc:385] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.836416  1769 raft_consensus.cc:359] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.837486  1768 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.837893  1769 raft_consensus.cc:385] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.838296  1769 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.839046  1768 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.839864  1769 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.842773  1768 ts_tablet_manager.cc:1434] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:00:33.843237  1769 ts_tablet_manager.cc:1434] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 13:00:33.844501  1768 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:33.844552  1769 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.854121  1471 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:33.859678  1771 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.860299  1771 ts_tablet_manager.cc:1403] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.004s
I20260709 13:00:33.860441  1864 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.861475  1864 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.864059  1864 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.866014  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:33.866842  1722 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.867419  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:33.868163  1651 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:33.868412  1771 raft_consensus.cc:359] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.869334  1771 raft_consensus.cc:385] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.869715  1771 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.870095  1769 tablet_bootstrap.cc:654] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.870514  1771 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.870728  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:33.872556  1864 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:33.873016  1771 ts_tablet_manager.cc:1434] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:00:33.873006  1864 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.874667  1771 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.874477  1864 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.877709  1768 tablet_bootstrap.cc:654] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.882688  1771 tablet_bootstrap.cc:654] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.896812  1771 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.897382  1771 ts_tablet_manager.cc:1403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.010s
I20260709 13:00:33.900631  1771 raft_consensus.cc:359] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.901427  1771 raft_consensus.cc:385] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.901716  1771 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.902652  1771 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.903918  1864 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.906746  1771 ts_tablet_manager.cc:1434] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:00:33.908209  1771 tablet_bootstrap.cc:492] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.908854  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:33.909794  1651 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.915517  1771 tablet_bootstrap.cc:654] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.920655  1771 tablet_bootstrap.cc:492] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.921160  1771 ts_tablet_manager.cc:1403] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:00:33.924253  1771 raft_consensus.cc:359] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.925271  1771 raft_consensus.cc:385] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.925657  1771 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.925310  1769 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.926698  1769 ts_tablet_manager.cc:1403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.082s	user 0.032s	sys 0.002s
I20260709 13:00:33.929382  1771 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.930032  1769 raft_consensus.cc:359] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.930804  1769 raft_consensus.cc:385] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.930958  1768 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.931156  1769 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.931562  1768 ts_tablet_manager.cc:1403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.087s	user 0.016s	sys 0.022s
I20260709 13:00:33.932106  1809 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:33.932003  1769 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.932797  1864 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:33.934572  1769 ts_tablet_manager.cc:1434] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:00:33.928438  1651 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:33.934725  1771 ts_tablet_manager.cc:1434] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:00:33.939002  1769 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:33.937769  1768 raft_consensus.cc:359] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.939531  1768 raft_consensus.cc:385] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.939836  1768 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.940305  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:33.940683  1771 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.942405  1809 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:33.940620  1768 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.944435  1838 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:33.947652  1769 tablet_bootstrap.cc:654] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.946559  1768 ts_tablet_manager.cc:1434] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 13:00:33.951037  1768 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:33.952384  1771 tablet_bootstrap.cc:654] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.942970  1809 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:33.960879  1768 tablet_bootstrap.cc:654] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:33.959980  1809 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.962967  1769 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:33.963673  1769 ts_tablet_manager.cc:1403] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.012s
I20260709 13:00:33.971024  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:33.972083  1722 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:33.973279  1771 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:33.973824  1771 ts_tablet_manager.cc:1403] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.007s
I20260709 13:00:33.976248  1768 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:33.977036  1768 ts_tablet_manager.cc:1403] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.011s
I20260709 13:00:33.977279  1771 raft_consensus.cc:359] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.978118  1771 raft_consensus.cc:385] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.978442  1771 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.979195  1771 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.980566  1768 raft_consensus.cc:359] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.981345  1768 raft_consensus.cc:385] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.981559  1771 ts_tablet_manager.cc:1434] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:33.981690  1768 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.982863  1771 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:33.982811  1768 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.987198  1768 ts_tablet_manager.cc:1434] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:00:33.990914  1838 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:33.968071  1769 raft_consensus.cc:359] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.991077  1768 tablet_bootstrap.cc:492] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:33.992377  1769 raft_consensus.cc:385] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:33.992797  1769 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:33.993587  1864 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.000784  1838 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.001060  1864 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.001411  1838 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:33.993752  1769 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.007503  1864 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.008301  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.009582  1724 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.009465  1649 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:34.012164  1769 ts_tablet_manager.cc:1434] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.001s
I20260709 13:00:34.018019  1769 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:34.025342  1771 tablet_bootstrap.cc:654] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:34.029505  1838 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:34.029342  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.030342  1838 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.131735  1769 tablet_bootstrap.cc:654] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:34.036715  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.154175  1833 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.085032  1722 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:34.164515  1833 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.166531  1575 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:34.167676  1833 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.036443  1864 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.041074  1839 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.145011  1838 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.035828  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.041391  1836 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.174335  1864 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.118806  1845 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260709 13:00:34.185405  1907 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.046857  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:34.188992  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:34.189580  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.078738  1809 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.098s
I20260709 13:00:34.193313  1907 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.183471  1833 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.202335  1833 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.184000  1864 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.180716  1649 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
W20260709 13:00:34.181267  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73272.1us
I20260709 13:00:34.218706  1833 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.221606  1768 tablet_bootstrap.cc:654] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:34.182596  1839 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.184914  1836 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:34.229590  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61065.9us
W20260709 13:00:34.225669  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:34.236124  1471 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:34.246275  1839 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.246894  1838 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:34.246738  1908 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:34.247466  1838 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.247852  1838 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.248229  1838 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.267208  1909 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.267940  1909 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.256033  1907 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:00:34.249501  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60639.6us
I20260709 13:00:34.277380  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.278609  1575 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:34.282974  1773 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.294381  1769 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:34.295172  1769 ts_tablet_manager.cc:1403] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.277s	user 0.018s	sys 0.011s
I20260709 13:00:34.296077  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:34.297875  1910 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.298915  1910 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.305331  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.306186  1909 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.306180  1836 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.325340  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.512217  1910 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:34.396203  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:34.471771  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 148404us
I20260709 13:00:34.366000  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.530107  1771 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:34.533417  1771 ts_tablet_manager.cc:1403] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.551s	user 0.019s	sys 0.014s
I20260709 13:00:34.533850  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.327445  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.383252  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:34.527776  1539 leader_election.cc:343] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:34.364971  1773 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.550384  1769 raft_consensus.cc:359] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.545056  1649 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:34.551532  1769 raft_consensus.cc:385] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:34.539212  1573 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:34.338218  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.551920  1769 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:34.426296  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.556140  1769 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.558538  1910 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.560719  1771 raft_consensus.cc:359] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.561416  1771 raft_consensus.cc:385] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:34.561666  1771 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:34.562971  1915 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.564429  1771 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.546607  1725 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:34.569571  1915 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.570019  1915 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.568497  1768 tablet_bootstrap.cc:492] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:34.568872  1769 ts_tablet_manager.cc:1434] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.273s	user 0.013s	sys 0.014s
I20260709 13:00:34.571238  1768 ts_tablet_manager.cc:1403] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.581s	user 0.026s	sys 0.049s
I20260709 13:00:34.568302  1574 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
W20260709 13:00:34.562378  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 134979us
I20260709 13:00:34.579757  1773 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.581338  1864 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.582618  1769 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:34.587248  1771 ts_tablet_manager.cc:1434] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.001s
I20260709 13:00:34.563580  1910 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.566861  1647 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:34.594913  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:34.591077  1864 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.596376  1866 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:34.600030  1907 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.621135  1771 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:34.633010  1768 raft_consensus.cc:359] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.634238  1768 raft_consensus.cc:385] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:34.634833  1768 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:34.636164  1768 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.638723  1768 ts_tablet_manager.cc:1434] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.067s	user 0.010s	sys 0.000s
I20260709 13:00:34.642290  1768 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:34.660413  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:34.660063  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.664599  1573 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:34.665395  1910 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:34.667096  1907 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:34.674871  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:34.676921  1725 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:34.681183  1771 tablet_bootstrap.cc:654] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:34.683795  1864 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:34.687510  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.688540  1866 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:34.693079  1864 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.693549  1864 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.694721  1864 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.698046  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:34.698334  1838 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.699003  1649 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.692796  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.700304  1721 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:00:34.702418  1769 tablet_bootstrap.cc:654] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:34.695391  1915 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.706079  1648 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:34.691078  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.709129  1651 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:34.708115  1915 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.712265  1864 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.713531  1616 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.711841  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:34.715113  1864 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.717047  1911 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.717227  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.717583  1911 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.679126  1690 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:34.718490  1911 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.719132  1907 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:34.719558  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45149.6us
I20260709 13:00:34.704995  1838 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:34.712389  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:34.725446  1864 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:00:34.724092  1915 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.732038  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.732220  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:34.732663  1915 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.733047  1574 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.733278  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.734684  1907 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.735127  1907 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.738029  1907 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.742935  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:34.733166  1915 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.740403  1866 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:34.753180  1866 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.778111  1866 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:34.811503  1688 leader_election.cc:343] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:34.813921  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:34.778111  1909 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.818117  1647 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.838912  1574 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:34.832229  1864 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.840962  1864 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.842854  1768 tablet_bootstrap.cc:654] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:34.818737  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 81887.4us
W20260709 13:00:34.829272  1614 leader_election.cc:343] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:34.826699  1909 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.870246  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:34.879992  1836 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:34.880908  1836 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:34.882202  1836 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:34.876781  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47608.9us
I20260709 13:00:34.905257  1909 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.909622  1907 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.915401  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:34.917075  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:34.917790  1575 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.927585  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:34.928190  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71318us
I20260709 13:00:34.876487  1864 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:34.943097  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:34.943992  1649 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:34.944259  1725 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:34.945608  1907 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:34.931977  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:34.961740  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:00:34.962781  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59939.7us
W20260709 13:00:34.965989  1614 leader_election.cc:343] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:34.965206  1771 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:34.968026  1771 ts_tablet_manager.cc:1403] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.347s	user 0.017s	sys 0.005s
W20260709 13:00:34.973223  1614 leader_election.cc:343] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:34.947757  1690 leader_election.cc:343] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:34.975874  1690 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:34.976511  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:34.978154  1907 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:34.981675  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:34.969347  1720 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:00:34.982649  1573 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:34.976711  1771 raft_consensus.cc:359] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:34.985177  1771 raft_consensus.cc:385] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:34.985881  1771 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:34.989889  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:35.000231  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:35.000516  1647 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:34.938047  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:34.969296  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:35.004815  1771 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:34.983006  1614 leader_election.cc:343] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.001889  1911 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.009521  1771 ts_tablet_manager.cc:1434] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:00:35.010809  1769 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:35.119943  1769 ts_tablet_manager.cc:1403] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.538s	user 0.056s	sys 0.063s
I20260709 13:00:35.123076  1771 tablet_bootstrap.cc:492] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:35.042137  1915 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:35.121592  1616 leader_election.cc:343] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.131991  1650 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:35.132172  1614 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:00:35.134552  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 135965us
I20260709 13:00:35.016522  1838 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.121539  1725 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:35.137928  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:35.018491  1866 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.139111  1907 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:35.139901  1907 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:35.139988  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:00:35.120671  1757 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.0.107.254:32927, user_credentials={real_user=slave}} blocked reactor thread for 79665us
I20260709 13:00:35.141866  1935 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.127379  1575 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:35.141426  1907 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.144770  1910 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.144179  1866 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.147396  1833 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:35.144416  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 161299us
I20260709 13:00:35.155254  1768 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:35.156013  1768 ts_tablet_manager.cc:1403] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.514s	user 0.017s	sys 0.019s
I20260709 13:00:35.158149  1769 raft_consensus.cc:359] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.159078  1769 raft_consensus.cc:385] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.159560  1769 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.159641  1768 raft_consensus.cc:359] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.160926  1768 raft_consensus.cc:385] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.161291  1768 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.162070  1768 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.165028  1768 ts_tablet_manager.cc:1434] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 13:00:35.166087  1915 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.166353  1768 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:35.167361  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:35.167901  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:35.168815  1574 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:35.172397  1769 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.179983  1769 ts_tablet_manager.cc:1434] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 13:00:35.181509  1769 tablet_bootstrap.cc:492] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:35.144132  1838 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.188779  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:35.190012  1723 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:00:35.190910  1771 tablet_bootstrap.cc:654] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.191809  1768 tablet_bootstrap.cc:654] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.191972  1725 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.196396  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:35.196619  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:35.197386  1574 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:35.197540  1575 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:35.199003  1616 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:35.199920  1915 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.202924  1725 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:00:35.206045  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 326610us
I20260709 13:00:35.204968  1471 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.209417  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:35.218349  1769 tablet_bootstrap.cc:654] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.221868  1472 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:35.226073  1911 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.227409  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:35.228518  1645 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:35.229662  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:35.231204  1838 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:35.232280  1648 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:35.232887  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:35.234257  1688 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:35.234603  1866 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:35.234968  1539 leader_election.cc:343] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.235200  1866 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:35.235268  1838 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.241357  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:35.247610  1866 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.275775  1768 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:35.276432  1768 ts_tablet_manager.cc:1403] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.110s	user 0.027s	sys 0.030s
I20260709 13:00:35.278560  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:00:35.279517  1911 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:35.282274  1911 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:35.284863  1911 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.290228  1771 tablet_bootstrap.cc:492] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:35.290982  1771 ts_tablet_manager.cc:1403] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.172s	user 0.019s	sys 0.021s
I20260709 13:00:35.295959  1768 raft_consensus.cc:359] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.296948  1768 raft_consensus.cc:385] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.297266  1768 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.300187  1768 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.302930  1768 ts_tablet_manager.cc:1434] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:00:35.304143  1768 tablet_bootstrap.cc:492] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:35.307255  1769 tablet_bootstrap.cc:492] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:35.307786  1769 ts_tablet_manager.cc:1403] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.127s	user 0.017s	sys 0.012s
I20260709 13:00:35.312295  1946 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.313571  1769 raft_consensus.cc:359] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.315889  1769 raft_consensus.cc:385] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.316201  1769 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.317068  1769 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.321403  1769 ts_tablet_manager.cc:1434] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:00:35.322868  1768 tablet_bootstrap.cc:654] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.329576  1769 tablet_bootstrap.cc:492] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
W20260709 13:00:35.331151  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:35.328227  1913 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:00:35.338794  1771 raft_consensus.cc:359] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.339540  1771 raft_consensus.cc:385] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.339809  1771 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.345394  1771 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.349123  1771 ts_tablet_manager.cc:1434] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.002s
W20260709 13:00:35.328946  1539 leader_election.cc:343] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.352116  1911 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:35.354789  1771 tablet_bootstrap.cc:492] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:35.357596  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:35.358556  1913 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.358749  1768 tablet_bootstrap.cc:492] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:35.359465  1768 ts_tablet_manager.cc:1403] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.000s
W20260709 13:00:35.363160  1539 leader_election.cc:343] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.364048  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:35.368783  1768 raft_consensus.cc:359] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.369300  1913 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.369686  1768 raft_consensus.cc:385] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.370021  1768 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.371646  1768 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.374358  1768 ts_tablet_manager.cc:1434] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:00:35.375705  1768 tablet_bootstrap.cc:492] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:35.376848  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:35.380292  1769 tablet_bootstrap.cc:654] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.391659  1771 tablet_bootstrap.cc:654] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.384831  1913 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.416762  1915 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.418061  1915 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.439849  1768 tablet_bootstrap.cc:654] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.457835  1769 tablet_bootstrap.cc:492] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:35.458573  1769 ts_tablet_manager.cc:1403] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.134s	user 0.018s	sys 0.017s
I20260709 13:00:35.461047  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:35.461871  1574 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:35.464957  1724 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.474602  1471 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.475616  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:35.479576  1911 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:35.480224  1911 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.480741  1911 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.488154  1915 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.490408  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:35.492816  1614 leader_election.cc:343] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.518792  1909 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.519500  1909 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.524958  1909 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.525840  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:35.526636  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:35.527863  1911 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.535092  1913 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.541683  1913 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.536456  1909 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.536674  1474 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.539963  1768 tablet_bootstrap.cc:492] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
W20260709 13:00:35.535208  1616 leader_election.cc:343] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:35.544171  1614 leader_election.cc:343] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.545553  1614 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:35.548236  1768 ts_tablet_manager.cc:1403] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.173s	user 0.026s	sys 0.001s
I20260709 13:00:35.540735  1769 raft_consensus.cc:359] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.549731  1769 raft_consensus.cc:385] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.551065  1769 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.552419  1769 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.554600  1913 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.558451  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:35.559561  1575 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.559149  1909 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.561054  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:35.563061  1539 leader_election.cc:343] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.562812  1768 raft_consensus.cc:359] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.563556  1768 raft_consensus.cc:385] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.563836  1768 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.563954  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:35.566074  1645 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:35.567384  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:35.568027  1768 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.568750  1913 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:35.569196  1913 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.569607  1913 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.571554  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:35.573917  1614 leader_election.cc:343] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.574201  1838 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.574990  1911 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.579384  1768 ts_tablet_manager.cc:1434] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.003s
I20260709 13:00:35.580766  1768 tablet_bootstrap.cc:492] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:35.585162  1769 ts_tablet_manager.cc:1434] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.126s	user 0.009s	sys 0.000s
I20260709 13:00:35.593518  1913 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.594156  1771 tablet_bootstrap.cc:492] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:35.595146  1771 ts_tablet_manager.cc:1403] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.242s	user 0.024s	sys 0.013s
I20260709 13:00:35.598590  1771 raft_consensus.cc:359] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.599545  1771 raft_consensus.cc:385] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.599833  1771 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.599989  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:35.600831  1771 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.601795  1913 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:35.602422  1539 leader_election.cc:343] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.585021  1575 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:35.604177  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:35.604790  1769 tablet_bootstrap.cc:492] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:35.604684  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:35.605453  1645 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.606839  1911 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:35.612342  1911 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:35.613845  1911 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.664776  1771 ts_tablet_manager.cc:1434] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 13:00:35.673458  1771 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:35.675837  1645 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:35.677049  1768 tablet_bootstrap.cc:654] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.618160  1907 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 13:00:35.681025  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:35.682372  1913 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:35.683161  1913 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:35.685169  1913 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.692270  1771 tablet_bootstrap.cc:654] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.725378  1768 tablet_bootstrap.cc:492] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:35.726181  1768 ts_tablet_manager.cc:1403] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.146s	user 0.016s	sys 0.008s
I20260709 13:00:35.696457  1769 tablet_bootstrap.cc:654] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.715086  1911 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.733875  1768 raft_consensus.cc:359] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.734791  1768 raft_consensus.cc:385] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.735075  1768 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.736927  1768 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.739459  1768 ts_tablet_manager.cc:1434] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:00:35.728125  1474 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.753252  1768 tablet_bootstrap.cc:492] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
W20260709 13:00:35.770172  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47267.3us
I20260709 13:00:35.800055  1866 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.808125  1866 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.811328  1866 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.818462  1769 tablet_bootstrap.cc:492] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:35.823648  1768 tablet_bootstrap.cc:654] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.825902  1769 ts_tablet_manager.cc:1403] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.221s	user 0.026s	sys 0.028s
I20260709 13:00:35.831099  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:35.832010  1645 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:35.832695  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:35.835896  1471 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.838456  1978 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.840860  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:35.847512  1913 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.849436  1913 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:35.850201  1913 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.856304  1913 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.864451  1771 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:35.865046  1771 ts_tablet_manager.cc:1403] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.193s	user 0.013s	sys 0.018s
I20260709 13:00:35.866824  1915 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.869176  1771 raft_consensus.cc:359] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.870054  1771 raft_consensus.cc:385] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.870354  1771 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.839040  1978 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.888908  1769 raft_consensus.cc:359] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.889834  1769 raft_consensus.cc:385] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.890172  1769 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:35.890272  1980 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.890928  1980 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:35.892825  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50038.4us
I20260709 13:00:35.871671  1771 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.895841  1769 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.897588  1771 ts_tablet_manager.cc:1434] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:00:35.897964  1981 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:35.898849  1771 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:35.901521  1768 tablet_bootstrap.cc:492] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:35.901679  1769 ts_tablet_manager.cc:1434] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.002s
I20260709 13:00:35.902092  1768 ts_tablet_manager.cc:1403] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.161s	user 0.018s	sys 0.023s
I20260709 13:00:35.902781  1769 tablet_bootstrap.cc:492] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:35.905406  1980 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.905764  1768 raft_consensus.cc:359] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.906618  1768 raft_consensus.cc:385] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:35.906385  1913 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.906960  1768 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
W20260709 13:00:35.907717  1539 leader_election.cc:343] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.915652  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:35.917954  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:35.919083  1724 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:35.919072  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:35.921160  1645 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:35.923180  1913 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:35.925941  1539 leader_election.cc:343] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.926831  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:35.927747  1980 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:35.928622  1980 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.929072  1980 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.931890  1769 tablet_bootstrap.cc:654] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.934206  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:35.937601  1539 leader_election.cc:343] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.939924  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:35.941311  1645 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:35.947618  1978 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:35.951572  1771 tablet_bootstrap.cc:654] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:35.957103  1768 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:35.966382  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:35.968856  1539 leader_election.cc:343] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:35.969409  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:35.970290  1978 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:35.974666  1768 ts_tablet_manager.cc:1434] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.004s
I20260709 13:00:35.988648  1768 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:35.990909  1769 tablet_bootstrap.cc:492] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:35.991567  1769 ts_tablet_manager.cc:1403] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.089s	user 0.018s	sys 0.000s
I20260709 13:00:35.992306  1645 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:35.994596  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:35.995931  1983 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:35.997112  1983 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:35.998307  1983 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.008625  1911 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.009585  1911 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.018115  1769 raft_consensus.cc:359] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.019745  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:36.021430  1911 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.022615  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:36.024250  1614 leader_election.cc:343] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:36.027469  1616 leader_election.cc:343] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.028450  1769 raft_consensus.cc:385] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.028173  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:36.033484  1769 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.035804  1911 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.035753  1769 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.039168  1769 ts_tablet_manager.cc:1434] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.002s
I20260709 13:00:36.046811  1769 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:36.047119  1980 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.049567  1768 tablet_bootstrap.cc:654] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.051354  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:36.055435  1645 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.055714  1771 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:36.056205  1771 ts_tablet_manager.cc:1403] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.158s	user 0.023s	sys 0.010s
I20260709 13:00:36.065603  1973 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.063971  1474 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:36.066781  1973 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.069274  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:36.070708  1724 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.071991  1771 raft_consensus.cc:359] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.073218  1771 raft_consensus.cc:385] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.073773  1771 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.075783  1973 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:36.075785  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.077113  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:36.077915  1648 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
W20260709 13:00:36.078361  1690 leader_election.cc:343] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.075156  1771 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.063850  1645 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:36.081890  1771 ts_tablet_manager.cc:1434] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:00:36.082716  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:36.083635  1771 tablet_bootstrap.cc:492] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:36.068173  1980 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.088264  1983 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:36.097807  1983 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:36.099150  1983 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.105791  1724 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:36.110069  1768 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:36.090229  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:00:36.111344  1768 ts_tablet_manager.cc:1403] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.133s	user 0.023s	sys 0.016s
I20260709 13:00:36.112107  1907 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:36.112936  1907 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.113363  1907 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.114897  1768 raft_consensus.cc:359] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.115768  1768 raft_consensus.cc:385] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.116122  1768 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.116953  1768 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.120222  1768 ts_tablet_manager.cc:1434] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:36.097262  1769 tablet_bootstrap.cc:654] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.121625  1768 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:36.089962  1973 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.124151  1973 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:36.095827  1614 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:36.129225  1907 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.131261  1768 tablet_bootstrap.cc:654] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.135766  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.138890  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:36.140326  1907 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:00:36.142979  1690 leader_election.cc:343] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.144729  1973 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:36.147250  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:36.148044  1771 tablet_bootstrap.cc:654] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.148311  1648 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:36.146550  1471 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:36.152910  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:36.155217  1980 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.155458  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:36.155809  1980 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.156389  1645 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.160944  1974 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.165457  1980 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.166206  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:36.167961  1539 leader_election.cc:343] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.168612  1768 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:36.169392  1768 ts_tablet_manager.cc:1403] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.048s	user 0.005s	sys 0.015s
I20260709 13:00:36.170584  1769 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:36.171226  1769 ts_tablet_manager.cc:1403] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.125s	user 0.005s	sys 0.016s
I20260709 13:00:36.176431  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:36.203466  1974 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.180856  1768 raft_consensus.cc:359] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.210551  1768 raft_consensus.cc:385] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.213199  1768 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.214712  1768 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.212484  1645 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
W20260709 13:00:36.181243  1539 leader_election.cc:343] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.218317  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:36.219229  1980 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.221588  1768 ts_tablet_manager.cc:1434] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.052s	user 0.012s	sys 0.000s
I20260709 13:00:36.223429  1768 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:36.229018  1771 tablet_bootstrap.cc:492] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:36.229729  1771 ts_tablet_manager.cc:1403] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.146s	user 0.025s	sys 0.029s
I20260709 13:00:36.233407  1768 tablet_bootstrap.cc:654] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.235327  1769 raft_consensus.cc:359] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.236418  1769 raft_consensus.cc:385] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.236756  1769 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.237533  1769 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.240329  1769 ts_tablet_manager.cc:1434] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.006s
I20260709 13:00:36.241878  1769 tablet_bootstrap.cc:492] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:36.245015  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:00:36.247238  1974 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:36.247957  1974 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:36.249476  1974 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.258502  1909 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.264119  1769 tablet_bootstrap.cc:654] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.265282  1771 raft_consensus.cc:359] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.269778  1771 raft_consensus.cc:385] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.270331  1771 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.271757  1771 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.274561  1771 ts_tablet_manager.cc:1434] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:00:36.275771  1771 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:36.273398  1471 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:36.285136  1771 tablet_bootstrap.cc:654] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.286326  1983 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.294246  1768 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:36.298305  1768 ts_tablet_manager.cc:1403] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.075s	user 0.018s	sys 0.005s
I20260709 13:00:36.298791  1980 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.323168  1915 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.323412  1768 raft_consensus.cc:359] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.324374  1768 raft_consensus.cc:385] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.323823  1915 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.324684  1768 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.325572  1768 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.332049  1915 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.334143  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:36.335433  1723 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:36.338580  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:36.340763  1915 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:36.341554  1915 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.342125  1915 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.342327  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.343163  1575 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:36.367311  1768 ts_tablet_manager.cc:1434] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 13:00:36.373190  1768 tablet_bootstrap.cc:492] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:36.380602  1909 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.381265  1909 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.385542  1909 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.387883  1915 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.394407  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.395298  1574 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:36.397557  1769 tablet_bootstrap.cc:492] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:36.398168  1769 ts_tablet_manager.cc:1403] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.157s	user 0.011s	sys 0.008s
I20260709 13:00:36.402184  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:36.412762  1909 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:36.413244  1909 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.413656  1909 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.414505  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:36.416851  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:36.421065  1771 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:36.421532  1771 ts_tablet_manager.cc:1403] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.146s	user 0.033s	sys 0.055s
W20260709 13:00:36.424991  1614 leader_election.cc:343] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.420679  1723 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.429744  1771 raft_consensus.cc:359] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.430637  1771 raft_consensus.cc:385] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.430951  1771 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.431859  1771 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.419446  1983 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:00:36.432983  1769 raft_consensus.cc:359] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.434055  1769 raft_consensus.cc:385] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.434350  1771 ts_tablet_manager.cc:1434] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:00:36.434530  1769 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.435096  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:36.435813  1771 tablet_bootstrap.cc:492] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:36.436120  1574 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.437477  1769 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.444594  1769 ts_tablet_manager.cc:1434] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.004s
I20260709 13:00:36.409600  1915 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.450595  1769 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:36.454169  1983 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.457558  1983 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.459987  1983 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.463032  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:36.464318  1651 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:00:36.467384  1768 tablet_bootstrap.cc:654] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:36.493896  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:36.496688  1725 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.506908  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:36.509812  1574 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:36.473632  1909 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.540841  1973 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.544227  1973 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.561357  1616 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:36.563349  1973 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:36.564693  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.563421  1915 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:36.565744  1574 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:36.566452  1915 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:00:36.587395  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43728.7us
I20260709 13:00:36.575136  1909 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.589253  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:36.589995  1575 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.567870  1915 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.579131  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:36.598991  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:36.603653  1913 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.567793  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:00:36.607661  1688 leader_election.cc:343] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.610739  1973 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:36.611449  1973 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.611932  1973 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.600939  1723 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:36.614583  1771 tablet_bootstrap.cc:654] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.616858  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:36.602630  1983 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:36.619415  1935 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.621860  1769 tablet_bootstrap.cc:654] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.622040  1935 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:36.632764  1751 tablet.cc:2406] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:36.637382  1575 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:36.637653  1935 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.638790  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.645592  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:36.647323  1616 leader_election.cc:343] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.648993  1911 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:36.659829  1614 leader_election.cc:343] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.659862  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:36.664856  1911 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:36.666033  1974 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.665990  1911 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.666738  1974 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.668947  1974 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:36.670591  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:36.671137  1614 leader_election.cc:343] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.671634  1651 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:36.674180  1983 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.674561  1768 tablet_bootstrap.cc:492] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:36.675045  1768 ts_tablet_manager.cc:1403] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.302s	user 0.021s	sys 0.023s
I20260709 13:00:36.675968  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:36.677080  1974 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:36.677490  1974 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.678392  1974 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:36.680857  1973 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.671778  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:36.683298  1973 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:36.683429  1915 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.683822  1769 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:36.684446  1769 ts_tablet_manager.cc:1403] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.234s	user 0.018s	sys 0.008s
I20260709 13:00:36.684792  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:36.688905  1913 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.689473  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.689522  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:36.690383  1574 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:36.690397  1575 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:36.693064  1688 leader_election.cc:343] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.700951  1575 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:36.706650  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:36.707968  1973 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:36.720032  1771 tablet_bootstrap.cc:492] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:36.726362  1771 ts_tablet_manager.cc:1403] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.291s	user 0.023s	sys 0.004s
I20260709 13:00:36.724031  1913 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.716477  1768 raft_consensus.cc:359] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.727901  1913 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.730137  1768 raft_consensus.cc:385] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.731848  1913 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.732156  1768 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.731372  1769 raft_consensus.cc:359] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.737138  1769 raft_consensus.cc:385] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.738020  1769 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.736431  1768 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.739385  1769 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.747303  1769 ts_tablet_manager.cc:1434] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.003s
I20260709 13:00:36.742070  1768 ts_tablet_manager.cc:1434] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.067s	user 0.010s	sys 0.002s
I20260709 13:00:36.751490  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:36.750985  1472 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:36.754377  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:36.753800  1472 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:36.755935  1974 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.756898  1771 raft_consensus.cc:359] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.757834  1771 raft_consensus.cc:385] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.758160  1771 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.759604  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:36.760054  1768 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:36.759800  1771 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.760293  1651 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:36.764892  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44470us
I20260709 13:00:36.766223  1769 tablet_bootstrap.cc:492] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
W20260709 13:00:36.767076  1539 leader_election.cc:343] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.788439  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:36.790362  1575 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:36.780279  1978 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:00:36.764925  1974 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:36.770774  1935 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.798892  1771 ts_tablet_manager.cc:1434] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.072s	user 0.012s	sys 0.000s
I20260709 13:00:36.783700  1973 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:00:36.809935  1539 leader_election.cc:343] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.811323  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:36.801236  1935 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.816047  1973 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.820995  1771 tablet_bootstrap.cc:492] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:36.824288  1935 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:36.825292  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:36.825745  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:36.827668  1614 leader_election.cc:343] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:36.828594  1616 leader_election.cc:343] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:36.829092  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:36.831991  1768 tablet_bootstrap.cc:654] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.831802  1913 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.836863  1651 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:00:36.831768  1935 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:36.841753  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:36.849937  2011 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:36.850744  2011 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:36.859025  2012 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.859295  2011 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.899856  1575 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:00:36.909080  1768 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:36.909689  1768 ts_tablet_manager.cc:1403] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.009s
I20260709 13:00:36.929620  2014 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:36.966890  1769 tablet_bootstrap.cc:654] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.990167  1768 raft_consensus.cc:359] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.991508  1768 raft_consensus.cc:385] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:36.991724  1771 tablet_bootstrap.cc:654] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:36.991992  1768 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:36.991756  1472 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:36.992965  1768 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:36.994853  1768 ts_tablet_manager.cc:1434] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.002s
I20260709 13:00:36.996309  1768 tablet_bootstrap.cc:492] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.003355  1771 tablet_bootstrap.cc:492] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:37.003993  1771 ts_tablet_manager.cc:1403] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.183s	user 0.012s	sys 0.006s
I20260709 13:00:37.007258  1771 raft_consensus.cc:359] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.013756  1771 raft_consensus.cc:385] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.014597  1771 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.015434  1771 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.018004  1771 ts_tablet_manager.cc:1434] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:00:37.019515  1771 tablet_bootstrap.cc:492] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:37.022248  2011 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.022910  2011 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.010411  1768 tablet_bootstrap.cc:654] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.012283  1769 tablet_bootstrap.cc:492] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:37.025141  1769 ts_tablet_manager.cc:1403] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.259s	user 0.014s	sys 0.013s
I20260709 13:00:37.028905  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:37.029243  1769 raft_consensus.cc:359] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.029976  1769 raft_consensus.cc:385] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.030294  1769 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.031076  1769 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.039691  2011 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:37.040931  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.042780  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:37.043804  2011 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:37.046897  1769 ts_tablet_manager.cc:1434] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:00:37.048545  1769 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:37.049425  1472 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 2, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:37.050675  1768 tablet_bootstrap.cc:492] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:37.051282  1768 ts_tablet_manager.cc:1403] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.055s	user 0.005s	sys 0.020s
I20260709 13:00:37.056376  1771 tablet_bootstrap.cc:654] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.056488  1769 tablet_bootstrap.cc:654] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.058774  1768 raft_consensus.cc:359] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.059463  1768 raft_consensus.cc:385] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.059697  1768 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.060230  1768 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.065447  1768 ts_tablet_manager.cc:1434] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:00:37.067049  1768 tablet_bootstrap.cc:492] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.073050  1771 tablet_bootstrap.cc:492] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:37.074040  1771 ts_tablet_manager.cc:1403] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.003s
I20260709 13:00:37.080016  1769 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:37.080554  2017 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.080623  1769 ts_tablet_manager.cc:1403] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.004s
I20260709 13:00:37.084237  2017 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.086900  2017 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.087723  1769 raft_consensus.cc:359] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.088629  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.091652  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.082587  1913 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.092012  1768 tablet_bootstrap.cc:654] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:37.093524  1539 leader_election.cc:343] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.088635  1769 raft_consensus.cc:385] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.092757  1645 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:37.095142  1769 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.093565  1913 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.096138  1769 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.097743  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.098769  1769 ts_tablet_manager.cc:1434] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:00:37.099860  1769 tablet_bootstrap.cc:492] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:37.100688  1913 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.101282  1913 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:37.101646  1913 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.102021  1913 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.110862  1769 tablet_bootstrap.cc:654] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.112154  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.108860  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.107328  1935 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 13:00:37.115319  1539 leader_election.cc:343] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.108731  1978 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.124737  1978 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.128437  1978 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.082484  1771 raft_consensus.cc:359] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.132417  1769 tablet_bootstrap.cc:492] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:37.132139  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.133103  1769 ts_tablet_manager.cc:1403] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.008s
I20260709 13:00:37.133320  1648 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:37.129827  1771 raft_consensus.cc:385] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:37.135537  1539 leader_election.cc:343] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.136184  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:37.136488  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.136970  1978 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:37.136786  1769 raft_consensus.cc:359] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.137540  1769 raft_consensus.cc:385] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.137866  1769 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.134590  1771 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.139365  1771 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.138711  1769 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.138687  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:37.141786  1539 leader_election.cc:343] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.141916  1771 ts_tablet_manager.cc:1434] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:00:37.145100  2017 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:37.144786  1913 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.145627  2017 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.146087  2017 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.144855  1769 ts_tablet_manager.cc:1434] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:00:37.147971  1771 tablet_bootstrap.cc:492] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:37.148296  1769 tablet_bootstrap.cc:492] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:37.155632  1915 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.157624  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:37.158255  1648 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.164291  1771 tablet_bootstrap.cc:654] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.166486  1913 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.183619  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:37.187263  1768 tablet_bootstrap.cc:492] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:37.187080  2017 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.188010  1768 ts_tablet_manager.cc:1403] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.121s	user 0.013s	sys 0.001s
I20260709 13:00:37.192531  1769 tablet_bootstrap.cc:654] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.193168  1935 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.194610  1935 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.195439  2017 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.182394  1648 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:37.195777  1768 raft_consensus.cc:359] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.193289  1725 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.196687  1768 raft_consensus.cc:385] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.197160  1768 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.196683  1909 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.198134  1768 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:37.198841  1539 leader_election.cc:343] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.202023  1935 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.201795  1768 ts_tablet_manager.cc:1434] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:00:37.204671  1768 tablet_bootstrap.cc:492] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.209051  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.209419  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:37.211546  1978 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:37.212162  1978 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:37.212240  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.213946  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:37.215292  1614 leader_election.cc:343] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.210857  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:37.216874  1648 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:37.226706  1539 leader_election.cc:343] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.230525  1909 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.213990  1978 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:37.265692  1616 leader_election.cc:343] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.266439  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.233554  1648 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:37.281028  1771 tablet_bootstrap.cc:492] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:37.281706  1771 ts_tablet_manager.cc:1403] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.134s	user 0.018s	sys 0.026s
I20260709 13:00:37.273022  1935 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.282837  1935 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.270968  1725 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.277697  1915 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:37.291620  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.292471  1768 tablet_bootstrap.cc:654] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.293640  1913 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:37.295416  1913 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:37.296866  1913 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.305263  1769 tablet_bootstrap.cc:492] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:37.305861  1769 ts_tablet_manager.cc:1403] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.158s	user 0.036s	sys 0.021s
I20260709 13:00:37.297917  1935 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.325011  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:37.326655  1614 leader_election.cc:343] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.330842  1474 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:37.339401  1771 raft_consensus.cc:359] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.340493  1771 raft_consensus.cc:385] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.340904  1771 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.342043  1771 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.344133  1574 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:37.348546  1771 ts_tablet_manager.cc:1434] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.067s	user 0.010s	sys 0.000s
I20260709 13:00:37.350733  1769 raft_consensus.cc:359] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.352128  1769 raft_consensus.cc:385] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.352803  1769 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.356683  1771 tablet_bootstrap.cc:492] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:37.356161  2018 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:00:37.355275  1769 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:37.366062  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100402us
I20260709 13:00:37.371477  1769 ts_tablet_manager.cc:1434] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.065s	user 0.012s	sys 0.001s
I20260709 13:00:37.376391  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:37.377549  1915 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.378075  1915 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.388720  1978 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.389370  1978 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.377249  1575 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:37.393090  1978 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.396443  1650 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:37.396991  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.400301  1768 tablet_bootstrap.cc:492] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:37.401005  1915 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.401770  1651 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.402035  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.402503  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.408455  1768 ts_tablet_manager.cc:1403] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.204s	user 0.017s	sys 0.010s
I20260709 13:00:37.408645  1769 tablet_bootstrap.cc:492] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:37.419570  1472 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:37.414439  1720 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:37.421005  1645 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:37.428717  2014 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.429956  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:37.431984  1768 raft_consensus.cc:359] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.432893  1768 raft_consensus.cc:385] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.433501  1768 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.423873  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:37.435853  1574 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:00:37.432412  2017 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:37.436823  2017 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.437966  2017 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.441799  1935 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.434459  1768 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.442446  1935 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.455526  1768 ts_tablet_manager.cc:1434] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:00:37.475095  1768 tablet_bootstrap.cc:492] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.489926  1769 tablet_bootstrap.cc:654] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.504505  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.502712  1935 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.511482  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.530663  1771 tablet_bootstrap.cc:654] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.531100  1909 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:37.531564  1909 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.532970  1909 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.539011  2017 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.548800  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:00:37.549094  1614 leader_election.cc:343] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.549638  1645 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.543076  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.553755  2016 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.553951  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:37.554855  1724 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.555126  2016 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.556910  1915 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:37.560626  2016 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.563185  1769 tablet_bootstrap.cc:492] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:37.563879  1769 ts_tablet_manager.cc:1403] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.004s
I20260709 13:00:37.553695  2017 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.565189  1973 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.569041  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:37.575500  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.569890  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:37.577472  1539 leader_election.cc:343] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:37.578642  1539 leader_election.cc:343] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.579021  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:37.581506  2017 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:37.588070  1768 tablet_bootstrap.cc:654] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.590699  1974 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:37.597653  1771 tablet_bootstrap.cc:492] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:37.599017  1771 ts_tablet_manager.cc:1403] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.243s	user 0.006s	sys 0.035s
W20260709 13:00:37.601333  1616 leader_election.cc:343] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.601928  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.603436  1915 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:37.612274  1769 raft_consensus.cc:359] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.614022  1769 raft_consensus.cc:385] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.614339  1769 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.615072  1769 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.620309  1973 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.621366  1973 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.614779  1724 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:37.625355  1973 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:37.626082  1909 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.627691  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:37.630281  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:37.630654  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.636548  1649 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:37.639606  1769 ts_tablet_manager.cc:1434] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.004s
I20260709 13:00:37.641323  1769 tablet_bootstrap.cc:492] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:37.652092  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:37.657442  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:37.659657  1973 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.660933  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:37.661959  1573 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:37.666437  1614 leader_election.cc:343] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.670182  1973 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:37.675382  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47842.1us
I20260709 13:00:37.680787  2016 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:37.681586  2016 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:37.682626  2016 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.686275  1771 raft_consensus.cc:359] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.686934  1771 raft_consensus.cc:385] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.687196  1771 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.690838  1771 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.693929  1771 ts_tablet_manager.cc:1434] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.094s	user 0.017s	sys 0.000s
I20260709 13:00:37.676970  1909 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.677342  1645 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:37.705749  1768 tablet_bootstrap.cc:492] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:37.706102  1573 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:37.706398  1768 ts_tablet_manager.cc:1403] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.232s	user 0.022s	sys 0.003s
I20260709 13:00:37.711565  1768 raft_consensus.cc:359] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.713019  1768 raft_consensus.cc:385] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.713456  1768 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.714448  1768 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.715862  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:37.724925  1909 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:37.728654  1909 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:37.729840  1909 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.742093  1771 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:37.746307  1768 ts_tablet_manager.cc:1434] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.003s
I20260709 13:00:37.780961  1768 tablet_bootstrap.cc:492] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.805352  1769 tablet_bootstrap.cc:654] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.805902  1474 catalog_manager.cc:5697] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:37.807057  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:37.808561  1768 tablet_bootstrap.cc:654] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.809367  1771 tablet_bootstrap.cc:654] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.830397  2016 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.835620  1472 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:37.836095  1978 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:37.851464  1913 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:37.860600  1768 tablet_bootstrap.cc:492] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:37.861171  1768 ts_tablet_manager.cc:1403] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.033s
I20260709 13:00:37.864233  1769 tablet_bootstrap.cc:492] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:37.864984  1769 ts_tablet_manager.cc:1403] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.224s	user 0.018s	sys 0.017s
I20260709 13:00:37.864825  1768 raft_consensus.cc:359] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.865644  1768 raft_consensus.cc:385] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.865978  1768 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.866686  1768 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.868613  1768 ts_tablet_manager.cc:1434] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 13:00:37.870428  1539 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:37.872915  1768 tablet_bootstrap.cc:492] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.881055  1771 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:37.881570  1771 ts_tablet_manager.cc:1403] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.140s	user 0.013s	sys 0.018s
I20260709 13:00:37.882800  1768 tablet_bootstrap.cc:654] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.900087  2016 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.900744  2016 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.902770  1769 raft_consensus.cc:359] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.904251  1909 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.907781  1909 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.903065  2016 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.916042  1771 raft_consensus.cc:359] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.916639  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:37.917384  1771 raft_consensus.cc:385] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.917768  1573 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:37.920359  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:37.921384  1911 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:37.921792  1911 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.923941  1768 tablet_bootstrap.cc:492] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:37.922814  1911 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.925202  1768 ts_tablet_manager.cc:1403] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.053s	user 0.027s	sys 0.004s
I20260709 13:00:37.926159  1909 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.926415  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:37.928458  1614 leader_election.cc:343] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.929569  1768 raft_consensus.cc:359] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.932216  1768 raft_consensus.cc:385] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.932830  1768 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.934372  1768 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.938468  1768 ts_tablet_manager.cc:1434] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:00:37.940594  1768 tablet_bootstrap.cc:492] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:37.954800  1769 raft_consensus.cc:385] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:37.955281  1769 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.955091  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:37.956795  1911 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.958151  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:37.959077  1725 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:37.917876  1771 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:37.963578  1771 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.964565  1768 tablet_bootstrap.cc:654] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:37.972726  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:37.957088  1769 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:37.966567  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:00:37.975898  1614 leader_election.cc:343] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.979372  1771 ts_tablet_manager.cc:1434] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.097s	user 0.010s	sys 0.000s
I20260709 13:00:37.981077  1769 ts_tablet_manager.cc:1434] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.116s	user 0.007s	sys 0.004s
W20260709 13:00:37.981660  1539 leader_election.cc:343] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:37.977926  1911 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:37.978868  1913 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:38.012681  1913 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.013441  1913 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:37.993433  1771 tablet_bootstrap.cc:492] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:37.994403  1769 tablet_bootstrap.cc:492] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:38.003865  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:38.032788  1574 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.008877  2039 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.036101  2039 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.039654  2039 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.041344  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.041895  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.042588  1576 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:38.044037  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:38.044705  1614 leader_election.cc:343] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.046373  1911 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:38.046844  1911 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.047015  1769 tablet_bootstrap.cc:654] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.047264  1911 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.049444  1974 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.050081  1974 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.050598  1574 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:38.055275  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:38.056905  1974 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:38.057308  2039 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:38.056855  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.058727  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.062704  1771 tablet_bootstrap.cc:654] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.062971  1911 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.064630  1688 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.065280  2017 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.066393  2017 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.066694  1974 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.058185  2039 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:38.069288  2017 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.070488  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.070225  2039 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.072767  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.074920  1650 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:38.075434  1913 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.076978  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:38.077982  2017 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:38.078421  2017 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.078781  2017 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.082641  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:38.083133  1768 tablet_bootstrap.cc:492] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:38.083719  1768 ts_tablet_manager.cc:1403] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.144s	user 0.038s	sys 0.005s
W20260709 13:00:38.084652  1539 leader_election.cc:343] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:38.089807  1614 leader_election.cc:343] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.092481  1911 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.099232  1913 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.167021  1768 raft_consensus.cc:359] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.170610  1768 raft_consensus.cc:385] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.171092  1768 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.172103  1768 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.176532  1768 ts_tablet_manager.cc:1434] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.092s	user 0.010s	sys 0.003s
I20260709 13:00:38.100992  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:38.180433  2017 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.188838  1983 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.189498  1983 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.191249  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:38.192039  1720 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.193051  1983 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.111505  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:38.196223  2017 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.197470  1769 tablet_bootstrap.cc:492] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:38.197930  1769 ts_tablet_manager.cc:1403] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.205s	user 0.017s	sys 0.029s
I20260709 13:00:38.199035  1771 tablet_bootstrap.cc:492] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:38.199645  1771 ts_tablet_manager.cc:1403] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.214s	user 0.015s	sys 0.010s
I20260709 13:00:38.201637  1473 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.213774  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.219118  1771 raft_consensus.cc:359] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.223747  1771 raft_consensus.cc:385] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.223541  1769 raft_consensus.cc:359] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.224102  1771 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.224442  1769 raft_consensus.cc:385] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.224967  1769 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.226020  1769 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.228641  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:38.224880  1771 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.231906  1771 ts_tablet_manager.cc:1434] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:00:38.231626  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:38.232589  1649 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.233168  1771 tablet_bootstrap.cc:492] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:38.235486  1768 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
W20260709 13:00:38.242637  1539 leader_election.cc:343] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:38.243361  1539 leader_election.cc:343] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:38.243883  1539 leader_election.cc:343] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:38.245221  1614 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:38.207165  1983 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:00:38.207484  1574 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.253679  1769 ts_tablet_manager.cc:1434] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.002s
I20260709 13:00:38.250032  1978 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:00:38.265779  1769 tablet_bootstrap.cc:492] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:38.249140  1720 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:38.276808  1649 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:38.281874  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.288832  1768 tablet_bootstrap.cc:654] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.268030  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.291316  1913 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:38.292001  1913 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:38.295518  1913 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.308109  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:38.311086  2017 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:38.311781  2017 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:38.314373  2017 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:38.321825  1539 leader_election.cc:343] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.322587  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.323010  1771 tablet_bootstrap.cc:654] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.328528  1978 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.357810  1574 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:38.365020  1768 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:38.365664  1768 ts_tablet_manager.cc:1403] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.131s	user 0.005s	sys 0.018s
I20260709 13:00:38.369711  1723 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.372895  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:38.374989  2039 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:38.375692  2039 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:38.375298  1768 raft_consensus.cc:359] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.376261  1768 raft_consensus.cc:385] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.376675  1768 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.376818  2039 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.377457  1768 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.388828  1915 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:38.403149  1768 ts_tablet_manager.cc:1434] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.037s	user 0.011s	sys 0.000s
I20260709 13:00:38.411851  1768 tablet_bootstrap.cc:492] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:38.411996  1474 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.449841  1472 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.453292  1769 tablet_bootstrap.cc:654] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.467228  1768 tablet_bootstrap.cc:654] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.475420  1769 tablet_bootstrap.cc:492] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:38.476131  1769 ts_tablet_manager.cc:1403] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.211s	user 0.008s	sys 0.012s
I20260709 13:00:38.481175  1769 raft_consensus.cc:359] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.482074  1769 raft_consensus.cc:385] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.482375  1769 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.483323  1474 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.484163  1768 tablet_bootstrap.cc:492] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:38.485869  1768 ts_tablet_manager.cc:1403] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.082s	user 0.004s	sys 0.022s
I20260709 13:00:38.483397  1769 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.491112  1769 ts_tablet_manager.cc:1434] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:00:38.493729  1769 tablet_bootstrap.cc:492] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:38.489784  1768 raft_consensus.cc:359] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.494568  1768 raft_consensus.cc:385] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.494870  1768 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.495541  1768 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.500475  1771 tablet_bootstrap.cc:492] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:38.501111  1771 ts_tablet_manager.cc:1403] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.268s	user 0.020s	sys 0.006s
I20260709 13:00:38.502362  1769 tablet_bootstrap.cc:654] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.505362  1771 raft_consensus.cc:359] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.506356  1771 raft_consensus.cc:385] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.506647  1771 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.513011  1768 ts_tablet_manager.cc:1434] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:00:38.514915  1768 tablet_bootstrap.cc:492] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:38.507615  1771 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.518527  1771 ts_tablet_manager.cc:1434] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:00:38.519785  1771 tablet_bootstrap.cc:492] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:38.522951  1769 tablet_bootstrap.cc:492] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:38.523670  1769 ts_tablet_manager.cc:1403] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.012s
I20260709 13:00:38.545897  1769 raft_consensus.cc:359] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.546770  1769 raft_consensus.cc:385] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.547066  1769 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.551774  2039 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:38.550918  1769 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.554734  1769 ts_tablet_manager.cc:1434] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.003s
I20260709 13:00:38.551903  1768 tablet_bootstrap.cc:654] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.559861  1771 tablet_bootstrap.cc:654] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.566525  1769 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:38.580598  2017 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.581283  2017 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.585223  1769 tablet_bootstrap.cc:654] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.585676  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.587301  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.588868  1771 tablet_bootstrap.cc:492] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
W20260709 13:00:38.588961  1539 leader_election.cc:343] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.589494  1771 ts_tablet_manager.cc:1403] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.005s
W20260709 13:00:38.590070  1539 leader_election.cc:343] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.590731  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.592202  1978 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.592881  1768 tablet_bootstrap.cc:492] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:38.593500  1768 ts_tablet_manager.cc:1403] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.002s
I20260709 13:00:38.598186  1768 raft_consensus.cc:359] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.599053  1768 raft_consensus.cc:385] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.599380  1768 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.600137  1768 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:38.602316  1614 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:38.605032  2017 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.606647  1771 raft_consensus.cc:359] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.607954  1771 raft_consensus.cc:385] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.608374  1771 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.610297  1771 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.614049  1768 ts_tablet_manager.cc:1434] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:00:38.615614  1768 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:38.620788  1771 ts_tablet_manager.cc:1434] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 13:00:38.622217  1771 tablet_bootstrap.cc:492] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:38.628608  1973 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.629462  1973 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.632948  1768 tablet_bootstrap.cc:654] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.634392  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.634923  1769 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:38.636910  1769 ts_tablet_manager.cc:1403] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.071s	user 0.029s	sys 0.013s
I20260709 13:00:38.637419  1974 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.637811  1771 tablet_bootstrap.cc:654] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.637966  1974 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.641091  1769 raft_consensus.cc:359] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.641930  1769 raft_consensus.cc:385] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.642282  1769 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.642987  1769 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.645236  1769 ts_tablet_manager.cc:1434] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:00:38.646281  1769 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:38.647632  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.649441  1576 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:38.650885  1973 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:38.650794  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.652513  1651 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:38.654603  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.656736  1769 tablet_bootstrap.cc:654] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.656780  1973 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.647694  1909 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.658699  1909 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.660743  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:38.661849  2011 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:38.662274  2011 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.662631  2011 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.683409  1973 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.689626  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.692730  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.693882  1723 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:38.695601  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:38.697022  1911 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.698463  1974 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:38.701119  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.701993  1651 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:38.705562  1911 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.692158  1768 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:38.707572  1768 ts_tablet_manager.cc:1403] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.092s	user 0.022s	sys 0.014s
I20260709 13:00:38.710855  1768 raft_consensus.cc:359] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.712551  1911 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.689420  1771 tablet_bootstrap.cc:492] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:38.713198  1911 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.724859  1771 ts_tablet_manager.cc:1403] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.103s	user 0.017s	sys 0.013s
I20260709 13:00:38.727226  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.728130  1576 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:38.728322  1771 raft_consensus.cc:359] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.729705  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:38.737190  1771 raft_consensus.cc:385] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.737556  1771 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.737869  2039 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:38.738409  2039 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.715790  1909 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.738600  1771 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.738880  2039 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.742256  1769 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:38.742753  1771 ts_tablet_manager.cc:1434] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:00:38.712322  1768 raft_consensus.cc:385] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.743368  1768 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.744236  1768 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.742834  1769 ts_tablet_manager.cc:1403] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.050s
I20260709 13:00:38.747318  1768 ts_tablet_manager.cc:1434] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:00:38.748699  1768 tablet_bootstrap.cc:492] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:38.749779  1769 raft_consensus.cc:359] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.750783  1769 raft_consensus.cc:385] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.751235  1769 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:38.752133  1769 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.734566  2011 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.755297  1769 ts_tablet_manager.cc:1434] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:38.755849  1911 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.757436  2011 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:38.757933  1768 tablet_bootstrap.cc:654] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.758208  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:38.758966  1576 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.761426  1771 tablet_bootstrap.cc:492] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:38.765903  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.762131  1769 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:38.768323  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:38.769125  1649 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.773818  1909 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.774487  1909 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.785534  1909 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.785789  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:38.786578  2064 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:38.788502  1616 leader_election.cc:343] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:38.790794  1614 leader_election.cc:343] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.794353  2017 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:38.753286  2039 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.807228  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:38.808025  1573 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.817039  1576 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:38.818802  2017 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.819381  2017 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.820508  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:38.821658  2085 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:38.822364  2085 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:38.824810  1573 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:38.824194  2085 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.826254  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:38.828469  1909 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:38.829110  1909 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:38.830081  1909 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.830875  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.832175  1649 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:38.835059  1978 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.826617  2064 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.821724  2017 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.843386  1978 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.843389  2064 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.836272  2039 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:38.836786  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.837802  1913 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:38.846127  1978 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:38.845995  1614 leader_election.cc:343] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.846901  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:38.847851  1915 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.853749  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:38.855300  1539 leader_election.cc:343] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:38.863710  1539 leader_election.cc:343] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.865595  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:38.866124  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.866452  1645 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:38.869087  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:38.871582  1539 leader_election.cc:343] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.872012  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.877452  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:38.866798  1911 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:00:38.884677  1913 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:38.884160  1771 tablet_bootstrap.cc:654] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.876482  1978 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:38.890043  1769 tablet_bootstrap.cc:654] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:38.876282  2017 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:38.897728  1768 tablet_bootstrap.cc:492] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:38.898528  1768 ts_tablet_manager.cc:1403] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.150s	user 0.025s	sys 0.053s
I20260709 13:00:38.903540  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.886124  2064 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:38.894209  1614 leader_election.cc:343] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.893971  1978 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.923614  1978 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:38.923132  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:38.946014  1471 catalog_manager.cc:5697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:38.941990  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52922.7us
W20260709 13:00:38.963696  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57078.6us
I20260709 13:00:38.968040  1771 tablet_bootstrap.cc:492] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:38.969396  1771 ts_tablet_manager.cc:1403] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.211s	user 0.034s	sys 0.019s
W20260709 13:00:38.977837  1539 leader_election.cc:343] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:38.979084  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:38.959961  1723 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:38.977885  1771 raft_consensus.cc:359] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:38.981375  1771 raft_consensus.cc:385] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:38.981976  1983 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:39.003151  1724 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.004665  1472 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:39.006933  1768 raft_consensus.cc:359] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.045699  1768 raft_consensus.cc:385] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.046181  1768 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.046007  1771 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.046990  1768 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.050865  1769 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.051856  1769 ts_tablet_manager.cc:1403] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.290s	user 0.015s	sys 0.010s
I20260709 13:00:39.056842  1771 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.064494  1768 ts_tablet_manager.cc:1434] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.166s	user 0.009s	sys 0.002s
I20260709 13:00:39.065623  1771 ts_tablet_manager.cc:1434] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:00:39.066275  1768 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:39.017033  1978 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.055665  1769 raft_consensus.cc:359] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.071040  1769 raft_consensus.cc:385] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.072749  1769 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.070487  1771 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.072991  2039 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.079095  2039 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.080792  2093 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.081382  2093 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:39.083118  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:39.083217  1725 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.082530  1769 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.085122  1978 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.093415  2039 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.093333  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:39.096539  1576 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:39.096720  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:39.097618  2093 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:39.098104  1614 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.104653  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:39.104686  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:39.106580  1650 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:39.096028  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:39.108145  1649 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.109817  1911 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:39.112879  1769 ts_tablet_manager.cc:1434] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
W20260709 13:00:39.114418  1539 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.114627  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:39.115686  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:39.116108  1723 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.119094  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:39.123950  1769 tablet_bootstrap.cc:492] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:39.126665  1911 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:39.127192  1911 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.127740  1911 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.127787  1649 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:39.129490  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:39.137022  1983 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:39.137885  1983 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:39.143106  1913 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.143795  1913 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.144315  1983 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.122929  1978 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:39.150861  1913 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.151258  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:39.161746  1539 leader_election.cc:343] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.156062  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:39.153949  1978 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.163211  1978 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:39.164966  1539 leader_election.cc:343] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.165527  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:39.169315  2093 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:39.197499  1472 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:39.210784  1911 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.216703  1723 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:39.217516  1768 tablet_bootstrap.cc:654] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.219213  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:39.218670  1771 tablet_bootstrap.cc:654] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.221228  1911 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.230561  1769 tablet_bootstrap.cc:654] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.230043  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:39.234234  1614 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.234181  1978 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.237164  1573 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:39.240958  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:39.243290  1911 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:39.248868  1771 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.249423  1771 ts_tablet_manager.cc:1403] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.179s	user 0.016s	sys 0.006s
I20260709 13:00:39.249472  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:39.250468  1650 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:39.251756  1771 raft_consensus.cc:359] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.252381  1771 raft_consensus.cc:385] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.252650  1771 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.253383  1771 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.256669  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:39.259073  1614 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:39.259537  1539 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.260036  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:39.260900  2093 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:39.262007  1978 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.256115  1771 ts_tablet_manager.cc:1434] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:00:39.265400  1771 tablet_bootstrap.cc:492] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.266101  2085 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:39.289767  1771 tablet_bootstrap.cc:654] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.305864  1973 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:39.307484  2085 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.308068  2085 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.310456  2085 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:39.313843  1909 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:39.315956  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:39.316804  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:39.318239  1690 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:39.324594  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:39.325451  1649 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:39.321126  2085 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:39.336963  2085 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.338317  2085 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.345144  1768 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:39.345747  1768 ts_tablet_manager.cc:1403] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.280s	user 0.016s	sys 0.009s
I20260709 13:00:39.353648  1768 raft_consensus.cc:359] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.354504  1768 raft_consensus.cc:385] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.354789  1768 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.356469  1768 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.359354  1768 ts_tablet_manager.cc:1434] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.005s
I20260709 13:00:39.363240  1771 tablet_bootstrap.cc:492] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.363973  1771 ts_tablet_manager.cc:1403] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.099s	user 0.022s	sys 0.004s
I20260709 13:00:39.367204  1771 raft_consensus.cc:359] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.372790  1771 raft_consensus.cc:385] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.373346  1771 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.374087  1771 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.382303  1771 ts_tablet_manager.cc:1434] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.001s
I20260709 13:00:39.382513  1769 tablet_bootstrap.cc:492] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.383071  1769 ts_tablet_manager.cc:1403] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.268s	user 0.023s	sys 0.030s
I20260709 13:00:39.369550  1768 tablet_bootstrap.cc:492] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:39.386746  1769 raft_consensus.cc:359] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.387552  1769 raft_consensus.cc:385] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.387861  1769 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.379912  2085 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.388953  1769 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.392233  1771 tablet_bootstrap.cc:492] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.401597  2085 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:39.402889  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:39.406278  1649 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.408792  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:39.409564  1576 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.422667  1771 tablet_bootstrap.cc:654] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.427686  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:39.428192  1769 ts_tablet_manager.cc:1434] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:00:39.430127  1769 tablet_bootstrap.cc:492] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:39.439121  1768 tablet_bootstrap.cc:654] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.440512  1690 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:39.442631  1771 tablet_bootstrap.cc:492] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.443176  1771 ts_tablet_manager.cc:1403] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.005s
I20260709 13:00:39.445462  1978 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.443017  1649 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:39.446193  1978 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.450140  2110 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:39.462073  1771 raft_consensus.cc:359] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.463011  1771 raft_consensus.cc:385] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.463294  1771 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.463898  1771 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.466391  1771 ts_tablet_manager.cc:1434] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.003s
I20260709 13:00:39.472746  1769 tablet_bootstrap.cc:654] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.475622  1771 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.468078  2110 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:39.481118  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:39.483316  1539 leader_election.cc:343] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.481956  2110 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.485172  1978 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.488008  1771 tablet_bootstrap.cc:654] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.487547  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:39.490056  1539 leader_election.cc:343] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.490576  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:39.491566  2093 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:39.498652  1768 tablet_bootstrap.cc:492] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:39.499563  1768 ts_tablet_manager.cc:1403] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.130s	user 0.016s	sys 0.009s
I20260709 13:00:39.504197  1768 raft_consensus.cc:359] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.505378  1768 raft_consensus.cc:385] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.505805  1768 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.507038  1768 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.510511  1768 ts_tablet_manager.cc:1434] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:00:39.511662  1768 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:39.512189  1769 tablet_bootstrap.cc:492] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.512785  1769 ts_tablet_manager.cc:1403] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.083s	user 0.021s	sys 0.005s
I20260709 13:00:39.516839  1769 raft_consensus.cc:359] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.517697  1769 raft_consensus.cc:385] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.518126  1769 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.519137  1769 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.519896  1768 tablet_bootstrap.cc:654] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.522593  1771 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.522540  1769 ts_tablet_manager.cc:1434] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:00:39.523247  1771 ts_tablet_manager.cc:1403] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.010s
I20260709 13:00:39.525449  1769 tablet_bootstrap.cc:492] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:39.528102  1771 raft_consensus.cc:359] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.532493  1771 raft_consensus.cc:385] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.532845  1771 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.533615  1771 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.537180  1771 ts_tablet_manager.cc:1434] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.008s
I20260709 13:00:39.537392  1769 tablet_bootstrap.cc:654] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.538292  1771 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.547907  1771 tablet_bootstrap.cc:654] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.551215  1769 tablet_bootstrap.cc:492] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.551709  1769 ts_tablet_manager.cc:1403] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.010s
I20260709 13:00:39.550876  1472 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:39.555140  1769 raft_consensus.cc:359] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.564962  1769 raft_consensus.cc:385] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.568802  1769 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.570341  1769 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.573330  1769 ts_tablet_manager.cc:1434] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.002s
I20260709 13:00:39.577482  1768 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:39.577899  1913 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.578120  1768 ts_tablet_manager.cc:1403] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.000s
I20260709 13:00:39.578548  1913 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.581382  1983 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:39.582194  1769 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:39.581705  1768 raft_consensus.cc:359] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.582782  1768 raft_consensus.cc:385] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.583077  1768 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.584017  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:39.583813  1768 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.584800  1650 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:39.586668  1768 ts_tablet_manager.cc:1434] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:00:39.587311  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:39.589600  2093 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:39.590049  2093 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.590407  2093 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.593562  1769 tablet_bootstrap.cc:654] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.595430  2085 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.596016  2085 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.601491  2093 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.604087  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:39.607161  2085 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:39.608722  1769 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.609040  1913 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.612185  1769 ts_tablet_manager.cc:1403] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.005s
I20260709 13:00:39.613605  2093 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.615792  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:39.622219  1771 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.622875  1771 ts_tablet_manager.cc:1403] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.085s	user 0.017s	sys 0.023s
I20260709 13:00:39.615636  1769 raft_consensus.cc:359] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.625280  1769 raft_consensus.cc:385] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.625662  1769 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.626441  1771 raft_consensus.cc:359] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.626598  1769 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.627355  1771 raft_consensus.cc:385] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.627715  1771 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.628577  1769 ts_tablet_manager.cc:1434] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:00:39.628481  1771 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.630424  1769 tablet_bootstrap.cc:492] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:39.630533  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:39.629297  1983 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:39.632743  1771 ts_tablet_manager.cc:1434] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:00:39.633989  1771 tablet_bootstrap.cc:492] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.629981  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:39.637149  2085 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:39.639194  1769 tablet_bootstrap.cc:654] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.642005  1771 tablet_bootstrap.cc:654] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.655968  1771 tablet_bootstrap.cc:492] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.656682  1771 ts_tablet_manager.cc:1403] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.001s
I20260709 13:00:39.657812  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:39.658562  1649 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:39.659814  1539 leader_election.cc:343] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.658442  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:39.660301  1771 raft_consensus.cc:359] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.670526  1771 raft_consensus.cc:385] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.671159  1771 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.671952  1771 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.677345  1769 tablet_bootstrap.cc:492] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.677913  1769 ts_tablet_manager.cc:1403] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.048s	user 0.021s	sys 0.005s
I20260709 13:00:39.680388  1769 raft_consensus.cc:359] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.681342  1769 raft_consensus.cc:385] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.680949  1649 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:39.681633  1769 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.682394  1769 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.685875  1769 ts_tablet_manager.cc:1434] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:00:39.687177  1769 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:39.588019  1768 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:39.689764  1771 ts_tablet_manager.cc:1434] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 13:00:39.693264  1771 tablet_bootstrap.cc:492] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:39.697204  1769 tablet_bootstrap.cc:654] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.696981  1909 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.698635  1909 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.699105  2110 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.702967  1909 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.703446  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:39.703996  1771 tablet_bootstrap.cc:654] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.699764  2110 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.705552  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:39.706329  1576 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:39.708042  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:39.713491  1539 leader_election.cc:343] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.714787  2121 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:39.715370  2121 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:39.715843  1909 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:39.716452  1909 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.716295  2121 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.717100  1909 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.733397  2110 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:39.782415  1768 tablet_bootstrap.cc:654] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:39.793370  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 83930.8us
I20260709 13:00:39.797742  1725 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.803725  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:39.792239  1911 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.809451  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:39.804638  1720 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:39.809348  1911 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.830049  1911 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.853654  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:39.853751  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:39.854614  1573 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:39.856647  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:39.857616  1614 leader_election.cc:343] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:39.855226  1909 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.866566  1769 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:39.869323  1769 ts_tablet_manager.cc:1403] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.018s
I20260709 13:00:39.871093  1909 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.872692  1909 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:39.873941  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:39.874749  1573 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.875901  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:39.877229  1723 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.873143  1909 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.878934  1909 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.887171  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:39.894229  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 129940us
I20260709 13:00:39.890919  1650 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:39.924708  2132 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.948060  1909 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.950832  1771 tablet_bootstrap.cc:492] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:39.951453  1771 ts_tablet_manager.cc:1403] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.258s	user 0.024s	sys 0.008s
I20260709 13:00:39.952658  2132 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.967816  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:39.968627  1576 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:39.991868  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94203.9us
I20260709 13:00:39.993806  1909 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:39.994138  1576 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:39.996820  1769 raft_consensus.cc:359] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.997838  1769 raft_consensus.cc:385] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.997352  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:39.998304  1769 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.998876  1909 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:39.999074  1769 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:39.995311  2093 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.017127  2093 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.031718  1909 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:40.033365  1909 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.036043  1768 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:40.040004  1768 ts_tablet_manager.cc:1403] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.452s	user 0.023s	sys 0.016s
I20260709 13:00:40.040189  2093 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.051553  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:40.065351  1616 leader_election.cc:343] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.076431  1771 raft_consensus.cc:359] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.077636  1771 raft_consensus.cc:385] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:40.078169  1771 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:40.079337  1771 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.083423  1769 ts_tablet_manager.cc:1434] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.214s	user 0.030s	sys 0.033s
I20260709 13:00:40.048177  1723 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:40.072634  2132 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.104503  1768 raft_consensus.cc:359] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.109812  1768 raft_consensus.cc:385] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:40.111253  1768 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
W20260709 13:00:40.083988  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 124528us
I20260709 13:00:40.147385  1771 ts_tablet_manager.cc:1434] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.195s	user 0.010s	sys 0.003s
I20260709 13:00:40.160501  1768 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.165614  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:40.166705  1650 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:40.145533  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:40.177922  1768 ts_tablet_manager.cc:1434] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.135s	user 0.013s	sys 0.023s
I20260709 13:00:40.196292  1768 tablet_bootstrap.cc:492] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:40.176211  1723 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:40.182551  2110 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.178892  1573 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:40.204268  2110 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.222815  2110 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:40.237324  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:40.246863  1768 tablet_bootstrap.cc:654] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:40.247342  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:40.253297  1722 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.253756  2138 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.254287  2138 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.256719  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:40.258229  1771 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:40.258816  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:40.265834  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:40.274245  2147 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:40.277979  2100 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:40.278479  2147 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.278662  2100 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:40.278937  2147 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.263645  2138 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.275401  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:40.279675  2100 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:40.287784  1614 leader_election.cc:343] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.292976  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:00:40.297060  1614 leader_election.cc:343] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.286113  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:40.312067  1539 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:40.298877  2085 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:40.334380  2126 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:40.346873  1688 leader_election.cc:343] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.348073  1771 tablet_bootstrap.cc:654] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:40.347863  2085 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:40.361428  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:40.362162  1724 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:40.362412  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:40.362725  2148 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.365453  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:40.367830  1471 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:40.368800  2148 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.373740  2142 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:40.379562  1769 tablet_bootstrap.cc:492] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:40.374300  2142 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:40.398272  1690 leader_election.cc:343] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.388015  2142 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.398885  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:40.412866  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:40.421159  1574 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:40.421234  2145 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:40.429548  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:40.430272  1651 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:40.431740  2148 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.432727  2144 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:00:40.436689  2121 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:40.440852  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:40.447767  2135 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:40.448310  2135 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.448738  2135 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.459132  1769 tablet_bootstrap.cc:654] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:40.472438  1768 tablet_bootstrap.cc:492] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:40.473129  1768 ts_tablet_manager.cc:1403] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.281s	user 0.022s	sys 0.012s
I20260709 13:00:40.479852  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:40.493762  1539 leader_election.cc:343] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:40.515784  2153 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:00:40.516837  1768 raft_consensus.cc:359] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.517658  1768 raft_consensus.cc:385] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:40.517941  1768 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:40.489668  2147 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.520236  1768 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.543419  1768 ts_tablet_manager.cc:1434] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 13:00:40.563055  2138 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:00:40.568933  1771 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:40.569581  1771 ts_tablet_manager.cc:1403] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.312s	user 0.019s	sys 0.022s
I20260709 13:00:40.579000  2142 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.587646  1768 tablet_bootstrap.cc:492] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:40.568737  2135 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.618752  1769 tablet_bootstrap.cc:492] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:40.619549  1769 ts_tablet_manager.cc:1403] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.240s	user 0.018s	sys 0.011s
I20260709 13:00:40.623185  2142 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.622709  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:00:40.629359  2156 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 13:00:40.628976  1650 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.630293  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:40.631170  1573 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:40.639951  1614 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:40.640100  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:40.640810  1724 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.641456  1771 raft_consensus.cc:359] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.642241  1771 raft_consensus.cc:385] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:40.642504  1771 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:40.643363  1771 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.643908  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:40.644583  1720 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.645927  1771 ts_tablet_manager.cc:1434] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.003s
I20260709 13:00:40.659768  1724 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:40.636065  2147 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.666236  1771 tablet_bootstrap.cc:492] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:40.672068  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:40.701838  2131 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:40.703064  2135 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.707899  2131 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:40.730247  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:40.747727  1769 raft_consensus.cc:359] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.748869  1769 raft_consensus.cc:385] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:40.749202  1769 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
W20260709 13:00:40.750422  1539 leader_election.cc:343] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.751400  1769 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.750360  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:40.755764  1644 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.757831  1720 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:40.758952  1573 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:40.764973  1768 tablet_bootstrap.cc:654] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:40.765911  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:40.760124  2131 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.768163  2132 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:40.773684  1769 ts_tablet_manager.cc:1434] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.154s	user 0.009s	sys 0.001s
I20260709 13:00:40.768620  2138 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:40.774596  2138 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:40.775063  1650 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:40.775202  1769 tablet_bootstrap.cc:492] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:40.775775  2138 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.807617  2148 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.808476  2148 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.811221  2148 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.835201  1771 tablet_bootstrap.cc:654] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:40.847421  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:40.848201  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:40.848311  1719 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:40.848940  1645 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:40.856159  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:40.864652  2148 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:40.837383  1472 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:00:40.865227  2148 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.865461  2110 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.872884  2110 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.876264  2110 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:40.879060  2148 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.882434  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:40.887867  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:40.891371  2134 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:40.864244  1471 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:00:40.898451  1688 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:40.899451  2145 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:40.844266  1644 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:40.919545  1913 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.920217  1913 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.906857  2148 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.922489  1913 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.922858  2148 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:40.921934  1472 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:40.968849  1724 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:40.971444  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 61501.9us
I20260709 13:00:40.972749  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:40.974857  1913 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.975449  1913 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:40.982227  2148 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:40.983577  1769 tablet_bootstrap.cc:654] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:40.984155  2148 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:40.986090  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:40.986992  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:40.987696  1723 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:40.989418  1539 leader_election.cc:343] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:40.992033  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:40.992717  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:40.993232  1645 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:40.992121  2148 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.024554  1768 tablet_bootstrap.cc:492] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:41.039784  1913 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:41.047753  1539 leader_election.cc:343] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.052124  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:41.011296  1473 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:41.059450  1769 tablet_bootstrap.cc:492] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:41.059966  1769 ts_tablet_manager.cc:1403] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.285s	user 0.015s	sys 0.004s
I20260709 13:00:41.060637  1913 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:41.054121  1771 tablet_bootstrap.cc:492] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:41.049099  1768 ts_tablet_manager.cc:1403] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.466s	user 0.034s	sys 0.037s
I20260709 13:00:41.054755  1723 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:41.063009  1771 ts_tablet_manager.cc:1403] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.397s	user 0.020s	sys 0.018s
I20260709 13:00:41.066020  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:41.066417  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:41.067451  1768 raft_consensus.cc:359] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.068187  1768 raft_consensus.cc:385] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.068483  1768 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.056596  2142 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.069234  2142 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.069365  1768 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.070339  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:41.072542  1768 ts_tablet_manager.cc:1434] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:00:41.073724  1769 raft_consensus.cc:359] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.074671  1769 raft_consensus.cc:385] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.075013  1769 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.075891  1769 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.076550  1650 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:41.071781  2142 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.082175  1771 raft_consensus.cc:359] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.084043  1771 raft_consensus.cc:385] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.084398  1771 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.085139  1771 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.087539  1768 tablet_bootstrap.cc:492] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:41.089903  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:41.090739  1573 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:41.092092  1771 ts_tablet_manager.cc:1434] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:00:41.093467  1771 tablet_bootstrap.cc:492] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:41.058403  1645 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:41.100406  1769 ts_tablet_manager.cc:1434] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.002s
I20260709 13:00:41.108387  1771 tablet_bootstrap.cc:654] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.109172  1769 tablet_bootstrap.cc:492] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:41.125840  1769 tablet_bootstrap.cc:654] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.126808  1768 tablet_bootstrap.cc:654] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:41.133339  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 67455.8us
I20260709 13:00:41.134156  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:41.135131  1913 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:41.135703  1913 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:41.141489  1913 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:41.145506  1614 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.146404  2145 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.146977  2145 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.156026  2145 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:41.158720  1472 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:41.159844  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:41.160439  1771 tablet_bootstrap.cc:492] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:41.160661  1572 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:41.162299  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:41.163342  2145 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:41.163760  2145 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.164111  2145 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.168805  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:41.169251  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:41.169934  2134 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:41.169981  1644 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:41.170468  2134 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.171375  2134 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:41.168728  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.177597  1771 ts_tablet_manager.cc:1403] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.015s
I20260709 13:00:41.181100  1771 raft_consensus.cc:359] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.181895  1771 raft_consensus.cc:385] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.182188  1771 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.183396  1768 tablet_bootstrap.cc:492] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:41.183207  1771 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.184540  1768 ts_tablet_manager.cc:1403] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.011s
I20260709 13:00:41.185030  1771 ts_tablet_manager.cc:1434] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:00:41.192574  1771 tablet_bootstrap.cc:492] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:41.200810  1769 tablet_bootstrap.cc:492] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:41.201442  1769 ts_tablet_manager.cc:1403] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.093s	user 0.014s	sys 0.002s
I20260709 13:00:41.196086  2134 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.205603  1771 tablet_bootstrap.cc:654] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.210655  1768 raft_consensus.cc:359] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.211714  1768 raft_consensus.cc:385] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.212041  1768 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.213438  1768 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.214761  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:41.212212  2148 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:41.215950  2148 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.219746  2148 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:41.216115  1768 ts_tablet_manager.cc:1434] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.027s	user 0.000s	sys 0.007s
I20260709 13:00:41.221977  1768 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:41.204797  1769 raft_consensus.cc:359] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.222718  1769 raft_consensus.cc:385] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.223130  1769 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
W20260709 13:00:41.218994  1614 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.224148  1769 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.233969  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:41.202785  2145 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.237483  1769 ts_tablet_manager.cc:1434] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
W20260709 13:00:41.237672  1539 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.239842  1769 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:41.245290  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:41.245714  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:41.246582  1572 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.248118  2145 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:41.246512  1650 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.258320  2134 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.260200  1769 tablet_bootstrap.cc:654] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.252393  2148 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.266884  1771 tablet_bootstrap.cc:492] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:41.267447  1771 ts_tablet_manager.cc:1403] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.075s	user 0.038s	sys 0.010s
I20260709 13:00:41.279178  2148 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.282301  1573 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:00:41.285841  1650 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:41.287309  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.282301  1771 raft_consensus.cc:359] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.288956  1771 raft_consensus.cc:385] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.289314  1771 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.289095  2011 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:41.290143  1771 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.290833  2011 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:41.291993  2011 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.294520  1771 ts_tablet_manager.cc:1434] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 13:00:41.297557  1572 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:41.300279  1768 tablet_bootstrap.cc:654] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.308760  1771 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:41.305806  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:41.317334  1645 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:41.311401  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:41.331687  2134 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:41.334153  2138 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.334717  2138 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.340395  1473 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:41.342888  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:41.344900  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:41.348731  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:41.352820  1769 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:41.353458  1769 ts_tablet_manager.cc:1403] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.114s	user 0.013s	sys 0.023s
I20260709 13:00:41.355885  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:41.360189  1724 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:41.362592  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.348137  1572 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:41.364607  2142 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:41.365015  2142 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.365422  2142 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.345616  2138 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:41.373133  1539 leader_election.cc:343] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.373899  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:41.374858  1913 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:41.379511  1771 tablet_bootstrap.cc:654] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.380491  1769 raft_consensus.cc:359] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.381310  1769 raft_consensus.cc:385] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.381637  1769 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.382754  1768 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:41.382421  1769 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.383706  1768 ts_tablet_manager.cc:1403] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.162s	user 0.011s	sys 0.006s
I20260709 13:00:41.384294  2142 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.386001  1769 ts_tablet_manager.cc:1434] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 13:00:41.387099  2148 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:41.389045  1769 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:41.389870  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:41.390619  1572 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.391654  1768 raft_consensus.cc:359] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.392542  1768 raft_consensus.cc:385] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.392885  1768 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.393916  1768 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.398820  1769 tablet_bootstrap.cc:654] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.406828  2134 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:41.408615  2134 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.409225  2134 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.411283  1472 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:41.413486  1572 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:41.414981  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.417172  2138 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:41.417531  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:41.417774  2138 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:41.418309  1719 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:41.418746  2138 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.420274  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.421202  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:41.421532  2100 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:41.421937  1573 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:41.421964  2100 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.422544  2100 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.425266  2142 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.442333  2148 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:41.416424  2134 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.452692  1768 ts_tablet_manager.cc:1434] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 13:00:41.459589  1768 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:41.461964  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:41.482577  1723 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.494962  1723 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:41.529016  1913 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:41.575870  1769 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:41.576923  1769 ts_tablet_manager.cc:1403] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.189s	user 0.012s	sys 0.020s
W20260709 13:00:41.595049  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 93804.9us
I20260709 13:00:41.597177  1771 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:41.597776  1771 ts_tablet_manager.cc:1403] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.289s	user 0.028s	sys 0.028s
W20260709 13:00:41.603513  1539 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:41.613651  2100 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.584116  2125 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:00:41.615285  2192 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:41.616097  2100 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.633538  2186 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.634409  2186 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.634975  1771 raft_consensus.cc:359] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.635746  1771 raft_consensus.cc:385] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.636081  1771 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.656693  1768 tablet_bootstrap.cc:654] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.655851  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:41.664745  1569 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
W20260709 13:00:41.669941  2188 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:00:41.653505  2186 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:41.702816  1771 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.703995  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:41.704669  1724 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.703920  1769 raft_consensus.cc:359] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.710695  1769 raft_consensus.cc:385] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.711134  1769 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.712149  1769 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.717804  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.722100  1769 ts_tablet_manager.cc:1434] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.018s
I20260709 13:00:41.726058  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:41.732544  1771 ts_tablet_manager.cc:1434] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.134s	user 0.013s	sys 0.013s
I20260709 13:00:41.740365  1771 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:41.740518  2100 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:41.778561  1769 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:41.769585  2100 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:41.810400  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45259.9us
I20260709 13:00:41.822830  1913 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:41.795743  2100 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:41.866600  1724 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:00:41.874926  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60313.8us
I20260709 13:00:41.893697  1768 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:41.894199  1768 ts_tablet_manager.cc:1403] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.435s	user 0.022s	sys 0.016s
W20260709 13:00:41.909178  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 131122us
W20260709 13:00:41.912240  1614 leader_election.cc:343] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.914950  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:41.918042  1771 tablet_bootstrap.cc:654] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.918901  2142 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:41.919711  2142 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:41.921057  2142 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:41.934288  2202 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.957340  2202 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:41.935635  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 164914us
I20260709 13:00:41.943502  2145 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 13:00:41.949606  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:42.023689  1768 raft_consensus.cc:359] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.001526  1570 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.051219  1769 tablet_bootstrap.cc:654] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.052453  1771 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:42.030494  1768 raft_consensus.cc:385] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.060959  2202 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:42.063958  1768 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.064002  1771 ts_tablet_manager.cc:1403] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.324s	user 0.029s	sys 0.017s
I20260709 13:00:42.069310  1768 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:42.063973  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66026.8us
I20260709 13:00:42.096771  1768 ts_tablet_manager.cc:1434] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.202s	user 0.013s	sys 0.002s
I20260709 13:00:42.105624  1768 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:42.112849  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:42.127764  1771 raft_consensus.cc:359] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.129380  1771 raft_consensus.cc:385] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.129904  1771 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
W20260709 13:00:42.127851  1614 leader_election.cc:343] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.168753  1771 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.123378  1570 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:42.165889  1473 catalog_manager.cc:5697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:42.198516  1771 ts_tablet_manager.cc:1434] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.128s	user 0.011s	sys 0.001s
I20260709 13:00:42.221078  1725 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.193495  2208 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.228227  1771 tablet_bootstrap.cc:492] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:42.211630  2212 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:00:42.242457  1769 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:42.243623  1769 ts_tablet_manager.cc:1403] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.472s	user 0.030s	sys 0.018s
I20260709 13:00:42.231072  2208 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.270314  1768 tablet_bootstrap.cc:654] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.258283  2100 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.269294  2209 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.303200  2209 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.336856  1769 raft_consensus.cc:359] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.351405  1769 raft_consensus.cc:385] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.351925  1769 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.360400  1769 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.312988  2142 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:00:42.378153  2100 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:42.379925  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:42.401646  1769 ts_tablet_manager.cc:1434] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.156s	user 0.010s	sys 0.001s
I20260709 13:00:42.405951  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:42.406781  1573 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:42.416409  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:42.426862  1769 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:42.428303  1771 tablet_bootstrap.cc:654] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.518069  1768 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:42.518936  1768 ts_tablet_manager.cc:1403] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.414s	user 0.010s	sys 0.019s
I20260709 13:00:42.525044  2110 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:42.514375  1573 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:00:42.541009  1614 leader_election.cc:343] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.528002  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:42.534797  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:42.535262  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:42.546566  2208 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:42.554512  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:00:42.558166  1539 leader_election.cc:343] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.554550  1575 raft_consensus.cc:2368] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2.
I20260709 13:00:42.588514  2216 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:42.588367  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:42.589705  1647 raft_consensus.cc:2368] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2.
I20260709 13:00:42.587177  1768 raft_consensus.cc:359] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.591418  1768 raft_consensus.cc:385] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.592327  1768 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.593358  2209 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:42.593703  1768 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.597287  1768 ts_tablet_manager.cc:1434] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.078s	user 0.027s	sys 0.001s
I20260709 13:00:42.599290  1768 tablet_bootstrap.cc:492] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:42.624423  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:42.633819  2216 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:00:42.637115  1539 leader_election.cc:343] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:42.626820  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60736.4us
I20260709 13:00:42.638294  1539 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:42.643291  1913 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.640924  2216 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.662344  1570 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
I20260709 13:00:42.664115  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:42.665030  2142 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:42.665483  2142 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.665879  2142 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.776881  1688 leader_election.cc:400] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2.
I20260709 13:00:42.799500  1688 leader_election.cc:403] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:42.807367  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78428.9us
I20260709 13:00:42.814862  1771 tablet_bootstrap.cc:492] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:42.815490  1771 ts_tablet_manager.cc:1403] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.603s	user 0.099s	sys 0.100s
I20260709 13:00:42.819139  1768 tablet_bootstrap.cc:654] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.827564  1690 leader_election.cc:400] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2.
I20260709 13:00:42.790601  2195 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:00:42.829314  2142 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:42.853982  2142 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:42.829399  2145 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:00:42.890445  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:42.899602  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58779.7us
I20260709 13:00:42.906783  2245 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:42.880537  2212 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:42.911588  1614 leader_election.cc:343] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:43.012454  2232 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:00:43.029872  1771 raft_consensus.cc:359] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.031538  1771 raft_consensus.cc:385] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.034183  1771 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
W20260709 13:00:43.032481  2236 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:42.917223  2212 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:43.040247  2240 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:00:43.059876  1771 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.062024  1769 tablet_bootstrap.cc:654] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.084869  1771 ts_tablet_manager.cc:1434] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.269s	user 0.009s	sys 0.002s
I20260709 13:00:43.107249  2212 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.151687  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:43.153627  2145 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2.
W20260709 13:00:43.175966  1614 leader_election.cc:343] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:43.209937  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 153085us
I20260709 13:00:43.222864  1768 tablet_bootstrap.cc:492] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:43.223546  1768 ts_tablet_manager.cc:1403] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.625s	user 0.014s	sys 0.054s
W20260709 13:00:43.220111  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 416365us
I20260709 13:00:43.216323  2193 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.227669  1769 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:43.229152  1769 ts_tablet_manager.cc:1403] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.815s	user 0.011s	sys 0.017s
I20260709 13:00:43.230945  2193 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.235778  1769 raft_consensus.cc:359] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.236873  1769 raft_consensus.cc:385] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.237380  1769 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.238061  1768 raft_consensus.cc:359] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.238224  1769 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.239611  2193 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.239925  1769 ts_tablet_manager.cc:1434] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
W20260709 13:00:43.279989  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68857.2us
W20260709 13:00:43.293845  1539 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:00:43.313939  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:43.326896  1768 raft_consensus.cc:385] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.327606  1768 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.327291  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:43.329231  1768 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.332963  1768 ts_tablet_manager.cc:1434] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.005s
I20260709 13:00:43.337518  1769 tablet_bootstrap.cc:492] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:43.338049  1771 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:43.343643  1473 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 106) took 1563 ms. Trace:
I20260709 13:00:43.319422  1571 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.376297  2145 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.377571  2145 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:43.415494  1616 leader_election.cc:343] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.419466  1616 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:43.421854  2202 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.421685  2145 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:43.427178  1768 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:43.346446  1473 rpcz_store.cc:276] 0709 13:00:41.800698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.805031 (+  4333us) service_pool.cc:224] Handling call
0709 13:00:43.343152 (+1538121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:42.286331 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:42.408917 (+122586us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:42.412861 (+  3944us) write_op.cc:435] Acquired schema lock
0709 13:00:42.416452 (+  3591us) tablet.cc:662] Decoding operations
0709 13:00:42.440484 (+ 24032us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:42.461737 (+ 21253us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:42.464695 (+  2958us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:42.474822 (+ 10127us) tablet.cc:701] Row locks acquired
0709 13:00:42.474866 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:00:42.475249 (+   383us) write_op.cc:270] Start()
0709 13:00:42.475434 (+   185us) write_op.cc:276] Timestamp: P: 1783602042475057 usec, L: 0
0709 13:00:42.475460 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:00:42.500815 (+ 25355us) log.cc:844] Serialized 540 byte log entry
0709 13:00:42.822471 (+321656us) op_driver.cc:464] REPLICATION: finished
0709 13:00:43.179381 (+356910us) write_op.cc:301] APPLY: starting
0709 13:00:43.189177 (+  9796us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:43.199502 (+ 10325us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:43.204871 (+  5369us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:43.205161 (+   290us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:43.205343 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:43.205394 (+    51us) write_op.cc:312] APPLY: finished
0709 13:00:43.205601 (+   207us) log.cc:844] Serialized 22 byte log entry
0709 13:00:43.205829 (+   228us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:43.205918 (+    89us) write_op.cc:454] Released schema lock
0709 13:00:43.205962 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:43.206329 (+   367us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:43.206359 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":354423,"apply.run_cpu_time_us":2508,"apply.run_wall_time_us":26957,"num_ops":1,"prepare.queue_time_us":90124,"prepare.run_cpu_time_us":10589,"prepare.run_wall_time_us":233998,"raft.queue_time_us":180234,"raft.run_cpu_time_us":84290,"raft.run_wall_time_us":352551,"replication_time_us":318798,"thread_start_us":499791,"threads_started":4,"wal-append.queue_time_us":88969}]]}
I20260709 13:00:43.436606  1769 tablet_bootstrap.cc:654] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.450776  1472 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 106) took 1270 ms. Trace:
I20260709 13:00:43.451397  1472 rpcz_store.cc:276] 0709 13:00:42.179935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.180189 (+   254us) service_pool.cc:224] Handling call
0709 13:00:43.450323 (+1270134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64106,"reported_tablets":2}
I20260709 13:00:43.463225  2195 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:43.465714  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:43.480455  1571 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:43.495973  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:43.496999  1643 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:00:43.528187  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:43.529461  1725 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:00:43.534271  1723 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.540858  1768 tablet_bootstrap.cc:654] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.545181  1771 tablet_bootstrap.cc:654] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.551209  2195 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.555245  2195 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.588548  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:43.599398  1769 tablet_bootstrap.cc:492] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:43.600144  1769 ts_tablet_manager.cc:1403] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.264s	user 0.020s	sys 0.033s
W20260709 13:00:43.606566  1614 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:43.602870  2195 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.613363  1769 raft_consensus.cc:359] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.614303  1769 raft_consensus.cc:385] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.614614  1769 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.619815  1768 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:43.620497  1768 ts_tablet_manager.cc:1403] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.211s	user 0.023s	sys 0.005s
I20260709 13:00:43.632078  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:43.636616  1769 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.639433  1769 ts_tablet_manager.cc:1434] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:00:43.641242  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:43.642125  1573 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
W20260709 13:00:43.644372  1614 leader_election.cc:343] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.648042  1472 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:00:43.648612  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:43.632175  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:43.652362  2011 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.653301  1769 tablet_bootstrap.cc:492] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:43.664093  2192 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:43.664811  2192 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.665273  2192 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:43.676213  1768 raft_consensus.cc:359] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.678625  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:43.683490  2194 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.684616  1768 raft_consensus.cc:385] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.684754  2194 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:43.685123  1768 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.685920  2194 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.686668  1771 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:43.687273  1771 ts_tablet_manager.cc:1403] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.350s	user 0.007s	sys 0.022s
I20260709 13:00:43.693202  1771 raft_consensus.cc:359] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.694103  1771 raft_consensus.cc:385] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.694480  1771 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.698355  1771 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.694912  1768 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.711165  1768 ts_tablet_manager.cc:1434] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.003s
I20260709 13:00:43.713966  1769 tablet_bootstrap.cc:654] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.720988  1616 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:43.731329  2241 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.731920  2241 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.732311  2241 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.737007  1771 ts_tablet_manager.cc:1434] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.002s
I20260709 13:00:43.752538  1771 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:43.721493  2192 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:43.755599  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:43.755846  1768 tablet_bootstrap.cc:492] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:43.779831  2192 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.784787  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:43.785543  1724 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:43.797791  1724 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:00:43.801295  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:43.803210  2218 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:43.804088  2218 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:43.809882  1768 tablet_bootstrap.cc:654] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.816215  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:43.818280  1642 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:43.819324  1769 tablet_bootstrap.cc:492] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:43.820092  1769 ts_tablet_manager.cc:1403] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.170s	user 0.021s	sys 0.003s
I20260709 13:00:43.769374  2241 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.829157  2225 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.829864  2225 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.832888  2225 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.808387  2218 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.839639  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:43.842006  1539 leader_election.cc:343] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.841481  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:43.845535  2226 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:43.846711  1539 leader_election.cc:343] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.846153  2226 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.847362  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:43.844790  1769 raft_consensus.cc:359] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.849037  1769 raft_consensus.cc:385] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.849367  1769 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.851701  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:43.852948  1572 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.854750  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:00:43.856968  1614 leader_election.cc:343] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.858424  1769 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.859856  2241 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.855012  2225 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.873854  2226 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.881119  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:43.883558  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:00:43.886837  1539 leader_election.cc:343] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:43.892383  1539 leader_election.cc:343] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.896051  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:43.898466  2192 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.903927  2249 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.904911  2249 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.907683  2249 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:43.911263  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:00:43.913646  1616 leader_election.cc:343] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.915284  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:00:43.917495  1614 leader_election.cc:343] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.918084  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:43.918540  1769 ts_tablet_manager.cc:1434] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.098s	user 0.008s	sys 0.003s
I20260709 13:00:43.922444  1771 tablet_bootstrap.cc:654] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.919468  2195 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.929605  1768 tablet_bootstrap.cc:492] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:43.930263  1768 ts_tablet_manager.cc:1403] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.175s	user 0.019s	sys 0.002s
I20260709 13:00:43.903491  1572 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:00:43.942796  1769 tablet_bootstrap.cc:492] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:43.961956  1616 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:43.973888  1768 raft_consensus.cc:359] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.974689  1768 raft_consensus.cc:385] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.974961  1768 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.965382  2194 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.964861  1642 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:00:43.979030  1768 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.979820  2194 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:43.989501  1768 ts_tablet_manager.cc:1434] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.001s
I20260709 13:00:43.995364  2194 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:43.997555  2011 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.998279  2011 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.004554  2011 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:44.004884  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:44.007530  1647 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:44.006217  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:44.010221  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:44.012434  2011 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.012957  2011 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.013401  2011 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.014329  1690 leader_election.cc:343] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.008876  1768 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:44.034138  1474 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.035804  1768 tablet_bootstrap.cc:654] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.042210  1769 tablet_bootstrap.cc:654] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.047272  1771 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:44.049324  1771 ts_tablet_manager.cc:1403] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.297s	user 0.008s	sys 0.010s
I20260709 13:00:44.053192  1771 raft_consensus.cc:359] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.054216  1771 raft_consensus.cc:385] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.054680  1771 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.064064  1768 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:44.064702  1768 ts_tablet_manager.cc:1403] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.003s
I20260709 13:00:44.069455  1768 raft_consensus.cc:359] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.070307  1768 raft_consensus.cc:385] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.070622  1768 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.071492  1768 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.072762  1769 tablet_bootstrap.cc:492] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:44.074151  1769 ts_tablet_manager.cc:1403] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.138s	user 0.016s	sys 0.003s
I20260709 13:00:44.074746  1768 ts_tablet_manager.cc:1434] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:00:44.076753  1768 tablet_bootstrap.cc:492] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:44.084510  1769 raft_consensus.cc:359] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.084198  1472 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 3, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.085407  1768 tablet_bootstrap.cc:654] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.085606  1769 raft_consensus.cc:385] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.086273  1769 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.107317  1768 tablet_bootstrap.cc:492] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:44.107930  1768 ts_tablet_manager.cc:1403] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.003s
I20260709 13:00:44.108253  2011 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.143553  2011 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:44.112033  1768 raft_consensus.cc:359] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.144371  1768 raft_consensus.cc:385] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.144724  1768 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.145483  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:44.146281  1646 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.147843  1769 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.151093  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:44.151363  1474 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.153436  1573 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.154649  1474 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.110284  1771 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.158684  1771 ts_tablet_manager.cc:1434] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.001s
I20260709 13:00:44.159973  1771 tablet_bootstrap.cc:492] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:44.160667  1769 ts_tablet_manager.cc:1434] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.086s	user 0.009s	sys 0.000s
I20260709 13:00:44.161926  1769 tablet_bootstrap.cc:492] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:44.169147  1771 tablet_bootstrap.cc:654] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.169448  1769 tablet_bootstrap.cc:654] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.170508  1768 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.174186  1646 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:44.182046  1771 tablet_bootstrap.cc:492] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:44.182139  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:44.183892  2145 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.183878  1768 ts_tablet_manager.cc:1434] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.076s	user 0.012s	sys 0.000s
I20260709 13:00:44.184617  2145 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:44.185810  1768 tablet_bootstrap.cc:492] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Bootstrap starting.
I20260709 13:00:44.188946  2218 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.189687  2218 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.189087  1771 ts_tablet_manager.cc:1403] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.004s
I20260709 13:00:44.194813  2218 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.197393  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:44.198263  1642 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:44.198946  1771 raft_consensus.cc:359] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.199718  1771 raft_consensus.cc:385] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.200881  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:44.201035  2011 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.201704  2011 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.202781  2192 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.203166  2192 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.203576  2192 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.203773  1768 tablet_bootstrap.cc:654] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.203915  2011 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:44.205152  1771 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.205986  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:44.188645  2145 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.206061  1771 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.213550  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:44.213917  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:44.214519  1576 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 1: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:00:44.215085  2192 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.215760  1690 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:44.203140  1769 tablet_bootstrap.cc:492] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:44.217450  1769 ts_tablet_manager.cc:1403] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.005s
I20260709 13:00:44.222168  1771 ts_tablet_manager.cc:1434] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:00:44.223480  1771 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
W20260709 13:00:44.223992  1539 leader_election.cc:343] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.218300  2011 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.232436  2192 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.236703  2011 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.246189  1771 tablet_bootstrap.cc:654] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.237380  1769 raft_consensus.cc:359] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.239310  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:44.261780  1769 raft_consensus.cc:385] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.262295  1771 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:44.262014  1646 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.262564  1769 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.262974  1771 ts_tablet_manager.cc:1403] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.003s
I20260709 13:00:44.264602  2202 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.266623  1771 raft_consensus.cc:359] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.267457  1771 raft_consensus.cc:385] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.267721  1771 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.268859  1771 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.271374  1771 ts_tablet_manager.cc:1434] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:44.272537  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:44.274132  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:44.274899  2192 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.275401  2192 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:44.276276  1771 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:44.276795  2192 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.272572  1769 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.286834  1771 tablet_bootstrap.cc:654] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.289698  1769 ts_tablet_manager.cc:1434] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.072s	user 0.004s	sys 0.004s
I20260709 13:00:44.306810  2202 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.310113  1473 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:44.313124  2244 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.313124  2195 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.313814  2244 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.314064  2195 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.319422  2244 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.322271  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:44.323064  1646 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:44.324487  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:44.325449  2244 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.325831  2244 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.326174  2244 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.333726  1769 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: Bootstrap starting.
I20260709 13:00:44.345496  1771 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:44.345846  1573 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:44.346108  1771 ts_tablet_manager.cc:1403] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.004s
I20260709 13:00:44.349406  1771 raft_consensus.cc:359] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.350703  1771 raft_consensus.cc:385] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.351047  1771 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.349539  1474 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.352150  1771 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.355991  1771 ts_tablet_manager.cc:1434] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:00:44.358696  1725 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.363003  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:44.364480  2195 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.364885  1771 tablet_bootstrap.cc:492] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:44.370688  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:44.386973  1647 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 13:00:44.397667  2011 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.398248  2011 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.398581  2226 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.397413  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:44.402293  2011 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:44.401918  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:44.403632  1573 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 0.
I20260709 13:00:44.405045  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:44.411019  1769 tablet_bootstrap.cc:654] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.400807  2244 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.412871  2145 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.420895  2145 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.421350  2145 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.422621  1723 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
W20260709 13:00:44.429944  1616 leader_election.cc:343] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.441805  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:44.442632  1722 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.433143  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:44.445397  1647 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.445708  1614 leader_election.cc:343] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.446223  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:44.449642  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:44.430250  2244 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:00:44.451968  1539 leader_election.cc:343] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.453846  2249 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.432268  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:44.462347  2244 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 13:00:44.474051  1688 leader_election.cc:343] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.486027  1768 tablet_bootstrap.cc:492] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: No bootstrap required, opened a new log
I20260709 13:00:44.486776  1768 ts_tablet_manager.cc:1403] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Time spent bootstrapping tablet: real 0.301s	user 0.067s	sys 0.029s
I20260709 13:00:44.494956  1771 tablet_bootstrap.cc:654] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.521520  2194 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.524230  1722 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:44.552878  2241 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.553097  2145 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.555591  2241 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.558099  2241 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.566897  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:44.568838  1573 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 0.
W20260709 13:00:44.572605  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 71337.8us
I20260709 13:00:44.574383  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:44.581718  2145 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:44.590492  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:44.591715  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:44.592475  1572 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.590646  1769 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: No bootstrap required, opened a new log
I20260709 13:00:44.597489  1769 ts_tablet_manager.cc:1403] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: Time spent bootstrapping tablet: real 0.264s	user 0.014s	sys 0.027s
W20260709 13:00:44.598527  1614 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:00:44.599145  1688 leader_election.cc:343] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.598096  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:44.600857  1614 leader_election.cc:343] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.602247  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:00:44.600023  1768 raft_consensus.cc:359] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.603611  1768 raft_consensus.cc:385] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.604324  1768 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.601480  1647 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:00:44.615146  1768 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.621781  2215 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.605566  2192 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.655911  1771 tablet_bootstrap.cc:492] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:44.679838  2192 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:44.680008  1771 ts_tablet_manager.cc:1403] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.323s	user 0.032s	sys 0.018s
I20260709 13:00:44.686141  1768 ts_tablet_manager.cc:1434] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Time spent starting tablet: real 0.199s	user 0.011s	sys 0.000s
I20260709 13:00:44.687294  1771 raft_consensus.cc:359] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.687121  2192 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.687995  1771 raft_consensus.cc:385] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.688289  1771 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.683130  1769 raft_consensus.cc:359] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.697221  1769 raft_consensus.cc:385] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.697741  1769 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.698763  1769 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.693127  1771 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.679260  1572 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:00:44.715889  1771 ts_tablet_manager.cc:1434] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:00:44.717770  1771 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:44.681222  2215 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.727061  2215 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.741034  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49687.3us
I20260709 13:00:44.744385  2200 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.745613  1769 ts_tablet_manager.cc:1434] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: Time spent starting tablet: real 0.148s	user 0.003s	sys 0.019s
I20260709 13:00:44.778266  1771 tablet_bootstrap.cc:654] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.788722  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:44.801877  2296 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.797612  2244 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:00:44.829268  2215 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.839283  2297 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:44.842547  1721 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.864264  2296 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:00:44.879261  2296 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.884330  2244 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.885040  2244 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.887382  2244 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.890156  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:44.895131  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:44.895994  1642 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 0.
I20260709 13:00:44.899127  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:00:44.900627  2290 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.902343  2290 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.902797  2290 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.903177  2290 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.904836  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:00:44.905517  1573 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.906507  2215 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.917656  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:44.923897  1573 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:00:44.948014  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 109308us
W20260709 13:00:44.971362  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45921us
I20260709 13:00:44.975214  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:00:44.972627  1539 leader_election.cc:343] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:44.982445  1614 leader_election.cc:343] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.985060  2215 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.996284  2290 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:44.998448  2215 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:00:44.999221  2290 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:44.999553  2215 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.010152  1474 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:45.036432  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50989.6us
I20260709 13:00:45.051936  1771 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:45.052647  1771 ts_tablet_manager.cc:1403] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.335s	user 0.022s	sys 0.027s
I20260709 13:00:45.040316  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:00:45.060798  1651 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.070822  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:00:45.076057  1771 raft_consensus.cc:359] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.098286  1771 raft_consensus.cc:385] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.098796  1771 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.099028  1651 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
W20260709 13:00:45.101224  1522 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:45.102620  1771 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:00:45.137437  1539 leader_election.cc:343] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.157917  1771 ts_tablet_manager.cc:1434] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.002s
I20260709 13:00:45.178354  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:45.179232  2226 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:45.180064  2226 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:00:45.181228  2226 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.193176  1723 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.198493  1771 tablet_bootstrap.cc:492] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:45.194449  2249 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.214668  2293 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:00:45.206380  2249 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.222013  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:45.222970  1574 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 1: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:00:45.257234  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57848.6us
I20260709 13:00:45.262511  2249 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:45.263692  1771 tablet_bootstrap.cc:654] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.265859  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:00:45.273082  1614 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:00:45.276527  2194 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.370448  1771 tablet_bootstrap.cc:492] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
W20260709 13:00:45.372653  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71602.2us
I20260709 13:00:45.372047  1771 ts_tablet_manager.cc:1403] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.177s	user 0.022s	sys 0.009s
I20260709 13:00:45.388902  2194 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:45.400214  1473 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:45.423349  2318 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.427572  2307 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.436486  1771 raft_consensus.cc:359] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.437378  1771 raft_consensus.cc:385] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.437811  1771 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.442909  1771 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.445757  1771 ts_tablet_manager.cc:1434] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 13:00:45.447170  1771 tablet_bootstrap.cc:492] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Bootstrap starting.
I20260709 13:00:45.471817  1771 tablet_bootstrap.cc:654] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.482548  1474 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:45.490257  1614 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:45.495343  2242 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.501986  2307 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.547016  1771 tablet_bootstrap.cc:492] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: No bootstrap required, opened a new log
I20260709 13:00:45.547966  1771 ts_tablet_manager.cc:1403] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent bootstrapping tablet: real 0.101s	user 0.025s	sys 0.028s
I20260709 13:00:45.570861  1771 raft_consensus.cc:359] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.571697  1771 raft_consensus.cc:385] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.572028  1771 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.591326  1472 catalog_manager.cc:5697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:45.591652  2316 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.593079  1771 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.598742  1771 ts_tablet_manager.cc:1434] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.002s
I20260709 13:00:45.604125  2301 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.604795  2301 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:00:45.608045  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:00:45.613888  2301 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:45.618721  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:00:45.621110  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:00:45.622285  2145 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:45.630015  1722 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.655920  2293 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:46.061326  1722 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.382337   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:46.415524  2333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:46.417309  2334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:46.419540  2336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:46.420881   431 server_base.cc:1061] running on GCE node
I20260709 13:00:46.424976   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:46.425485   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:46.425732   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602046425690 us; error 0 us; skew 500 ppm
I20260709 13:00:46.426427   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:46.431540   431 webserver.cc:533] Webserver started at http://127.0.107.196:33933/ using document root <none> and password file <none>
I20260709 13:00:46.433636   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:46.434019   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:46.435950   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:46.438519   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-3-root/instance:
uuid: "da3b57321031416cba9db75450ef302a"
format_stamp: "Formatted at 2026-07-09 13:00:46 on dist-test-slave-ftjc"
I20260709 13:00:46.445641   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.000s	sys 0.005s
I20260709 13:00:46.452098  2341 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:46.453879   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:00:46.454404   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-3-root
uuid: "da3b57321031416cba9db75450ef302a"
format_stamp: "Formatted at 2026-07-09 13:00:46 on dist-test-slave-ftjc"
I20260709 13:00:46.454865   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:46.479902   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:46.482385   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:46.484392   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:46.487308   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:46.487556   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:46.487792   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:46.487946   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:46.638842   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:45309
I20260709 13:00:46.638976  2404 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:45309 every 8 connection(s)
I20260709 13:00:46.646301   431 auto_leader_rebalancer-test.cc:384] add a tserver: da3b57321031416cba9db75450ef302a
I20260709 13:00:46.669991  2405 heartbeater.cc:344] Connected to a master server at 127.0.107.254:32927
I20260709 13:00:46.670882  2405 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:46.672214  2405 heartbeater.cc:507] Master 127.0.107.254:32927 requested a full tablet report, sending...
I20260709 13:00:46.675175  1474 ts_manager.cc:194] Registered new tserver with Master: da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:00:46.677289  1474 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37380
I20260709 13:00:47.264557  1573 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.275581  1720 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.275926  1643 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.278025  2293 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:47.279815  2316 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:47.309484  2316 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
W20260709 13:00:47.314832  1539 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8513b00817042619ae0bf0c8da58904. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:47.316690  1651 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.327767  1573 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.333034  1474 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:47.310262  1720 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.349555  1722 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.350481  1647 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.354212  2192 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:47.373203  1539 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fac0a563dc584b4fbe82f01f362fa966. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:47.374327  2192 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:47.398085  2316 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.407927  1647 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.462826  1472 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.463992  1717 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.486320  1647 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.542305  1572 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.544998  2194 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:47.549038  1717 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.551986  2352 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:47.558585  1614 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 363173567dd940e99e459f02148a42e0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:47.566682  2194 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.568717  1573 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.577246  1720 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.586766  1472 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:47.624681  1720 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.638829  1573 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.641346  1647 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:47.643337  2200 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:47.646781  2424 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:47.655637  1688 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c55c47030a294c17ad5561abe541b993. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:47.664156  2436 ts_tablet_manager.cc:933] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:47.668699  2200 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.677414  1647 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.680486  2405 heartbeater.cc:499] Master 127.0.107.254:32927 was elected leader, sending a full tablet report...
I20260709 13:00:47.683821  2436 tablet_copy_client.cc:323] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:00:47.721199  1474 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.743968  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet b8513b00817042619ae0bf0c8da58904 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:00:47.744745  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet b8513b00817042619ae0bf0c8da58904 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-b8513b00817042619ae0bf0c8da58904
I20260709 13:00:47.746799  1573 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:47.764302  1586 tablet_copy_source_session.cc:215] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:47.770466  2436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8513b00817042619ae0bf0c8da58904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:47.799628  2436 tablet_copy_client.cc:806] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:47.800863  2436 tablet_copy_client.cc:670] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:47.805965  2436 tablet_copy_client.cc:538] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:47.822194  2436 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:47.848695  2436 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:47.849982  2436 tablet_bootstrap.cc:492] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:47.850766  2436 ts_tablet_manager.cc:1403] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.009s
I20260709 13:00:47.854172  2436 raft_consensus.cc:359] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.855023  2436 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:47.855911  2436 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.860927  2436 ts_tablet_manager.cc:1434] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.002s
I20260709 13:00:47.864154  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-b8513b00817042619ae0bf0c8da58904 received from {username='slave'} at 127.0.0.1:45556
I20260709 13:00:47.864831  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-b8513b00817042619ae0bf0c8da58904 on tablet b8513b00817042619ae0bf0c8da58904 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:47.887991  2436 ts_tablet_manager.cc:933] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:47.895910  2436 tablet_copy_client.cc:323] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:00:47.903599  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet fac0a563dc584b4fbe82f01f362fa966 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:00:47.904245  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet fac0a563dc584b4fbe82f01f362fa966 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-fac0a563dc584b4fbe82f01f362fa966
I20260709 13:00:47.912243  1586 tablet_copy_source_session.cc:215] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:47.916538  2436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac0a563dc584b4fbe82f01f362fa966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:47.933548  2436 tablet_copy_client.cc:806] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:47.934710  2436 tablet_copy_client.cc:670] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:47.942853  2436 tablet_copy_client.cc:538] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:47.953754  2436 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:47.981886  2436 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:47.983140  2436 tablet_bootstrap.cc:492] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:47.983956  2436 ts_tablet_manager.cc:1403] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.008s
I20260709 13:00:47.986557  2436 raft_consensus.cc:359] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.987376  2436 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:47.988032  2436 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:47.990608  2436 ts_tablet_manager.cc:1434] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:00:47.993911  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-fac0a563dc584b4fbe82f01f362fa966 received from {username='slave'} at 127.0.0.1:45556
I20260709 13:00:47.994684  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-fac0a563dc584b4fbe82f01f362fa966 on tablet fac0a563dc584b4fbe82f01f362fa966 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:48.082762  2436 ts_tablet_manager.cc:933] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:48.085101  2436 tablet_copy_client.cc:323] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:48.108938  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet 363173567dd940e99e459f02148a42e0 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:48.109669  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet 363173567dd940e99e459f02148a42e0 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-363173567dd940e99e459f02148a42e0
I20260709 13:00:48.118407  1661 tablet_copy_source_session.cc:215] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:48.122365  2436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363173567dd940e99e459f02148a42e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.140676  2436 tablet_copy_client.cc:806] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:48.141260  2436 tablet_copy_client.cc:670] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:48.146914  2436 tablet_copy_client.cc:538] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:48.156436  2436 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:48.158612  2380 raft_consensus.cc:1217] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:48.198674  2436 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:48.199935  2436 tablet_bootstrap.cc:492] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:48.200769  2436 ts_tablet_manager.cc:1403] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.008s
I20260709 13:00:48.203878  2436 raft_consensus.cc:359] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:48.204901  2436 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:48.205559  2436 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:48.208395  2436 ts_tablet_manager.cc:1434] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:00:48.211845  1661 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-363173567dd940e99e459f02148a42e0 received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:48.212440  1661 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-363173567dd940e99e459f02148a42e0 on tablet 363173567dd940e99e459f02148a42e0 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:48.259513  2436 ts_tablet_manager.cc:933] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:48.261214  2436 tablet_copy_client.cc:323] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:00:48.275065  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet c55c47030a294c17ad5561abe541b993 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:00:48.275827  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet c55c47030a294c17ad5561abe541b993 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-c55c47030a294c17ad5561abe541b993
I20260709 13:00:48.290347  1735 tablet_copy_source_session.cc:215] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:48.301385  2436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c55c47030a294c17ad5561abe541b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:48.321059  2436 tablet_copy_client.cc:806] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:48.321837  2436 tablet_copy_client.cc:670] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:48.326087  2436 tablet_copy_client.cc:538] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:48.335867  2436 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:48.354475  2380 raft_consensus.cc:1217] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:48.371601  2436 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:48.372849  2436 tablet_bootstrap.cc:492] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:48.377107  2436 ts_tablet_manager.cc:1403] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.013s
I20260709 13:00:48.382167  2436 raft_consensus.cc:359] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:48.383332  2436 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:48.384004  2436 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:48.397233  2436 ts_tablet_manager.cc:1434] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:00:48.407408  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-c55c47030a294c17ad5561abe541b993 received from {username='slave'} at 127.0.0.1:60218
I20260709 13:00:48.408062  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-c55c47030a294c17ad5561abe541b993 on tablet c55c47030a294c17ad5561abe541b993 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:48.622929  2448 raft_consensus.cc:1064] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:48.626825  2448 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:48.634981  1720 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:48.637434  2380 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:48.637663  1651 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:48.640836  2428 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:48.642884  2316 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:48.643945  2428 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:48.654851  2380 raft_consensus.cc:1217] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:48.687209  2380 raft_consensus.cc:1217] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:00:48.695279  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45455.9us
I20260709 13:00:48.721930  2428 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.724841  2379 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.725543  1651 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.741941  1723 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.750267  1473 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.802103  2448 raft_consensus.cc:1064] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:48.806723  2448 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:48.822786  2379 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:48.823115  1717 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:48.825012  2450 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:48.826164  2192 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:48.826639  1651 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:48.837057  2192 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:48.914100  2450 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.921042  1717 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.925294  1572 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:48.942795  2380 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:48.944255  1723 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:48.947068  2192 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:48.949870  1647 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.951766  2428 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:48.951930  1473 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.958019  2379 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:48.991833  1572 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:48.997581  2316 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.026580  1473 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:49.027617  1717 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.037207  1647 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.031489  2379 raft_consensus.cc:2955] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.040894  2380 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.042887  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8513b00817042619ae0bf0c8da58904 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:49.043507  2192 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:49.045413  2192 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:49.052357  2316 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.058477  1647 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.069175  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fac0a563dc584b4fbe82f01f362fa966 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:49.073086  2380 raft_consensus.cc:2955] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.078389  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet b8513b00817042619ae0bf0c8da58904 with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:00:49.127975  2303 raft_consensus.cc:1064] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:49.131081  2303 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:49.161782  1643 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.164748  1572 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.166360  2379 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.166625  2303 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:49.168360  2453 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:49.169582  2200 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:49.170300  2465 tablet_replica.cc:333] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:00:49.171012  1474 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.192783  2352 raft_consensus.cc:1064] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:49.196043  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet fac0a563dc584b4fbe82f01f362fa966 with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:00:49.210067  2470 tablet_replica.cc:333] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:00:49.211391  2470 raft_consensus.cc:2243] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:49.214213  2470 raft_consensus.cc:2272] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:49.219015  2470 ts_tablet_manager.cc:1916] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:49.198942  2352 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:49.231937  2453 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.242699  1574 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.248706  2465 raft_consensus.cc:2243] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:49.249811  2465 raft_consensus.cc:2272] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:49.254748  2465 ts_tablet_manager.cc:1916] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:49.263952  1643 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.278564  2470 ts_tablet_manager.cc:1929] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:49.279546  2470 log.cc:1199] T fac0a563dc584b4fbe82f01f362fa966 P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/fac0a563dc584b4fbe82f01f362fa966
I20260709 13:00:49.279268  2379 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.292660  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet fac0a563dc584b4fbe82f01f362fa966 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:49.299122  2465 ts_tablet_manager.cc:1929] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:49.300083  2465 log.cc:1199] T b8513b00817042619ae0bf0c8da58904 P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/b8513b00817042619ae0bf0c8da58904
I20260709 13:00:49.302472  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet b8513b00817042619ae0bf0c8da58904 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:49.318464  1471 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:49.335100  2379 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.346165  1722 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:49.352931  1723 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.355383  2380 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.358727  1572 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:49.361622  2471 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:49.367045  2471 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:49.369380  1572 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.370237  2352 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:49.373171  2467 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:49.386236  2453 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:49.399080  2453 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.402403  2378 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.455725  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c55c47030a294c17ad5561abe541b993 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:49.456444  1573 raft_consensus.cc:2955] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.459676  1471 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.474885  2471 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.505519  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet c55c47030a294c17ad5561abe541b993 with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:00:49.506745  2465 tablet_replica.cc:333] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:00:49.507941  2465 raft_consensus.cc:2243] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:49.508857  2465 raft_consensus.cc:2272] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:49.514365  2465 ts_tablet_manager.cc:1916] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:49.519016  2380 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.525645  1572 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.537808  1721 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.585304  1472 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.609328  2465 ts_tablet_manager.cc:1929] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:49.609970  2465 log.cc:1199] T c55c47030a294c17ad5561abe541b993 P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/c55c47030a294c17ad5561abe541b993
I20260709 13:00:49.611888  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet c55c47030a294c17ad5561abe541b993 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:49.622419  1647 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:49.630812  2380 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.632851  1721 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:49.633868  2352 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:49.637738  2352 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:49.642829  2471 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.644847  2380 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.657275  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 363173567dd940e99e459f02148a42e0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:49.673966  1717 raft_consensus.cc:2955] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:49.685705  1471 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:49.718963  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet 363173567dd940e99e459f02148a42e0 with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44738
I20260709 13:00:49.733210  2480 tablet_replica.cc:333] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:00:49.734678  2480 raft_consensus.cc:2243] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:49.735622  2480 raft_consensus.cc:2272] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:49.740552  2480 ts_tablet_manager.cc:1916] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:49.760943  2480 ts_tablet_manager.cc:1929] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:49.761646  2480 log.cc:1199] T 363173567dd940e99e459f02148a42e0 P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/363173567dd940e99e459f02148a42e0
I20260709 13:00:49.763717  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet 363173567dd940e99e459f02148a42e0 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:50.794405  1720 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:50.803035  1572 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:50.803356  1688 consensus_peers.cc:597] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f6d77c909fe243b197ef61ea6b271908. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:50.803045  1647 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:50.805255  2467 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.806334  2483 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:50.817653  2479 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.821879  1647 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.824923  1574 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.837739  1651 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:50.855211  1574 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:50.855397  1717 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:50.857683  2474 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:50.866643  1614 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78d12cc2bc7b4654acd41e6c45eb9753. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:50.862314  1474 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.868757  2481 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:50.873189  2471 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.874420  1574 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.904542  1720 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.925719  1725 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:50.935932  1647 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:50.938500  1574 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:50.941256  2479 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:50.942590  2467 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:50.947016  1688 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c9e58595214defa013274c5193de1a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:50.976666  2483 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.984678  1471 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:50.998389  1574 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:51.031540  1474 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
W20260709 13:00:51.053496  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52002.8us
I20260709 13:00:51.052855  1574 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.063226  1651 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:51.077286  1647 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:51.082981  1725 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:51.086714  1539 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26917aa1f55f48918fedd4960446f38c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:51.094801  2464 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:51.098927  2464 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:51.115545  2502 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:51.120908  1725 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:51.123970  1651 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:51.145131  1474 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:51.309131  2505 ts_tablet_manager.cc:933] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:51.311779  2505 tablet_copy_client.cc:323] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:00:51.313997  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet f6d77c909fe243b197ef61ea6b271908 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:00:51.314605  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet f6d77c909fe243b197ef61ea6b271908 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-f6d77c909fe243b197ef61ea6b271908
I20260709 13:00:51.322101  1735 tablet_copy_source_session.cc:215] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:51.325459  2505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d77c909fe243b197ef61ea6b271908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:51.365332  2507 ts_tablet_manager.cc:933] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:00:51.373013  2505 tablet_copy_client.cc:806] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:51.373878  2507 tablet_copy_client.cc:323] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:00:51.373978  2505 tablet_copy_client.cc:670] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:51.391005  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet 53c9e58595214defa013274c5193de1a from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:00:51.393195  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet 53c9e58595214defa013274c5193de1a from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-53c9e58595214defa013274c5193de1a
I20260709 13:00:51.392526  2505 tablet_copy_client.cc:538] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:51.402698  1735 tablet_copy_source_session.cc:215] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:51.406101  2505 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:51.406597  2507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c9e58595214defa013274c5193de1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:51.430752  2507 tablet_copy_client.cc:806] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:51.431695  2507 tablet_copy_client.cc:670] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:51.444451  2507 tablet_copy_client.cc:538] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:51.447446  2510 ts_tablet_manager.cc:933] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:51.451797  2510 tablet_copy_client.cc:323] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:51.453892  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet 78d12cc2bc7b4654acd41e6c45eb9753 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:51.454648  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet 78d12cc2bc7b4654acd41e6c45eb9753 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-78d12cc2bc7b4654acd41e6c45eb9753
I20260709 13:00:51.459247  2507 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:51.462782  2505 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:51.464541  1661 tablet_copy_source_session.cc:215] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:51.468745  2510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d12cc2bc7b4654acd41e6c45eb9753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:51.473192  2505 tablet_bootstrap.cc:492] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:51.474056  2505 ts_tablet_manager.cc:1403] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.013s
I20260709 13:00:51.478966  2505 raft_consensus.cc:359] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.479866  2505 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:51.480830  2505 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.501362  2505 ts_tablet_manager.cc:1434] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.027s	user 0.016s	sys 0.001s
I20260709 13:00:51.593525  2514 ts_tablet_manager.cc:933] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:51.598066  2510 tablet_copy_client.cc:806] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:51.599342  2510 tablet_copy_client.cc:670] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:51.603370  2514 tablet_copy_client.cc:323] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:00:51.607635  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-f6d77c909fe243b197ef61ea6b271908 received from {username='slave'} at 127.0.0.1:60218
I20260709 13:00:51.608152  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-f6d77c909fe243b197ef61ea6b271908 on tablet f6d77c909fe243b197ef61ea6b271908 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:51.612653  2507 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:51.614022  2507 tablet_bootstrap.cc:492] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:51.614995  2507 ts_tablet_manager.cc:1403] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.157s	user 0.021s	sys 0.024s
I20260709 13:00:51.626386  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 26917aa1f55f48918fedd4960446f38c from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:00:51.627125  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 26917aa1f55f48918fedd4960446f38c from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-26917aa1f55f48918fedd4960446f38c
I20260709 13:00:51.629822  2507 raft_consensus.cc:359] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.634684  2507 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:51.637560  2507 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.642951  2507 ts_tablet_manager.cc:1434] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.004s
I20260709 13:00:51.646431  1586 tablet_copy_source_session.cc:215] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:51.653659  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-53c9e58595214defa013274c5193de1a received from {username='slave'} at 127.0.0.1:60218
I20260709 13:00:51.654312  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-53c9e58595214defa013274c5193de1a on tablet 53c9e58595214defa013274c5193de1a with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:51.668804  2514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26917aa1f55f48918fedd4960446f38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:51.672991  2510 tablet_copy_client.cc:538] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:51.704892  2380 raft_consensus.cc:1217] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:51.718961  2514 tablet_copy_client.cc:806] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:51.719796  2514 tablet_copy_client.cc:670] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:51.733784  2514 tablet_copy_client.cc:538] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:51.734557  2510 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:51.745999  2514 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:51.770110  2510 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:51.771278  2510 tablet_bootstrap.cc:492] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:51.772141  2510 ts_tablet_manager.cc:1403] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.014s
I20260709 13:00:51.776525  2510 raft_consensus.cc:359] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.777616  2510 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:51.778409  2510 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.779223  2514 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:51.781616  2510 ts_tablet_manager.cc:1434] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:00:51.783282  2514 tablet_bootstrap.cc:492] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:51.784739  1661 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-78d12cc2bc7b4654acd41e6c45eb9753 received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:51.785241  1661 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-78d12cc2bc7b4654acd41e6c45eb9753 on tablet 78d12cc2bc7b4654acd41e6c45eb9753 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:51.784191  2514 ts_tablet_manager.cc:1403] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.038s	user 0.031s	sys 0.001s
I20260709 13:00:51.792512  2514 raft_consensus.cc:359] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.793807  2514 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:51.794696  2514 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:51.798255  2514 ts_tablet_manager.cc:1434] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:00:51.800783  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-26917aa1f55f48918fedd4960446f38c received from {username='slave'} at 127.0.0.1:45556
I20260709 13:00:51.801378  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-26917aa1f55f48918fedd4960446f38c on tablet 26917aa1f55f48918fedd4960446f38c with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:51.853505  2380 raft_consensus.cc:1217] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:51.956076  2380 raft_consensus.cc:1217] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:51.986513  2380 raft_consensus.cc:1217] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:52.238922  2499 raft_consensus.cc:1064] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:52.242270  2499 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.253753  2380 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.253753  1651 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.256502  1573 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.256094  2499 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:52.258790  2500 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:52.259742  2479 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:52.297079  2479 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.323295  2380 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.329847  1573 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.349457  1651 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.365902  1473 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:52.402420  2499 raft_consensus.cc:1064] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:52.406720  2499 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.432042  2380 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.434152  2479 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.449749  2464 raft_consensus.cc:1064] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:52.454269  2464 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.466756  1645 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.470064  2479 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.471001  1573 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.475358  2500 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.490165  1725 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.490165  2380 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.495424  1651 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.501423  2464 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.511456  2448 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:00:52.542219  2464 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.548501  2530 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.551847  1572 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.630503  2520 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.631443  1645 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.632110  2380 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.636886  2378 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.652740  2471 raft_consensus.cc:1064] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:52.670799  2471 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.710095  2378 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.712365  2536 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.717402  1721 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.719515  2536 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.727381  1572 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:52.729228  2491 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:52.711685  1471 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.738058  1717 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.748728  2380 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:52.751617  2518 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.761528  2530 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.777755  2491 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.780594  1572 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.782064  1647 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.789371  2380 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.794509  1645 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:52.795274  1471 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.799070  2518 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.800127  1717 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.803364  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f6d77c909fe243b197ef61ea6b271908 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:52.819422  1645 raft_consensus.cc:2955] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.841059  1725 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.848426  2380 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.852084  1472 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.855461  1721 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.901000  1645 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:52.902302  2380 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:52.900089  1474 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.902639  2500 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.905696  2500 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.915683  2518 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.923735  2380 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.929708  1645 raft_consensus.cc:2955] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.934779  1647 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.951172  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53c9e58595214defa013274c5193de1a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:52.952723  1572 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:52.954828  2471 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.955068  2378 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:52.959700  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet f6d77c909fe243b197ef61ea6b271908 with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44738
I20260709 13:00:52.964224  2534 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.968132  1574 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:52.978041  2534 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.980574  1473 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:52.985803  2378 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.009370  2379 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:53.013285  1647 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:53.023473  1572 raft_consensus.cc:2955] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.031388  2502 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.033525  2520 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.063269  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78d12cc2bc7b4654acd41e6c45eb9753 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:53.065506  2545 tablet_replica.cc:333] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:00:53.067540  2545 raft_consensus.cc:2243] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:53.083103  1471 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.086907  2545 raft_consensus.cc:2272] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:53.090200  1645 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.092015  2379 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.086920  2520 raft_consensus.cc:2955] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.102133  2545 ts_tablet_manager.cc:1916] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:53.121976  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d12cc2bc7b4654acd41e6c45eb9753 with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:00:53.131690  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet 53c9e58595214defa013274c5193de1a with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44738
I20260709 13:00:53.135221  2545 ts_tablet_manager.cc:1929] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:53.135666  2545 log.cc:1199] T f6d77c909fe243b197ef61ea6b271908 P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/f6d77c909fe243b197ef61ea6b271908
I20260709 13:00:53.137303  2545 tablet_replica.cc:333] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:00:53.138337  2545 raft_consensus.cc:2243] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:53.138954  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet f6d77c909fe243b197ef61ea6b271908 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:53.139134  2545 raft_consensus.cc:2272] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:53.140172  2546 tablet_replica.cc:333] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:00:53.141871  1474 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:53.143615  2545 ts_tablet_manager.cc:1916] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:53.141860  2546 raft_consensus.cc:2243] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:53.146919  2546 raft_consensus.cc:2272] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:53.160284  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26917aa1f55f48918fedd4960446f38c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:53.162233  2545 ts_tablet_manager.cc:1929] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:53.162722  2545 log.cc:1199] T 53c9e58595214defa013274c5193de1a P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/53c9e58595214defa013274c5193de1a
I20260709 13:00:53.163630  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 26917aa1f55f48918fedd4960446f38c with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:00:53.166417  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet 53c9e58595214defa013274c5193de1a (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:53.167805  2546 ts_tablet_manager.cc:1916] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:53.181373  2546 ts_tablet_manager.cc:1929] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:53.181797  2546 log.cc:1199] T 78d12cc2bc7b4654acd41e6c45eb9753 P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/78d12cc2bc7b4654acd41e6c45eb9753
I20260709 13:00:53.183178  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 78d12cc2bc7b4654acd41e6c45eb9753 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:53.183343  2546 tablet_replica.cc:333] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:00:53.184719  2546 raft_consensus.cc:2243] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:53.185305  2546 raft_consensus.cc:2272] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:53.188330  2546 ts_tablet_manager.cc:1916] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:53.201498  2546 ts_tablet_manager.cc:1929] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:53.201856  2546 log.cc:1199] T 26917aa1f55f48918fedd4960446f38c P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/26917aa1f55f48918fedd4960446f38c
I20260709 13:00:53.203312  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 26917aa1f55f48918fedd4960446f38c (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:54.291831  1572 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:54.302462  1651 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:54.303423  1717 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:54.304133  1539 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08c9dfcb3286481dbbb396ccf74ada24. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:54.306051  2551 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.307121  2549 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.320045  2549 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.321892  1645 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.331985  1725 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.349076  1474 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.351953  1572 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:54.369431  1645 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:54.369095  1725 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:54.371474  1539 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5938eec8e53f4d08b26ec3011f1cdbfc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:54.372568  2519 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.373720  2549 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:54.404282  2549 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.410571  1647 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.427588  1725 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.437284  1473 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.440439  1645 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
W20260709 13:00:54.476751  1614 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3646214b1a14514955e904e05fadcf1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:54.479442  1721 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:54.481542  2471 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.509912  1572 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:54.512131  2536 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.545616  2567 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.555788  1572 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.558763  1725 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.575337  1645 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:54.575349  1473 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
W20260709 13:00:54.600581  1614 consensus_peers.cc:597] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb93f4d4caa2474bb5d42f9940f0dcdc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:54.601544  1720 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:54.604257  2565 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.606720  1570 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:54.610441  2565 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:54.621047  2567 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.623430  1725 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.630477  1572 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.651746  1474 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:54.854028  2572 ts_tablet_manager.cc:933] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:54.857666  2572 tablet_copy_client.cc:323] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:00:54.862030  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 08c9dfcb3286481dbbb396ccf74ada24 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:00:54.862852  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 08c9dfcb3286481dbbb396ccf74ada24 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-08c9dfcb3286481dbbb396ccf74ada24
I20260709 13:00:54.873917  1586 tablet_copy_source_session.cc:215] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:54.881671  2572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c9dfcb3286481dbbb396ccf74ada24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:54.897480  2574 ts_tablet_manager.cc:933] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:54.899961  2574 tablet_copy_client.cc:323] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:00:54.902295  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 5938eec8e53f4d08b26ec3011f1cdbfc from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:00:54.903021  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 5938eec8e53f4d08b26ec3011f1cdbfc from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-5938eec8e53f4d08b26ec3011f1cdbfc
I20260709 13:00:54.913460  1586 tablet_copy_source_session.cc:215] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:54.924064  2572 tablet_copy_client.cc:806] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:54.926016  2572 tablet_copy_client.cc:670] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:54.928473  2574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5938eec8e53f4d08b26ec3011f1cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:54.931658  2572 tablet_copy_client.cc:538] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:54.948719  2572 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:54.955641  2574 tablet_copy_client.cc:806] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:54.956557  2574 tablet_copy_client.cc:670] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:54.965817  2574 tablet_copy_client.cc:538] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:54.979873  2574 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:54.990592  2572 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:54.992290  2572 tablet_bootstrap.cc:492] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:54.993582  2572 ts_tablet_manager.cc:1403] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.046s	user 0.027s	sys 0.009s
I20260709 13:00:55.004704  2572 raft_consensus.cc:359] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.007184  2572 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:55.008098  2572 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.024623  2574 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:55.026268  2574 tablet_bootstrap.cc:492] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:55.027139  2572 ts_tablet_manager.cc:1434] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.033s	user 0.015s	sys 0.001s
I20260709 13:00:55.027140  2574 ts_tablet_manager.cc:1403] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.020s
I20260709 13:00:55.044229  2574 raft_consensus.cc:359] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.045588  2574 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:55.046623  2574 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.048504  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-08c9dfcb3286481dbbb396ccf74ada24 received from {username='slave'} at 127.0.0.1:45556
I20260709 13:00:55.049235  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-08c9dfcb3286481dbbb396ccf74ada24 on tablet 08c9dfcb3286481dbbb396ccf74ada24 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:55.050393  2574 ts_tablet_manager.cc:1434] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:00:55.059072  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-5938eec8e53f4d08b26ec3011f1cdbfc received from {username='slave'} at 127.0.0.1:45556
I20260709 13:00:55.059657  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-5938eec8e53f4d08b26ec3011f1cdbfc on tablet 5938eec8e53f4d08b26ec3011f1cdbfc with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:55.102649  2574 ts_tablet_manager.cc:933] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:55.121937  2574 tablet_copy_client.cc:323] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:55.124976  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet f3646214b1a14514955e904e05fadcf1 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:55.125659  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet f3646214b1a14514955e904e05fadcf1 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-f3646214b1a14514955e904e05fadcf1
I20260709 13:00:55.134975  1661 tablet_copy_source_session.cc:215] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:55.139046  2574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3646214b1a14514955e904e05fadcf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:55.160517  2574 tablet_copy_client.cc:806] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:55.161417  2574 tablet_copy_client.cc:670] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:55.166270  2574 tablet_copy_client.cc:538] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:55.166743  2572 ts_tablet_manager.cc:933] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:55.176019  2572 tablet_copy_client.cc:323] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:55.178316  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet fb93f4d4caa2474bb5d42f9940f0dcdc from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:55.179015  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet fb93f4d4caa2474bb5d42f9940f0dcdc from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-fb93f4d4caa2474bb5d42f9940f0dcdc
I20260709 13:00:55.186916  2574 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:55.215754  1661 tablet_copy_source_session.cc:215] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:55.220299  2572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb93f4d4caa2474bb5d42f9940f0dcdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:55.222344  2574 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:55.223529  2574 tablet_bootstrap.cc:492] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:55.225008  2574 ts_tablet_manager.cc:1403] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.016s
I20260709 13:00:55.228549  2574 raft_consensus.cc:359] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.229604  2574 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:55.230398  2574 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.243662  2574 ts_tablet_manager.cc:1434] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:00:55.250433  1661 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-f3646214b1a14514955e904e05fadcf1 received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:55.251056  1661 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-f3646214b1a14514955e904e05fadcf1 on tablet f3646214b1a14514955e904e05fadcf1 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:55.252890  2572 tablet_copy_client.cc:806] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:55.261061  2572 tablet_copy_client.cc:670] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:55.265565  2572 tablet_copy_client.cc:538] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:55.277406  2572 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:55.302420  2572 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:55.303364  2572 tablet_bootstrap.cc:492] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:55.304171  2572 ts_tablet_manager.cc:1403] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.001s
I20260709 13:00:55.307693  2572 raft_consensus.cc:359] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.308759  2572 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:55.309540  2572 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:55.315963  2572 ts_tablet_manager.cc:1434] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:00:55.319873  1661 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-fb93f4d4caa2474bb5d42f9940f0dcdc received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:55.320549  1661 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-fb93f4d4caa2474bb5d42f9940f0dcdc on tablet fb93f4d4caa2474bb5d42f9940f0dcdc with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:55.404301  2379 raft_consensus.cc:1217] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:55.428464  2379 raft_consensus.cc:1217] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:55.468739  2379 raft_consensus.cc:1217] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:55.546725  2379 raft_consensus.cc:1217] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:55.822773  2548 raft_consensus.cc:1064] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:55.826203  2548 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:55.837939  2379 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:55.840436  1720 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:55.840867  2549 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:55.843202  2549 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:55.844249  1647 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:55.847986  2548 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:55.879575  2520 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:55.902628  1721 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:55.907895  2379 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:55.919143  1647 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:55.955955  1473 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:55.990384  2551 raft_consensus.cc:1064] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:55.993470  2551 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:56.006839  2378 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.011390  1647 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.014223  2548 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.020021  2551 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.029690  1725 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.032778  2566 raft_consensus.cc:1064] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:56.039407  2566 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:56.042884  2548 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.055339  2379 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.057482  1572 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.057890  2568 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.053378  2548 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.059662  2567 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.062816  1725 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.064507  2567 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.103658  1651 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.129266  2568 raft_consensus.cc:1064] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:56.127251  2565 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.147807  2568 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:56.212201  1471 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.278851  2379 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.283978  1574 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:00:56.301719  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 126637us
I20260709 13:00:56.311197  2378 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.315551  1570 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.317155  1720 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.345999  1474 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:56.387502  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44244.6us
I20260709 13:00:56.416162  1570 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.421348  2603 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.425760  1651 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:56.434366  2378 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.431331  1722 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.441884  1721 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:56.446728  2378 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.444628  2565 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:00:56.471601  1572 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.472565  2567 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:56.487764  2565 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.502203  2379 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.508905  1721 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.514559  2567 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.520586  2601 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.528442  2379 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.547392  1573 raft_consensus.cc:2955] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.561781  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f3646214b1a14514955e904e05fadcf1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:00:56.573397  2607 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 13:00:56.584285  1474 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.597792  1573 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:56.606784  2566 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.613471  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet f3646214b1a14514955e904e05fadcf1 with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:00:56.623063  2548 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:56.622105  1570 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.631939  1717 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.643824  2610 tablet_replica.cc:333] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:00:56.645385  2610 raft_consensus.cc:2243] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:56.646461  2610 raft_consensus.cc:2272] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:56.657344  2549 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.660215  2379 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.668061  2378 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.674718  1471 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.675504  2548 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.677845  1717 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.690330  2551 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.691076  2606 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.701987  1651 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:56.722530  1720 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.724886  2567 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.750437  1717 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.769217  2380 raft_consensus.cc:2955] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.771299  2610 ts_tablet_manager.cc:1916] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:56.771541  2549 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.786904  2379 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:56.798408  1471 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.815062  2602 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:56.801888  2378 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.806609  2565 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.810411  1722 raft_consensus.cc:2955] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.800275  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5938eec8e53f4d08b26ec3011f1cdbfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:56.829257  1721 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.833617  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet 5938eec8e53f4d08b26ec3011f1cdbfc with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:00:56.849071  2610 ts_tablet_manager.cc:1929] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:56.851389  2610 log.cc:1199] T f3646214b1a14514955e904e05fadcf1 P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/f3646214b1a14514955e904e05fadcf1
I20260709 13:00:56.853440  2379 raft_consensus.cc:2955] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.864439  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet f3646214b1a14514955e904e05fadcf1 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:56.887849  1474 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.891577  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb93f4d4caa2474bb5d42f9940f0dcdc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:00:56.899891  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 94628.3us
I20260709 13:00:56.904111  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08c9dfcb3286481dbbb396ccf74ada24 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:56.919248  1640 tablet_service.cc:1558] Processing DeleteTablet for tablet 08c9dfcb3286481dbbb396ccf74ada24 with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:00:56.928282  1471 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:56.940516  2612 tablet_replica.cc:333] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:00:56.967260  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet fb93f4d4caa2474bb5d42f9940f0dcdc with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44738
I20260709 13:00:56.976754  2613 tablet_replica.cc:333] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:00:56.978358  2613 raft_consensus.cc:2243] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:56.984206  2613 raft_consensus.cc:2272] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:56.996662  2613 ts_tablet_manager.cc:1916] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:56.998382  2612 raft_consensus.cc:2243] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:57.000497  2612 raft_consensus.cc:2272] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:57.004621  2612 ts_tablet_manager.cc:1916] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:57.027946  2613 ts_tablet_manager.cc:1929] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:57.030836  2613 log.cc:1199] T fb93f4d4caa2474bb5d42f9940f0dcdc P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/fb93f4d4caa2474bb5d42f9940f0dcdc
I20260709 13:00:57.032310  2612 ts_tablet_manager.cc:1929] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:57.033630  2612 log.cc:1199] T 5938eec8e53f4d08b26ec3011f1cdbfc P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/5938eec8e53f4d08b26ec3011f1cdbfc
I20260709 13:00:57.036585  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet fb93f4d4caa2474bb5d42f9940f0dcdc (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:57.037580  2612 tablet_replica.cc:333] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:00:57.036123  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet 5938eec8e53f4d08b26ec3011f1cdbfc (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:57.039168  2612 raft_consensus.cc:2243] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:57.040062  2612 raft_consensus.cc:2272] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:57.044232  2612 ts_tablet_manager.cc:1916] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:57.067906  2612 ts_tablet_manager.cc:1929] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:57.068455  2612 log.cc:1199] T 08c9dfcb3286481dbbb396ccf74ada24 P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/08c9dfcb3286481dbbb396ccf74ada24
I20260709 13:00:57.071189  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet 08c9dfcb3286481dbbb396ccf74ada24 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:00:58.061986  1643 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.073854  1717 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.075376  1573 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.075430  2603 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:58.080866  1614 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4391f9a46a9542a7bf726b05393b7609. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.083226  2603 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:58.093447  2602 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.095789  1717 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.098872  1573 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.123821  1643 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.134415  1473 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
W20260709 13:00:58.169219  1614 consensus_peers.cc:597] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 414132da4f974e5c912e5d5dfa077654. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.169488  1717 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.173338  2618 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:58.173416  1573 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.176545  2618 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:58.193010  2618 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.198156  1717 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.258728  1573 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.259858  1574 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.288419  1651 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.290643  1717 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.285096  1474 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.293360  2611 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:58.297646  1539 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01c1afc8c26047b0b3d8c006f9df5a30. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.298986  2614 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:58.316761  2606 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.324082  1651 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.336107  1722 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.346696  1643 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.366657  1472 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:58.383055  1574 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:58.383963  1614 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f048890019da43548110ab19fce7a878. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:58.386221  1725 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:58.389603  2603 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:58.426560  2602 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:58.430444  2603 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.441567  1574 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.456445  1725 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.468824  1474 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:00:58.480203  2634 ts_tablet_manager.cc:933] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:58.498922  2634 tablet_copy_client.cc:323] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:58.500844  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet 4391f9a46a9542a7bf726b05393b7609 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:58.501536  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet 4391f9a46a9542a7bf726b05393b7609 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-4391f9a46a9542a7bf726b05393b7609
I20260709 13:00:58.511227  1661 tablet_copy_source_session.cc:215] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:58.515105  2634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4391f9a46a9542a7bf726b05393b7609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.538537  2634 tablet_copy_client.cc:806] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:58.540437  2634 tablet_copy_client.cc:670] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:58.547168  2634 tablet_copy_client.cc:538] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.566915  2634 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:58.592474  2634 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:58.593905  2634 tablet_bootstrap.cc:492] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:58.595057  2634 ts_tablet_manager.cc:1403] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.011s
I20260709 13:00:58.599342  2634 raft_consensus.cc:359] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.600368  2634 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:58.601146  2634 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.605042  2634 ts_tablet_manager.cc:1434] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.001s
I20260709 13:00:58.607926  1661 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-4391f9a46a9542a7bf726b05393b7609 received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:58.608500  1661 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-4391f9a46a9542a7bf726b05393b7609 on tablet 4391f9a46a9542a7bf726b05393b7609 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:58.753433  2634 ts_tablet_manager.cc:933] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:58.756759  2634 tablet_copy_client.cc:323] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:58.759279  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet 414132da4f974e5c912e5d5dfa077654 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:58.759840  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet 414132da4f974e5c912e5d5dfa077654 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-414132da4f974e5c912e5d5dfa077654
I20260709 13:00:58.765427  1661 tablet_copy_source_session.cc:215] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:58.775574  2634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414132da4f974e5c912e5d5dfa077654. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.798631  2634 tablet_copy_client.cc:806] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:58.800603  2634 tablet_copy_client.cc:670] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:58.804255  2634 tablet_copy_client.cc:538] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.816148  2634 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:58.832553  2641 ts_tablet_manager.cc:933] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:00:58.865978  2641 tablet_copy_client.cc:323] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43325
I20260709 13:00:58.868450  1661 tablet_copy_service.cc:140] P cd6d377efe514bc58ae49c3f4a107232: Received BeginTabletCopySession request for tablet f048890019da43548110ab19fce7a878 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:34482)
I20260709 13:00:58.869331  1661 tablet_copy_service.cc:161] P cd6d377efe514bc58ae49c3f4a107232: Beginning new tablet copy session on tablet f048890019da43548110ab19fce7a878 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:34482: session id = da3b57321031416cba9db75450ef302a-f048890019da43548110ab19fce7a878
I20260709 13:00:58.896888  2634 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:58.898581  2634 tablet_bootstrap.cc:492] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:58.899788  2634 ts_tablet_manager.cc:1403] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.084s	user 0.031s	sys 0.014s
I20260709 13:00:58.903869  2634 raft_consensus.cc:359] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.905226  2634 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:58.906154  2634 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:58.916162  2634 ts_tablet_manager.cc:1434] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:00:58.920393  2640 ts_tablet_manager.cc:933] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:00:58.921586  1660 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-414132da4f974e5c912e5d5dfa077654 received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:58.924050  1660 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-414132da4f974e5c912e5d5dfa077654 on tablet 414132da4f974e5c912e5d5dfa077654 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:58.925403  1661 tablet_copy_source_session.cc:215] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:58.929832  2641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f048890019da43548110ab19fce7a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.944494  2640 tablet_copy_client.cc:323] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:00:58.946900  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 01c1afc8c26047b0b3d8c006f9df5a30 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:00:58.947585  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 01c1afc8c26047b0b3d8c006f9df5a30 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-01c1afc8c26047b0b3d8c006f9df5a30
I20260709 13:00:58.957373  2641 tablet_copy_client.cc:806] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:58.958216  2641 tablet_copy_client.cc:670] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:58.964048  2641 tablet_copy_client.cc:538] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:58.960777  1586 tablet_copy_source_session.cc:215] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:58.970003  2378 raft_consensus.cc:1217] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:58.980736  2640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c1afc8c26047b0b3d8c006f9df5a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:58.985507  2641 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:59.081964  2641 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:59.083079  2641 tablet_bootstrap.cc:492] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:59.083925  2641 ts_tablet_manager.cc:1403] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.099s	user 0.024s	sys 0.042s
I20260709 13:00:59.087896  2641 raft_consensus.cc:359] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:59.089128  2641 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:59.092710  2640 tablet_copy_client.cc:806] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:59.090041  2641 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:59.095786  2641 ts_tablet_manager.cc:1434] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:00:59.093657  2640 tablet_copy_client.cc:670] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:59.098481  1661 tablet_copy_service.cc:342] P cd6d377efe514bc58ae49c3f4a107232: Request end of tablet copy session da3b57321031416cba9db75450ef302a-f048890019da43548110ab19fce7a878 received from {username='slave'} at 127.0.0.1:34482
I20260709 13:00:59.098942  1661 tablet_copy_service.cc:434] P cd6d377efe514bc58ae49c3f4a107232: ending tablet copy session da3b57321031416cba9db75450ef302a-f048890019da43548110ab19fce7a878 on tablet f048890019da43548110ab19fce7a878 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:59.126580  2640 tablet_copy_client.cc:538] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:59.143697  2378 raft_consensus.cc:1217] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:59.145915  2640 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:00:59.211266  2640 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:59.213042  2640 tablet_bootstrap.cc:492] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:00:59.213960  2640 ts_tablet_manager.cc:1403] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.068s	user 0.028s	sys 0.010s
I20260709 13:00:59.216532  2640 raft_consensus.cc:359] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:59.217476  2640 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:00:59.218163  2640 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:00:59.221863  2640 ts_tablet_manager.cc:1434] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:59.224246  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-01c1afc8c26047b0b3d8c006f9df5a30 received from {username='slave'} at 127.0.0.1:45556
I20260709 13:00:59.224864  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-01c1afc8c26047b0b3d8c006f9df5a30 on tablet 01c1afc8c26047b0b3d8c006f9df5a30 with peer da3b57321031416cba9db75450ef302a
I20260709 13:00:59.238715  2378 raft_consensus.cc:1217] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:59.393716  2378 raft_consensus.cc:1217] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:59.528021  2567 raft_consensus.cc:1064] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:59.536054  2567 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:59.544826  2378 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.546029  1722 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.547187  2602 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.549084  2618 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.570989  1573 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.574324  2603 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.580936  2618 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.601058  2379 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.614653  2602 raft_consensus.cc:1064] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:59.613943  1574 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.622768  2602 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:59.629639  1474 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.644901  1722 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.666802  2655 raft_consensus.cc:1064] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:00:59.686173  2655 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:59.696899  1574 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.697013  1725 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.699373  2379 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.699339  2602 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:59.702131  2654 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.705238  2618 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.782835  2379 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.784096  1570 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.788055  2654 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.788867  1721 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:59.797586  2603 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:59.799705  2618 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.800944  2603 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:59.808951  1725 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.810170  1574 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.822441  2603 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.823287  1722 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.851012  1573 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.857546  2378 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.860695  1643 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:59.878062  1473 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.885524  2380 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:59.886507  1473 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.887281  2655 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.890295  1570 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:59.894712  2567 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.896628  2380 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.909392  2567 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.912744  2378 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.937026  1570 raft_consensus.cc:2955] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.968917  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4391f9a46a9542a7bf726b05393b7609 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:59.970268  1643 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:00:59.974206  1471 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:00:59.999869  2379 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:00.002154  2602 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:00.002807  2611 raft_consensus.cc:1064] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:00.008687  1722 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:00.017655  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 4391f9a46a9542a7bf726b05393b7609 with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:01:00.015504  2618 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:01:00.023691  2611 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:00.026662  2567 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.039454  2378 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.046239  1721 raft_consensus.cc:2955] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.036473  2669 tablet_replica.cc:333] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:01:00.064519  1471 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.074071  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 414132da4f974e5c912e5d5dfa077654 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:00.082578  2378 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:00.087493  1645 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:00.094945  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet 414132da4f974e5c912e5d5dfa077654 with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44738
I20260709 13:01:00.099850  2668 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:00.107209  2379 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:00.107692  1573 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:00.109400  2602 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:00.111531  2618 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:00.121083  1725 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:00.125170  1645 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:00.125772  2643 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:00.132546  2618 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.141021  2380 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.138947  2644 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:00.174290  2671 tablet_replica.cc:333] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:01:00.174644  1570 raft_consensus.cc:2955] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.195816  2611 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.203902  2669 raft_consensus.cc:2243] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:00.204955  2669 raft_consensus.cc:2272] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:00.208915  2671 raft_consensus.cc:2243] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:00.209897  2671 raft_consensus.cc:2272] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:00.211249  1651 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.212121  1474 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.217993  2669 ts_tablet_manager.cc:1916] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:00.221376  1721 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.246320  2380 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.223009  2671 ts_tablet_manager.cc:1916] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:00.224530  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f048890019da43548110ab19fce7a878 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:01:00.270006  1458 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45502.4us
I20260709 13:01:00.272193  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet f048890019da43548110ab19fce7a878 with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:01:00.275321  1474 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.265388  2669 ts_tablet_manager.cc:1929] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:00.288679  2669 log.cc:1199] T 4391f9a46a9542a7bf726b05393b7609 P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/4391f9a46a9542a7bf726b05393b7609
I20260709 13:01:00.291139  2669 tablet_replica.cc:333] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:01:00.293258  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 4391f9a46a9542a7bf726b05393b7609 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:00.294271  2669 raft_consensus.cc:2243] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:00.295285  2669 raft_consensus.cc:2272] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:00.301584  2669 ts_tablet_manager.cc:1916] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:00.310535  1570 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:00.313398  2671 ts_tablet_manager.cc:1929] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:00.314947  2671 log.cc:1199] T 414132da4f974e5c912e5d5dfa077654 P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/414132da4f974e5c912e5d5dfa077654
I20260709 13:01:00.320464  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet 414132da4f974e5c912e5d5dfa077654 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:00.321683  1721 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:00.320593  2380 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:00.324265  2668 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:01:00.328171  2675 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:00.336591  2669 ts_tablet_manager.cc:1929] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:00.337298  2669 log.cc:1199] T f048890019da43548110ab19fce7a878 P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/f048890019da43548110ab19fce7a878
I20260709 13:01:00.339326  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet f048890019da43548110ab19fce7a878 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:00.347015  2644 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.351135  1721 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.367591  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 01c1afc8c26047b0b3d8c006f9df5a30 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:00.374123  2380 raft_consensus.cc:2955] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:00.380002  1474 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:00.404657  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet 01c1afc8c26047b0b3d8c006f9df5a30 with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:01:00.408682  2676 tablet_replica.cc:333] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:01:00.410981  2676 raft_consensus.cc:2243] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:00.411809  2676 raft_consensus.cc:2272] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:00.415314  2676 ts_tablet_manager.cc:1916] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:00.432979  2676 ts_tablet_manager.cc:1929] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:00.433490  2676 log.cc:1199] T 01c1afc8c26047b0b3d8c006f9df5a30 P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/01c1afc8c26047b0b3d8c006f9df5a30
I20260709 13:01:00.435446  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet 01c1afc8c26047b0b3d8c006f9df5a30 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:01.516584  1725 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
W20260709 13:01:01.526252  1688 consensus_peers.cc:597] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6834f8da7764493a94dc534e4471b3dc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:01.527007  1643 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.527648  1574 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.528470  2677 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.529663  2585 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.547793  1570 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.546183  2585 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.579047  1472 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.595243  1570 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:01.618486  1643 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.626300  1725 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.631235  1651 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:01.632366  1539 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81bce6c3bf1547e4b8d33586cafe5f6c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:01.647620  2643 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.673266  2690 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.689520  2643 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.702350  1651 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.710964  1722 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:01.711266  1725 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.724064  1474 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:01.730829  1688 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb755eb990bc40778167e79183a40b84. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:01.733294  1651 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.730151  1570 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.737553  2585 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.740942  2585 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:01.762782  2677 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.768994  1651 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.770216  1570 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.785832  1574 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:01.790040  1474 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:01.815994  1539 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba20622bf78343c5bf6ac194af441026. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:01.820178  1725 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.823606  2643 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.827840  1651 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:01.829511  2643 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:01.837930  2690 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.844928  1725 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.848273  1651 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.875682  1472 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:01.917814  2700 ts_tablet_manager.cc:933] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:01.923362  2700 tablet_copy_client.cc:323] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:01:01.925724  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet 6834f8da7764493a94dc534e4471b3dc from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:01:01.926405  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet 6834f8da7764493a94dc534e4471b3dc from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-6834f8da7764493a94dc534e4471b3dc
I20260709 13:01:01.935340  1735 tablet_copy_source_session.cc:215] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:01.940019  2700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834f8da7764493a94dc534e4471b3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:01.965983  2700 tablet_copy_client.cc:806] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:01.967025  2700 tablet_copy_client.cc:670] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:01.973767  2700 tablet_copy_client.cc:538] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:01.989796  2700 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:02.021040  2700 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:02.022255  2700 tablet_bootstrap.cc:492] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:02.023136  2700 ts_tablet_manager.cc:1403] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 13:01:02.027582  2700 raft_consensus.cc:359] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:02.028724  2700 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:02.029551  2700 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:02.033991  2700 ts_tablet_manager.cc:1434] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:01:02.038071  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-6834f8da7764493a94dc534e4471b3dc received from {username='slave'} at 127.0.0.1:60218
I20260709 13:01:02.039431  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-6834f8da7764493a94dc534e4471b3dc on tablet 6834f8da7764493a94dc534e4471b3dc with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:02.185092  2700 ts_tablet_manager.cc:933] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:02.187265  2700 tablet_copy_client.cc:323] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:01:02.189263  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet fb755eb990bc40778167e79183a40b84 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:01:02.189944  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet fb755eb990bc40778167e79183a40b84 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-fb755eb990bc40778167e79183a40b84
I20260709 13:01:02.201977  1735 tablet_copy_source_session.cc:215] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:02.205737  2700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb755eb990bc40778167e79183a40b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:02.242708  2705 ts_tablet_manager.cc:933] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:01:02.254334  2705 tablet_copy_client.cc:323] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:01:02.257119  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 81bce6c3bf1547e4b8d33586cafe5f6c from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:01:02.257846  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 81bce6c3bf1547e4b8d33586cafe5f6c from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-81bce6c3bf1547e4b8d33586cafe5f6c
I20260709 13:01:02.263674  2700 tablet_copy_client.cc:806] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:02.265712  2700 tablet_copy_client.cc:670] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:02.269557  1586 tablet_copy_source_session.cc:215] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:02.278360  2705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bce6c3bf1547e4b8d33586cafe5f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:02.279290  2700 tablet_copy_client.cc:538] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:02.289860  2700 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:02.302479  2705 tablet_copy_client.cc:806] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:02.303267  2705 tablet_copy_client.cc:670] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:02.309062  2705 tablet_copy_client.cc:538] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:02.318378  2700 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:02.319914  2700 tablet_bootstrap.cc:492] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:02.321029  2705 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:02.321043  2700 ts_tablet_manager.cc:1403] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.031s	user 0.024s	sys 0.005s
I20260709 13:01:02.325176  2700 raft_consensus.cc:359] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:02.326545  2700 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:02.327473  2700 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:02.334455  2700 ts_tablet_manager.cc:1434] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:01:02.340122  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-fb755eb990bc40778167e79183a40b84 received from {username='slave'} at 127.0.0.1:60218
I20260709 13:01:02.340952  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-fb755eb990bc40778167e79183a40b84 on tablet fb755eb990bc40778167e79183a40b84 with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:02.372226  2379 raft_consensus.cc:1217] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:02.402667  2705 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:02.404084  2705 tablet_bootstrap.cc:492] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:02.405115  2705 ts_tablet_manager.cc:1403] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.084s	user 0.025s	sys 0.013s
I20260709 13:01:02.421733  2705 raft_consensus.cc:359] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:02.422801  2705 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:02.423648  2705 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:02.426842  2705 ts_tablet_manager.cc:1434] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:01:02.470958  2700 ts_tablet_manager.cc:933] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:01:02.617273  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-81bce6c3bf1547e4b8d33586cafe5f6c received from {username='slave'} at 127.0.0.1:45556
I20260709 13:01:02.617938  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-81bce6c3bf1547e4b8d33586cafe5f6c on tablet 81bce6c3bf1547e4b8d33586cafe5f6c with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:02.691324  2379 raft_consensus.cc:1217] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:02.762831  2377 raft_consensus.cc:1217] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:02.831630  2700 tablet_copy_client.cc:323] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:01:02.835486  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet ba20622bf78343c5bf6ac194af441026 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:01:02.836194  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet ba20622bf78343c5bf6ac194af441026 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-ba20622bf78343c5bf6ac194af441026
I20260709 13:01:02.920547  1586 tablet_copy_source_session.cc:215] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:01:02.921584  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53245.9us
I20260709 13:01:02.933247  2700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba20622bf78343c5bf6ac194af441026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:02.977945  2700 tablet_copy_client.cc:806] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:02.978753  2700 tablet_copy_client.cc:670] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:02.999912  2685 raft_consensus.cc:1064] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:03.002796  2685 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:03.005167  2700 tablet_copy_client.cc:538] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:03.022825  2377 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.025118  2683 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.027070  2700 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:03.037292  1651 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.041963  2718 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.061298  1576 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.063477  2713 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.097787  2718 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.099797  1574 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.115952  2378 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.155900  1472 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.160766  2683 raft_consensus.cc:1064] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:03.163483  2683 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:03.169669  2700 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:03.171000  2700 tablet_bootstrap.cc:492] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:03.173894  1651 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.172129  2700 ts_tablet_manager.cc:1403] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.145s	user 0.032s	sys 0.006s
I20260709 13:01:03.190238  2700 raft_consensus.cc:359] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:03.193522  2700 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:03.194293  2700 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:03.198271  2700 ts_tablet_manager.cc:1434] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:01:03.200675  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-ba20622bf78343c5bf6ac194af441026 received from {username='slave'} at 127.0.0.1:45556
I20260709 13:01:03.201431  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-ba20622bf78343c5bf6ac194af441026 on tablet ba20622bf78343c5bf6ac194af441026 with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:03.215678  1651 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.217760  2377 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.220952  2677 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.223306  2677 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.252157  2710 raft_consensus.cc:1064] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:03.256125  2710 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:03.266137  1576 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.274225  2683 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.315182  2683 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.318216  1576 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.339651  1651 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.341583  2712 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.347678  1723 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.349711  2378 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:03.350811  2712 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.353627  2378 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.354647  1645 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.356990  2720 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:03.376804  2380 raft_consensus.cc:1217] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:03.424207  2720 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.431172  2377 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.451743  1645 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.455451  1723 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.466032  1720 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:03.472685  1473 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.531527  2377 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:03.528995  1474 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:03.545535  1645 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:03.550597  2685 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:03.554821  1597 debug-util.cc:398] Leaking SignalData structure 0x7b080017e680 after lost signal to thread 434
W20260709 13:01:03.556303  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad940 after lost signal to thread 1508
I20260709 13:01:03.551813  2718 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
W20260709 13:01:03.559712  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc7a0 after lost signal to thread 1600
W20260709 13:01:03.563434  1597 debug-util.cc:398] Leaking SignalData structure 0x7b0800311c60 after lost signal to thread 1675
W20260709 13:01:03.572233  1597 debug-util.cc:398] Leaking SignalData structure 0x7b0800310e40 after lost signal to thread 1749
W20260709 13:01:03.576040  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08002e82e0 after lost signal to thread 2404
I20260709 13:01:03.581120  2677 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.587059  1576 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:03.603546  2377 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:03.603099  2378 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.612721  2690 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:03.614351  1651 raft_consensus.cc:2955] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.620111  1723 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:03.676615  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6834f8da7764493a94dc534e4471b3dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:03.689664  1473 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 3 to 4, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.703233  2675 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.734901  2711 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:01:03.754658  1725 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:03.760833  2377 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.848207  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet 6834f8da7764493a94dc534e4471b3dc with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44738
I20260709 13:01:03.862504  1723 raft_consensus.cc:2955] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:03.921545  2378 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:03.919060  2742 tablet_replica.cc:333] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:01:03.965977  2713 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:03.988020  1573 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:03.990926  2683 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.008713  2713 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.012761  2380 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:01:04.020856  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 77324.4us
I20260709 13:01:04.025064  2675 raft_consensus.cc:1064] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:04.025854  2742 raft_consensus.cc:2243] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:04.027441  2742 raft_consensus.cc:2272] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:04.054813  1473 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.043905  2675 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:04.086274  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81bce6c3bf1547e4b8d33586cafe5f6c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:04.069722  2742 ts_tablet_manager.cc:1916] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:04.094620  1570 raft_consensus.cc:2955] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.100085  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb755eb990bc40778167e79183a40b84 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:04.137121  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet 81bce6c3bf1547e4b8d33586cafe5f6c with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:01:04.149853  1472 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 3 to 4, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:04.156891  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65444.9us
I20260709 13:01:04.174536  2745 tablet_replica.cc:333] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:01:04.207656  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet fb755eb990bc40778167e79183a40b84 with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49746
I20260709 13:01:04.224300  2745 raft_consensus.cc:2243] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:04.225518  2745 raft_consensus.cc:2272] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:04.230594  2745 ts_tablet_manager.cc:1916] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:04.258301  2742 ts_tablet_manager.cc:1929] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:04.259259  2742 log.cc:1199] T 6834f8da7764493a94dc534e4471b3dc P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/6834f8da7764493a94dc534e4471b3dc
I20260709 13:01:04.271612  2745 ts_tablet_manager.cc:1929] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:04.272375  2745 log.cc:1199] T 81bce6c3bf1547e4b8d33586cafe5f6c P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/81bce6c3bf1547e4b8d33586cafe5f6c
I20260709 13:01:04.276105  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet 81bce6c3bf1547e4b8d33586cafe5f6c (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:04.277918  2745 tablet_replica.cc:333] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:01:04.279491  2745 raft_consensus.cc:2243] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:04.280795  2745 raft_consensus.cc:2272] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:04.285794  2745 ts_tablet_manager.cc:1916] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:04.301789  2380 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.303983  1720 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.304196  1643 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.313901  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet 6834f8da7764493a94dc534e4471b3dc (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:04.316152  2745 ts_tablet_manager.cc:1929] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:04.317178  2745 log.cc:1199] T fb755eb990bc40778167e79183a40b84 P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/fb755eb990bc40778167e79183a40b84
I20260709 13:01:04.334245  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet fb755eb990bc40778167e79183a40b84 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:04.349129  2711 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.351984  2675 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.367497  2711 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.472071  2690 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.479844  1643 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.485378  1722 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.541326  1472 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.560221  2380 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.584203  1570 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:04.597154  1643 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:04.601871  2380 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:04.605409  2711 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.613369  2741 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.634575  2741 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.662499  2380 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.689344  1651 raft_consensus.cc:2955] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:04.690616  1460 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ba20622bf78343c5bf6ac194af441026 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:04.705590  1472 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index 3 to 4, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.731027  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet ba20622bf78343c5bf6ac194af441026 with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55254
I20260709 13:01:04.773821  2749 tablet_replica.cc:333] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:01:04.775506  2749 raft_consensus.cc:2243] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:04.776731  2749 raft_consensus.cc:2272] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:04.781203  2749 ts_tablet_manager.cc:1916] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:04.820430  2749 ts_tablet_manager.cc:1929] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:04.821053  2749 log.cc:1199] T ba20622bf78343c5bf6ac194af441026 P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/ba20622bf78343c5bf6ac194af441026
I20260709 13:01:04.823388  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet ba20622bf78343c5bf6ac194af441026 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:01:05.981673  1721 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
W20260709 13:01:06.012794  1688 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1acf73e529394004842219637cff7cfc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.013331  1651 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.015163  2713 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:06.015566  1576 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.018791  2713 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.072165  2718 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.085837  1643 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.087209  1576 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.127116  1473 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.132121  1721 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:06.153918  1643 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:06.155014  1688 consensus_peers.cc:597] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b837a5ec9a0d4f2a8635dedf45d51aa2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.158492  2758 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:06.159034  1570 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:06.163100  2760 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.188222  2760 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.202512  1651 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.227166  1570 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.253340  1576 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.1, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:06.266392  1472 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.301438  1723 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:06.302814  1651 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:06.309780  2754 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:06.311755  1539 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 996f59fd8f2340a49403b3e9ef4c8e86. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.312047  2690 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.340449  2690 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Committing config change with OpId 3.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.342366  1651 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Committing config change with OpId 3.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.353845  1723 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Committing config change with OpId 3.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.377238  1576 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:06.379694  1473 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.401510  1721 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.404313  2740 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:06.406701  1539 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62836cb7c4a548b5a6c44c89782a958c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.413578  1643 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.427429  2751 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.443423  2754 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.447144  1721 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.481495  2774 ts_tablet_manager.cc:933] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:06.492511  1473 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.501672  1651 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:06.531455  2774 tablet_copy_client.cc:323] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:01:06.534106  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet 1acf73e529394004842219637cff7cfc from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:01:06.534999  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet 1acf73e529394004842219637cff7cfc from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-1acf73e529394004842219637cff7cfc
I20260709 13:01:06.600441  1735 tablet_copy_source_session.cc:215] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:06.609310  2774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1acf73e529394004842219637cff7cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:06.691409  2776 ts_tablet_manager.cc:933] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:06.694697  2776 tablet_copy_client.cc:323] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:01:06.698880  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:01:06.699491  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet b837a5ec9a0d4f2a8635dedf45d51aa2 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-b837a5ec9a0d4f2a8635dedf45d51aa2
I20260709 13:01:06.703302   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:06.705013  2774 tablet_copy_client.cc:806] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:06.708189  2774 tablet_copy_client.cc:670] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:06.710951  1735 tablet_copy_source_session.cc:215] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:06.713965  2774 tablet_copy_client.cc:538] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:06.717204  2776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b837a5ec9a0d4f2a8635dedf45d51aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:06.737287  2774 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:06.740002  2776 tablet_copy_client.cc:806] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:06.741257  2776 tablet_copy_client.cc:670] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:06.745729  2776 tablet_copy_client.cc:538] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:06.766265  2776 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:06.891016  2776 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:06.892096  2776 tablet_bootstrap.cc:492] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:06.913162  2776 ts_tablet_manager.cc:1403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.147s	user 0.022s	sys 0.019s
W20260709 13:01:06.937211  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76386.4us
I20260709 13:01:06.945588  2774 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:06.946885  2774 tablet_bootstrap.cc:492] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:06.949690  1651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:06.950470  1651 raft_consensus.cc:606] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:06.952298  2781 ts_tablet_manager.cc:933] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
W20260709 13:01:06.964071  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 101658us
I20260709 13:01:06.969442  2776 raft_consensus.cc:359] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:06.995235  1576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:06.995916  1576 raft_consensus.cc:606] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:06.998905  2788 ts_tablet_manager.cc:933] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275)
I20260709 13:01:07.000535  2774 ts_tablet_manager.cc:1403] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.263s	user 0.017s	sys 0.018s
I20260709 13:01:07.000190  1576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:07.001828  1576 raft_consensus.cc:606] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:07.007731  1576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:07.008690  1576 raft_consensus.cc:606] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:07.009565  2774 raft_consensus.cc:359] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.012531  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:07.014794  1573 raft_consensus.cc:606] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:07.015025  2781 tablet_copy_client.cc:323] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:01:07.018996  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:07.019722  1573 raft_consensus.cc:606] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:07.022516   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:06.970574  2776 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:07.024755   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:07.025946  2776 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.031122  2774 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:07.037477  2774 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.041944  2788 tablet_copy_client.cc:323] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:45275
I20260709 13:01:07.044951  1586 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 62836cb7c4a548b5a6c44c89782a958c from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:01:07.045954  1586 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 62836cb7c4a548b5a6c44c89782a958c from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-62836cb7c4a548b5a6c44c89782a958c
I20260709 13:01:07.049355  1585 tablet_copy_service.cc:140] P 4f737473248e4c059fa74af3b8e1f77e: Received BeginTabletCopySession request for tablet 996f59fd8f2340a49403b3e9ef4c8e86 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:45556)
I20260709 13:01:07.050266  1585 tablet_copy_service.cc:161] P 4f737473248e4c059fa74af3b8e1f77e: Beginning new tablet copy session on tablet 996f59fd8f2340a49403b3e9ef4c8e86 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:45556: session id = da3b57321031416cba9db75450ef302a-996f59fd8f2340a49403b3e9ef4c8e86
I20260709 13:01:07.050557  2776 ts_tablet_manager.cc:1434] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.003s
I20260709 13:01:07.052433  2690 raft_consensus.cc:993] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: : Instructing follower 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.053263  2690 raft_consensus.cc:1081] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Signalling peer 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.053088  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-b837a5ec9a0d4f2a8635dedf45d51aa2 received from {username='slave'} at 127.0.0.1:60218
I20260709 13:01:07.053875  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-b837a5ec9a0d4f2a8635dedf45d51aa2 on tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:07.067749  1585 tablet_copy_source_session.cc:215] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:07.074807  2774 ts_tablet_manager.cc:1434] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.009s
I20260709 13:01:07.086656  1723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:07.087307  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-1acf73e529394004842219637cff7cfc received from {username='slave'} at 127.0.0.1:60218
I20260709 13:01:07.087476  1723 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.087777  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-1acf73e529394004842219637cff7cfc on tablet 1acf73e529394004842219637cff7cfc with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:07.087880  1723 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.090777  1586 tablet_copy_source_session.cc:215] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:07.098275  1723 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.101518  1723 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:07.104575  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:07.105305  1573 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.105600  1573 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:07.109227  2788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 996f59fd8f2340a49403b3e9ef4c8e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:07.113571  2781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62836cb7c4a548b5a6c44c89782a958c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:07.106297  1573 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.128772  1573 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.136833  2690 raft_consensus.cc:993] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: : Instructing follower 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.141047  2378 raft_consensus.cc:1217] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:01:07.147663  2740 raft_consensus.cc:1081] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Signalling peer 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.156977  1723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:07.157701  1723 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.158114  1723 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.163121  1573 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.164445  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:07.165102  1643 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.165225  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:07.166321  2760 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.166371  1723 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.166990  2760 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:07.168072  2760 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.171581  1723 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:07.188781  2783 raft_consensus.cc:993] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: : Instructing follower 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.189302  2783 raft_consensus.cc:1081] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Signalling peer 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.190954  1722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:01:07.191457  1722 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.191911  1722 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.191291  1471 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.183205  1643 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.206367  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:07.206931  1643 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.209939  1722 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.212648  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:07.213225  1576 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.218612  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:07.219236  1573 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.219542  1573 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:07.220873  1576 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.222116  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:07.223237  2760 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.223742  2760 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:07.224740  2760 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.229729  1722 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:07.233417  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:07.233965  1651 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.234274  1651 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:07.234987  1651 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.236167  1651 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.248690  1651 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.254746  2784 raft_consensus.cc:993] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: : Instructing follower da3b57321031416cba9db75450ef302a to start an election
I20260709 13:01:07.255259  2784 raft_consensus.cc:1081] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Signalling peer da3b57321031416cba9db75450ef302a to start an election
I20260709 13:01:07.253885  1573 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.256846  2377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc"
dest_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:41530
I20260709 13:01:07.257380  2377 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.257848  2377 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.270020  2377 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.257380  1573 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.285912  2788 tablet_copy_client.cc:806] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:07.289793  2788 tablet_copy_client.cc:670] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:07.292779  2740 raft_consensus.cc:993] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: : Instructing follower da3b57321031416cba9db75450ef302a to start an election
I20260709 13:01:07.293344  2784 raft_consensus.cc:1081] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Signalling peer da3b57321031416cba9db75450ef302a to start an election
I20260709 13:01:07.296600  2380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c"
dest_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:41530
I20260709 13:01:07.297112  2380 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.297451  2380 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.311028  2781 tablet_copy_client.cc:806] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:07.311658  2781 tablet_copy_client.cc:670] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:07.313196  1643 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.321614  1573 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.323383  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:07.324411  2760 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.325080  2760 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:07.320844  1471 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.326130  2760 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.339807  2380 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.353484  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:07.354147  1570 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.354446  1570 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:07.355140  1570 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.355296  2377 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:07.359578  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:07.360293  1573 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.360618  1573 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:07.361537  1573 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.356288  1570 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.366614  2788 tablet_copy_client.cc:538] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:07.373376  2781 tablet_copy_client.cc:538] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:07.378291  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:07.379065  1723 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.385342  1473 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.404577  1723 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:07.407429  1573 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.411722  2788 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:07.417043  2380 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:07.419261  1570 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:07.420486  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:07.422045  2791 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.422556  2791 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:07.423511  2791 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.425909  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:07.426527  1722 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.437394  2380 raft_consensus.cc:1217] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:07.440795  1573 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:07.441325  1722 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:07.441758  2781 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:01:07.446223  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:07.446900  2740 raft_consensus.cc:993] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: : Instructing follower 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.447400  2690 raft_consensus.cc:1081] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Signalling peer 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:07.447948  2791 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.449615  2791 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:07.451830  2791 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.453471  1473 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.479077  1722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:07.479785  1722 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:07.480194  1722 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.482141  1473 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.491318  1722 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.495097  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:07.496008  1570 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:07.496363  1570 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:07.497376  1570 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.501171  1570 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.505018  2788 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:07.505971  2788 tablet_bootstrap.cc:492] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:07.506665  2788 ts_tablet_manager.cc:1403] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.095s	user 0.030s	sys 0.000s
I20260709 13:01:07.516129  2788 raft_consensus.cc:359] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.516997  2788 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:07.517673  2788 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.520121  2788 ts_tablet_manager.cc:1434] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 13:01:07.524675  1722 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:07.531998  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:07.532913  1645 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:07.551527  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-996f59fd8f2340a49403b3e9ef4c8e86 received from {username='slave'} at 127.0.0.1:45556
I20260709 13:01:07.564823  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-996f59fd8f2340a49403b3e9ef4c8e86 on tablet 996f59fd8f2340a49403b3e9ef4c8e86 with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:07.564172  2793 raft_consensus.cc:1064] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:07.585391  2793 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:07.554260  1645 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.548437  1570 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:07.595466  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:07.596609  2758 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:07.609907  2758 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:07.614131  2758 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:07.631770  2781 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:07.633284  2781 tablet_bootstrap.cc:492] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:01:07.633965  2781 ts_tablet_manager.cc:1403] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.193s	user 0.059s	sys 0.053s
I20260709 13:01:07.639022  2781 raft_consensus.cc:359] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.639919  2781 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:01:07.642648  2781 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:07.645215  2781 ts_tablet_manager.cc:1434] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:01:07.650349  1586 tablet_copy_service.cc:342] P 4f737473248e4c059fa74af3b8e1f77e: Request end of tablet copy session da3b57321031416cba9db75450ef302a-62836cb7c4a548b5a6c44c89782a958c received from {username='slave'} at 127.0.0.1:45556
I20260709 13:01:07.651026  1586 tablet_copy_service.cc:434] P 4f737473248e4c059fa74af3b8e1f77e: ending tablet copy session da3b57321031416cba9db75450ef302a-62836cb7c4a548b5a6c44c89782a958c on tablet 62836cb7c4a548b5a6c44c89782a958c with peer da3b57321031416cba9db75450ef302a
I20260709 13:01:07.666644  2380 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:07.668395  2718 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:07.683202  1643 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:07.683738  1573 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:07.687566  2760 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:07.696430  1643 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.706506  2377 raft_consensus.cc:1217] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:07.704872  2807 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:01:07.724833  2808 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:07.726016  1643 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.740613  2718 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:01:07.761968  1472 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:07.767313  1570 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:07.761916  2760 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:07.862512  2718 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:07.871347  1722 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:07.874297  2791 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:08.794708  2815 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.800211  2815 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.800088  2817 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.801656  2817 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.803260  2815 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:08.805814  2760 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.953s	user 0.013s	sys 0.003s
I20260709 13:01:08.807857  2817 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.810225  2815 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.821380  2815 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:08.822124  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.013s	sys 0.036s
W20260709 13:01:08.826231  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.015s	sys 0.036s
I20260709 13:01:08.853014  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 761) took 1114 ms. Trace:
I20260709 13:01:08.853309  2815 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.854424  1651 rpcz_store.cc:276] 0709 13:01:07.738618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.739098 (+   480us) service_pool.cc:224] Handling call
0709 13:01:07.739387 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.739741 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.739766 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.740308 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.838777 (+ 98469us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:07.838838 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.838978 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:08.852606 (+1013628us) raft_consensus.cc:1685] finished
0709 13:01:08.852631 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.852807 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92,"raft.queue_time_us":2689,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":97965,"threads_started":1,"wal-append.queue_time_us":138437}
I20260709 13:01:08.866606  2812 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:08.867851  2812 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.857494  2817 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:08.871627  2817 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.871204  2812 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.874624  2817 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.877542  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:08.879972  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:08.880721  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:08.886484  2815 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:08.888337  2815 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.891127  2815 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.893989  2815 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:07.765230  1643 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:08.895215  2815 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.901172  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.882802  2818 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.894326  2816 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.911262  2816 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:08.913916  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.916481  2815 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.918355  2760 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.918676  2816 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:08.912806  2818 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.922196  2793 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:08.925942  2793 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.934161  2793 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:08.934233  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.935138  2793 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.927677  2818 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.938750  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.941964  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.943953  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.944391  2818 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.944994  2818 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.925741  2760 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.955883  2740 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:08.956899  2740 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:08.959896  2740 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.961601  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:08.971376  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:08.973006  2818 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:08.973507  2760 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:08.970729  2740 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.974494  2740 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:08.979816  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 763) took 1215 ms. Trace:
I20260709 13:01:08.950803  2793 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:08.980291  1643 rpcz_store.cc:276] 0709 13:01:07.764477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.764726 (+   249us) service_pool.cc:224] Handling call
0709 13:01:07.764963 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.765132 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:08.924838 (+1159706us) log.cc:844] Serialized 14 byte log entry
0709 13:01:08.979434 (+ 54596us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.979511 (+    77us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:08.979553 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.979625 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.979754 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":129,"thread_start_us":54190,"threads_started":1}
I20260709 13:01:08.986032  2825 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.015112  2753 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.025676  2831 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.027107  2831 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:08.981827  2760 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.001458  2822 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:08.987846  2793 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:08.985692  2826 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.022460  2753 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.014706  2825 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.017239  2818 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.032792  2740 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.054845  2812 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.044516  2793 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.154029  2818 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.034380  2760 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:09.097213  2815 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.153450  2817 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.285533  2618 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.168618  2828 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.306159  2821 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.286382  2618 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:09.314761  2740 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.099988  2826 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.285662  2830 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.335749  2836 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.336515  2836 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.338734  2836 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.339285  2836 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.307030  2819 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.307020  2824 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.181185  2812 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.188047  2793 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.344467  2618 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.288733  2815 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.083097  2822 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.323047  2740 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.309759  2817 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.345830  2818 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.352205  2832 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.353377  2824 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.352880  2823 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:09.385648  1505 debug-util.cc:398] Leaking SignalData structure 0x7b0800138880 after lost signal to thread 434
I20260709 13:01:09.393496  2834 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.363612  2819 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.352021  2833 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.396234  2835 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.347759  2836 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.331446  2821 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:09.464581  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08000389e0 after lost signal to thread 1508
I20260709 13:01:09.397396  2818 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.348598  2830 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.472918  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:09.330262  2828 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.479380  2815 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.480455  2837 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.481755  2760 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.468511  2829 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.483655  2831 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.481320  2837 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.505340  2819 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.485953  2826 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.501962  2821 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.489624  2753 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.465830  2834 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.471050  2835 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.477810  2818 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:09.527968  2822 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.462637  2823 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.459462  2832 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.520782  2831 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.523475  2825 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.537535  2793 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.468993  2833 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.610447  2823 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.543185  2812 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.595219  2793 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.546113  2760 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.631570  2839 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.573581  2825 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.623868  2740 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.614130  2826 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.546026  2831 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.634042  2823 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.651117  2793 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.654397  2760 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.713551  2740 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.529029  2829 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.737031  2740 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:09.739784  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.996s	user 0.017s	sys 0.017s
I20260709 13:01:09.719663  2818 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.741449  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:09.745062  2834 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.746028  2834 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.746702  2834 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.749384  2380 raft_consensus.cc:1217] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 LEARNER]: Deduplicated request from leader. Original: 3.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 13:01:09.750212  1724 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:09.687934  2825 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.755445  2832 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.721824  2824 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.702536  2823 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.760193  2837 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:09.763157  2829 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.662483  2830 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.658991  2839 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.723834  2835 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.766629  2831 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.768702  2828 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.771114  2793 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.773016  2785 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.761986  2833 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.772585  2818 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.692266  2826 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.790949  2847 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:09.799130  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 511989us
I20260709 13:01:09.800047  2690 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.802335  2812 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:09.807667  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.064s	user 0.018s	sys 0.018s
I20260709 13:01:09.815963  2856 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.819989  2858 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.831324  2845 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.831970  2857 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.833243  1576 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:09.834810  1572 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:09.841921  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.845216  2844 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.847404  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.849381  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.855396  2861 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.857406  2862 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.864495  2843 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.811852  2841 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.868710  2740 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.868813  2832 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.881507  2690 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.881366  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 720) took 2147 ms. Trace:
I20260709 13:01:09.910593  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.879382  2784 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:09.924553  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65096.6us
I20260709 13:01:09.924186  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:09.927621  1573 rpcz_store.cc:276] 0709 13:01:07.731769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.731965 (+   196us) service_pool.cc:224] Handling call
0709 13:01:07.732646 (+   681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.733123 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.733151 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.743336 (+ 10185us) log.cc:844] Serialized 564 byte log entry
0709 13:01:09.879204 (+2135868us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:09.879234 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.879332 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.879376 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:09.879386 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.879524 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":2135596,"threads_started":1,"wal-append.queue_time_us":810317,"wal-append.run_cpu_time_us":2334,"wal-append.run_wall_time_us":1005827}
I20260709 13:01:09.894178  2821 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.932147  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.934633  2690 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:09.903518  2824 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.892652  2818 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.939791  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.948447  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.893582  2812 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.897018  2857 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.886487  2785 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.940770  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.924770  2858 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.929175  2844 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:09.896976  2856 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.904714  2847 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:09.948613  2760 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:08.111511   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:10.052469  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.023774  1472 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 202) took 2299 ms. Trace:
I20260709 13:01:10.053118  2380 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:01:10.049431  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.285s	user 0.011s	sys 0.043s
I20260709 13:01:09.932036  2793 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:10.057879  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.294s	user 0.012s	sys 0.043s
I20260709 13:01:09.933987  2835 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:09.963289  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:09.936261  2861 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.075210  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 762) took 2323 ms. Trace:
I20260709 13:01:09.999022  2863 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.026337  2855 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.007598  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:09.973790  2834 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.928920  2845 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.058720  2830 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.076426  2379 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:09.974761  2825 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.932420  2832 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.016641  2823 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.096010  2850 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:09.939775  2821 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.075964  1645 rpcz_store.cc:276] 0709 13:01:07.747563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.750047 (+  2484us) service_pool.cc:224] Handling call
0709 13:01:07.750412 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.750821 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.750850 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.752788 (+  1938us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.069178 (+2316390us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:10.069244 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.069316 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.070452 (+  1136us) raft_consensus.cc:1685] finished
0709 13:01:10.070476 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.070658 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":9979,"spinlock_wait_cycles":1239296,"thread_start_us":2305238,"threads_started":1,"wal-append.queue_time_us":821154,"wal-append.run_cpu_time_us":1950,"wal-append.run_wall_time_us":1022699}
I20260709 13:01:10.100787  2856 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.110399  2861 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.069766  2816 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:10.065835  2839 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:09.938225  2862 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.078434  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.113426  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:09.949016  2824 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.122128  2824 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.122661  2824 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.122751  2857 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.123117  2824 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.120057  2380 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.077167  2863 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:10.130151  1691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.022s
W20260709 13:01:10.130578  1691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.000s	sys 0.023s
I20260709 13:01:10.131398  2818 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.103327  2826 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.101820  2825 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.079389  2835 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.131247  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.135612  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.146389  2380 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:10.073926  2793 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.045536  2864 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 13:01:10.077474  2855 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.141963  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:10.143043  2821 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.111424  2785 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.141400  2852 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:09.935206  2784 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.153817  2812 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.102944  2830 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.156600  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:09.948010  2843 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.053166  1472 rpcz_store.cc:276] 0709 13:01:07.716841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.717007 (+   166us) service_pool.cc:224] Handling call
0709 13:01:07.736095 (+ 19088us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5400002490. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c4a0b2 00007fc839bc3a89 00007fc839b5bda6 00007fc839b2d088 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc
0709 13:01:10.016480 (+2280385us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:07.840100 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:07.840909 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:07.840955 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:01:07.841005 (+    50us) tablet.cc:662] Decoding operations
0709 13:01:07.841501 (+   496us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:07.841578 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:07.841632 (+    54us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:07.841910 (+   278us) tablet.cc:701] Row locks acquired
0709 13:01:07.841939 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:01:07.842147 (+   208us) write_op.cc:270] Start()
0709 13:01:07.842333 (+   186us) write_op.cc:276] Timestamp: P: 1783602067842082 usec, L: 0
0709 13:01:07.842365 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:01:07.843295 (+   930us) log.cc:844] Serialized 541 byte log entry
0709 13:01:07.846451 (+  3156us) op_driver.cc:464] REPLICATION: finished
0709 13:01:07.847012 (+   561us) write_op.cc:301] APPLY: starting
0709 13:01:07.847101 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:07.847233 (+   132us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:07.847364 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:07.847681 (+   317us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:07.847874 (+   193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:07.847934 (+    60us) write_op.cc:312] APPLY: finished
0709 13:01:07.848190 (+   256us) log.cc:844] Serialized 23 byte log entry
0709 13:01:07.856663 (+  8473us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:07.856781 (+   118us) write_op.cc:454] Released schema lock
0709 13:01:07.856839 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:07.857422 (+   583us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:07.857483 (+    61us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13085,"reported_tablets":1,"spinlock_wait_cycles":25357184,"child_traces":[["op",{"apply.queue_time_us":307,"apply.run_cpu_time_us":2094,"apply.run_wall_time_us":10257,"num_ops":1,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":3479,"prepare.run_wall_time_us":4825,"replication_time_us":3926}]]}
I20260709 13:01:10.158154  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:09.954998  2841 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.169729  2829 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.180464  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.119024  2816 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.160025  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.176872  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.176990  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:10.158962  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.178426  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.182137  2785 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:10.145212  2825 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.180824  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.206533  1651 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:10.206844  2842 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.194104  1649 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:10.158486  2844 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.204769  1643 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:10.179050  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.202589  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:10.171893  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1336) took 1257 ms. Trace:
I20260709 13:01:10.183082  2376 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.212821  1575 rpcz_store.cc:276] 0709 13:01:08.898424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.733384 (+834960us) service_pool.cc:224] Handling call
0709 13:01:09.733826 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.870387 (+136561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.870405 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.870433 (+    28us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:09.870450 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.155401 (+284951us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400026cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:10.155449 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.155564 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313657088}
I20260709 13:01:10.215061  1571 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:10.125157  2850 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.216620  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.207307  2826 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.210994  2817 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.207733  1642 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:10.215459  2842 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:10.204331  2829 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.228551  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:10.231031  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.221413  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.233741  1575 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.227552  2854 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.242058  2853 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.242213  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.242641  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.243026  1722 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:10.242872  2853 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.242295  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.223804  2863 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.227854  2864 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.247524  2866 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.247511  2867 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.248138  1722 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:10.227962  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.251263  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.225734  2826 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.235694  2824 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.249282  2864 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.225284  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:10.249581  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.255491  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1488) took 1377 ms. Trace:
I20260709 13:01:10.255749  2840 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.171985  2852 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.235105  2842 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.246708  2832 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.234241  2841 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.263465  1569 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
W20260709 13:01:10.268927  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46733.1us
I20260709 13:01:10.269248  2834 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.269825  2834 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:10.256254  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.253345  2858 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.258255  2866 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.267063  2854 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.272089  2834 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.277141  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.265368  2840 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:10.280023  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.263777  1648 rpcz_store.cc:276] 0709 13:01:08.878219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.249915 (+1371696us) service_pool.cc:224] Handling call
0709 13:01:10.253729 (+  3814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.254993 (+  1264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.255044 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.255095 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.255139 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.255221 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.255334 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.269402  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.273804  2830 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.263067  2843 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.258636  2850 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.263067  2867 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.270292  1567 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.290884  2855 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:10.294013  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08002a7660 after lost signal to thread 1600
I20260709 13:01:10.278765  2866 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.269930  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:10.300436  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.300148  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.288277  2840 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.306758  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.309489  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.312211  2845 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.305464  2825 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.313138  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.313884  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.327064  1575 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:10.323071  2853 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.325395  2829 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.330802  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 260) took 1007 ms. Trace:
I20260709 13:01:10.312564  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.304136  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.321841  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:10.314431  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44626.1us
I20260709 13:01:10.337554  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1515) took 1175 ms. Trace:
I20260709 13:01:10.325347  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.338788  1716 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.331336  2377 rpcz_store.cc:276] 0709 13:01:09.322972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.185065 (+862093us) service_pool.cc:224] Handling call
0709 13:01:10.185549 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.235597 (+ 50048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.238545 (+  2948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.244970 (+  6425us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.246958 (+  1988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.329589 (+ 82631us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b54004acc54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:10.330522 (+   933us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.330719 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95111040}
I20260709 13:01:10.344960  2845 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.297364  2847 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.338558  1723 rpcz_store.cc:276] 0709 13:01:09.159248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.159497 (+   249us) service_pool.cc:224] Handling call
0709 13:01:09.159839 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.346316 (+186477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.346363 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.346421 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:09.346464 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.333923 (+987459us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400213754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:10.333963 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.334094 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428244736}
I20260709 13:01:10.348850  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.324003  2855 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.349700  1720 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.326377  2826 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.353580  2836 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.353889  2867 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.353509  2845 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:10.356070  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.360440  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.365615  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.366410  1645 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:10.352385  2855 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:10.368520  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.359401  2342 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:10.367398  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.370571  1723 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:01:10.366251  1505 debug-util.cc:398] Leaking SignalData structure 0x7b0800263140 after lost signal to thread 1675
I20260709 13:01:10.358100  2864 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.360133  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.356398  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.389226  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:10.369814  2855 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:10.375866  1648 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:10.371668  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.382648  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.501403  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.558127  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.717545  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.640589  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.644161  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.722378  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.519809  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1514) took 1365 ms. Trace:
I20260709 13:01:10.724639  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.382926  1646 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:10.732942  1721 rpcz_store.cc:276] 0709 13:01:09.154162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.160000 (+1005838us) service_pool.cc:224] Handling call
0709 13:01:10.160280 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.176081 (+ 15801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.176121 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.176169 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:10.176197 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.504066 (+327869us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:10.504135 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.519724 (+ 15589us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35805056}
W20260709 13:01:10.749990  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 91108.6us
I20260709 13:01:10.417514  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.736043  1573 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:10.587783  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.751971  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.623636  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1327) took 1734 ms. Trace:
I20260709 13:01:10.594944  2784 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.639721  2793 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.745527  2862 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.599520  2842 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.585446  2816 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.767783  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.768432  1573 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:10.768990  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.769263  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.505090  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1334) took 1607 ms. Trace:
I20260709 13:01:10.581568  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.564566  2854 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.583349  2835 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:10.644022  2845 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.460327  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.577066  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1522) took 1112 ms. Trace:
I20260709 13:01:10.473333  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.749428  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1520) took 1361 ms. Trace:
I20260709 13:01:10.432312  2852 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:10.747673  2374 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:10.778616  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.776341  1647 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:10.778996  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.747510  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.390249  2847 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:10.446900  2870 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.158s	user 0.003s	sys 0.003s
I20260709 13:01:10.754339  1570 rpcz_store.cc:276] 0709 13:01:08.889331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.015207 (+1125876us) service_pool.cc:224] Handling call
0709 13:01:10.015563 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.072231 (+ 56668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.072269 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.072313 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:10.072383 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.623284 (+550901us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b5400309b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:10.623343 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.623515 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129794944}
I20260709 13:01:10.756062  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.752004  1572 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:10.760522  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.460191  1569 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:10.436908  2753 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:10.751634  1650 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:10.588263  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.574056  2822 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.603878  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.581557  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.418382  2836 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.772724  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 200) took 1819 ms. Trace:
I20260709 13:01:10.873347  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.874122  1575 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:10.575179  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.601712  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.774142  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.807765  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:10.856380  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.873533  1471 rpcz_store.cc:276] 0709 13:01:08.943545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.946048 (+  2503us) service_pool.cc:224] Handling call
0709 13:01:10.762464 (+1816416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1067777,"reported_tablets":1}
I20260709 13:01:10.776283  1724 rpcz_store.cc:276] 0709 13:01:09.464227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.465000 (+   773us) service_pool.cc:224] Handling call
0709 13:01:10.576848 (+1111848us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400171bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:10.576972 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636007168}
I20260709 13:01:10.797593  1725 rpcz_store.cc:276] 0709 13:01:09.388283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.388641 (+   358us) service_pool.cc:224] Handling call
0709 13:01:10.557930 (+1169289us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:10.749370 (+191440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5622144}
I20260709 13:01:10.857506  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:10.767833  2859 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.867292  2854 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.756806  2866 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.806950  2847 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.888839  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.855214  2753 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:10.838719  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08002c7ba0 after lost signal to thread 1749
I20260709 13:01:10.756165  2840 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.839020  2835 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:10.886725  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.858533  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:10.778246  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1524) took 1303 ms (client timeout 1679 ms). Trace:
I20260709 13:01:10.867937  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.881320  1723 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:10.946302  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.870589  2853 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:10.945657  1474 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 213) took 1752 ms. Trace:
I20260709 13:01:10.958144  1572 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:10.948740  1651 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:10.777654  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.769764  2837 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:01:10.951143  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.958153  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.884416  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.882449  2376 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:10.961755  1474 rpcz_store.cc:276] 0709 13:01:09.020921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.024484 (+  3563us) service_pool.cc:224] Handling call
0709 13:01:10.771663 (+1747179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1029595,"reported_tablets":1}
I20260709 13:01:10.893968  2859 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:11.004426  2866 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.894832  2847 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.021189  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.031162  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.022032  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:10.957454  1718 rpcz_store.cc:269] 0709 13:01:09.475208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.368017 (+892809us) service_pool.cc:224] Handling call
0709 13:01:10.778187 (+410170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:11.024828  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:10.902720  2862 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.035105  1574 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:11.036244  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.037539  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.038800  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.975214  2852 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.039646  2852 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.040299  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.025857  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.903543  2753 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.029572  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.045413  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.062155  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.062388  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.901479  2854 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.063660  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.063817  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.064623  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.065114  2854 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:10.879328  2822 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:11.065448  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.040282  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:10.881687  2836 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:10.951170  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.005903  2853 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:11.076321  2378 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:11.075668  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.094201  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.095444  2853 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:11.095610  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.081292  1575 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:11.102662  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.083133  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.092614  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.111653  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.012794  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.098942  1472 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:11.028131  2841 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:10.960795  2835 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.165743  1717 raft_consensus.cc:2410] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:11.167315  2835 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:11.028717  1567 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
W20260709 13:01:11.146622  1505 debug-util.cc:398] Leaking SignalData structure 0x7b0800318840 after lost signal to thread 2404
W20260709 13:01:11.114030  2834 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.826s	user 0.015s	sys 0.002s
I20260709 13:01:11.147238  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.149391  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.152971  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.152161  2847 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.153156  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.154670  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.042883  2371 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
W20260709 13:01:11.189193  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 333642us
I20260709 13:01:11.190984  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.225167  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:11.209115  1505 debug-util.cc:398] Leaking SignalData structure 0x7b0800312da0 after lost signal to thread 2788
W20260709 13:01:11.178344  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 373890us
I20260709 13:01:11.091428  2840 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:11.207027  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.042447  2862 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.082312  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.096661  2852 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:11.090157  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:10.770738  1568 rpcz_store.cc:276] 0709 13:01:08.897808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.033199 (+1135391us) service_pool.cc:224] Handling call
0709 13:01:10.033601 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.088727 (+ 55126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.092878 (+  4151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.102036 (+  9158us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.104171 (+  2135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.504788 (+400617us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b540030a054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:10.504851 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.504993 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93772672}
I20260709 13:01:11.232427  1647 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:11.234851  1646 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:11.234961  1643 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:11.238358  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.193398  2837 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.205390  2853 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.206278  1642 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:10.965291  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.209223  2834 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.196585  2841 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:11.281662  1651 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:11.245227  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:11.349395  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 399396us
W20260709 13:01:11.241626  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08002ef580 after lost signal to thread 2790
I20260709 13:01:11.089352  2842 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:11.371152  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 126066us
I20260709 13:01:11.264365  2816 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.260763  2862 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.235929  2859 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:11.374919  2379 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:11.305974  2822 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:11.178786  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.374121  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.284741  2852 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.374517  2815 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.393532  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:11.372017  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08002968a0 after lost signal to thread 2803
I20260709 13:01:11.394866  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.395864  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.409412  1720 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
W20260709 13:01:11.375360  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.552s (SENT) [suppressed 2 similar messages]
W20260709 13:01:11.417546  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 179173us
I20260709 13:01:11.418870  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.381098  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:11.425194  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51167.9us
I20260709 13:01:11.416172  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.428032  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.428725  1650 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
W20260709 13:01:11.429657  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70265.7us
I20260709 13:01:11.429935  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.434098  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.369287  2760 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:01:11.356719  2837 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:11.451575  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:11.399278  2856 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.374904  2853 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:11.457764  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:11.459811  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.462018  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.431282  2859 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.429497  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.429382  1648 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:11.433296  2816 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.468223  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:11.458586  2835 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.470541  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.481563  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.486070  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.463718  2836 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:11.472239  1716 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:11.473004  2879 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.474063  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:11.468465  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93149.9us
I20260709 13:01:11.421847  2852 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:11.489632  2855 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.004s
I20260709 13:01:11.477391  2815 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.485052  2835 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.524892  2835 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.525053  2879 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.531474  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.532917  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.533469  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.534118  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.534483  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.536304  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.538436  2853 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:11.538677  2841 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:11.540396  2837 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:11.505438  2847 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:11.514345  2875 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:11.542486  2875 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.505048  2859 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.543258  2859 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.543673  2859 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.464679  2856 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.493432  2869 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.493206  2858 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.541500  2834 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:11.542900  2875 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:11.468196  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.549265  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.554945  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.534582  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:11.558514  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 90412.7us
W20260709 13:01:11.552134  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 70646.7us
I20260709 13:01:11.560236  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:11.563952  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:11.570008  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:11.571933  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.575937  2866 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:11.577921  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:11.580417  2873 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.580905  2873 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.581329  2873 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.584486  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:11.593300  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:11.527884  2884 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:11.597965  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 129818us
I20260709 13:01:11.600062  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:11.547763  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.573236  2819 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.617789  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:11.626317  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:11.630154  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.632635  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.573052  2847 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.648587  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.546895  2855 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:11.637828  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.588048  2815 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.587148  2836 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:11.644670  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84934us
I20260709 13:01:11.568125  2856 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:11.604645  2884 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.655090  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.661504  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.662600  2896 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.667933  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.622298  2837 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.615244  2819 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.680678  1650 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:11.692178  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.693321  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.693485  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.694747  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.597011  2834 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:01:11.658389  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.710842  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:11.718245  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.670043  2836 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.647275  2847 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.651371  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:01:11.702265  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 47308.3us
I20260709 13:01:11.675922  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.658875  2853 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.710851  1573 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:11.663275  2815 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:11.711233  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.726423  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.714367  1576 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:11.728404  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:11.678529  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:11.695248  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:11.728813  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.719794  2793 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.675805  2846 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:11.734586  2880 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:01:11.702772  2837 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:11.779793  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.712926  2881 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.701661  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 172) took 1300 ms. Trace:
I20260709 13:01:11.677526  2884 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.714403  2819 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.756805  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.756986  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:11.732648  2897 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:11.758185  2815 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.724963  2373 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:11.773423  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 44740us
I20260709 13:01:11.728747  2836 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.781276  2378 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:11.782651  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:11.737284  2847 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:11.762621  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:11.786270  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68133.9us
I20260709 13:01:12.079829  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:11.713994  2896 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:11.819620  2881 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:11.751616  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:12.107813  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 755) took 1690 ms. Trace:
I20260709 13:01:11.736196  2852 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:11.821797  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44175.4us
I20260709 13:01:11.769711  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 29) took 1166 ms. Trace:
I20260709 13:01:11.788599  2858 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:11.788177  2837 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:11.857561  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 260020us
I20260709 13:01:11.856813  1717 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:11.851545  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:11.768638  2853 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:12.108485  1572 rpcz_store.cc:276] 0709 13:01:10.417721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.083359 (+665638us) service_pool.cc:224] Handling call
0709 13:01:11.084171 (+   812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.084632 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.084663 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.106244 (+ 21581us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.401305 (+295061us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:11.401341 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.401405 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.107510 (+706105us) raft_consensus.cc:1685] finished
0709 13:01:12.107539 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.107720 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":121,"thread_start_us":194530,"threads_started":1,"wal-append.queue_time_us":383780}
I20260709 13:01:12.114768  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:12.109757  2874 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.067337  2845 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:12.108577  2892 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.251s	user 0.002s	sys 0.007s
I20260709 13:01:12.064685  2882 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.166244  1646 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:12.178344  1644 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:12.183121  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68451.8us
I20260709 13:01:12.111850  2881 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.107981  2846 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:12.187206  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:12.107304  2816 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:12.191859  1724 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:12.116709  1642 rpcz_store.cc:276] 0709 13:01:10.587760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.587931 (+   171us) service_pool.cc:224] Handling call
0709 13:01:11.728906 (+1140975us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540005cdd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:11.751570 (+ 22664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736149248}
W20260709 13:01:12.190110  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 110359us
I20260709 13:01:12.112051  2841 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.117426  2858 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:12.199019  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:12.199615  2858 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.200296  1648 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:12.193208  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 430712us
I20260709 13:01:12.109364  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:12.178867  2852 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:12.203764  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:12.175464  2824 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:12.185019  2784 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.183179  2855 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.182511  2903 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:12.194710  2856 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:12.201567  2897 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.304524  2897 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:12.180016  2907 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.178761  2874 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.192288  2845 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.305122  2897 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:12.328665  2862 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.930s	user 0.007s	sys 0.007s
I20260709 13:01:12.321938  2907 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.286968  2824 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.335428  2824 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.338418  2845 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:12.181178  2882 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.261631  2376 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:12.307149  2846 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:12.329418  2862 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:12.349327  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 687929us
I20260709 13:01:12.305758  2856 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.353673  2856 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.287297  2841 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.360307  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:12.293399  2784 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.366187  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:12.368925  1616 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:11.823922  2374 rpcz_store.cc:276] 0709 13:01:10.374968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.744837 (+369869us) service_pool.cc:224] Handling call
0709 13:01:10.747234 (+  2397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.747518 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:11.081731 (+334213us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.555624 (+473893us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b
0709 13:01:11.613127 (+ 57503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:11.616557 (+  3430us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:11.646310 (+ 29753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.657474 (+ 11164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.675751 (+ 18277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42869,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":183,"spinlock_wait_cycles":66095872,"thread_start_us":491376,"threads_started":1,"wal-append.queue_time_us":497244}
W20260709 13:01:12.371414  2905 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.193s	user 0.002s	sys 0.000s
I20260709 13:01:12.308461  2903 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:12.375349  2835 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.019s
I20260709 13:01:12.373782  2816 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:12.416193  2816 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:12.416831  2816 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.374059  2824 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:12.376997  2904 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:12.377784  2879 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.434012  2784 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:12.445446  2859 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.901s	user 0.022s	sys 0.000s
I20260709 13:01:12.424875  2904 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:12.421175  2824 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:12.457955  2859 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.471952  2904 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:12.455773  2753 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.348s	user 0.014s	sys 0.014s
I20260709 13:01:12.419364  2835 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.481463  2874 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.396684  2837 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:12.474109  2873 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.892s	user 0.016s	sys 0.016s
I20260709 13:01:12.474627  2853 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:12.485116  2815 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.528s	user 0.015s	sys 0.002s
W20260709 13:01:12.493160  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 289531us
I20260709 13:01:12.472163  2824 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:12.439810  2881 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.500301  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:12.507988  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:12.495224  2907 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.496795  2859 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.512151  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:12.513866  2846 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:12.498447  2896 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.487540  2856 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.490309  2753 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.509526  2853 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:12.511596  2862 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:12.499823  2873 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.504413  2815 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:12.511605  2854 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:12.533700  2884 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.676s	user 0.017s	sys 0.004s
I20260709 13:01:12.553213  2879 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:12.554050  2879 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.568586  2879 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.569703  2835 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:12.572817  2856 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.575784  2815 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.576728  2882 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.576512  2858 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.546131  2841 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:12.549693  2837 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:12.580142  2858 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:12.542763  2917 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:01:12.565639  2884 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.599429  2753 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:12.600919  2898 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.908s	user 0.003s	sys 0.004s
W20260709 13:01:12.601002  2836 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.633s	user 0.018s	sys 0.004s
I20260709 13:01:12.601650  2836 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:12.599509  2873 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.598430  2881 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:12.604128  2856 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:12.604818  2856 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.604492  2836 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.608882  2884 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:12.610143  2897 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.610913  2904 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.613637  2897 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:12.614181  2904 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:12.618541  2875 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.064s	user 0.016s	sys 0.016s
I20260709 13:01:12.619076  2875 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.617069  2847 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:12.622159  2875 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:12.623416  2372 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:12.742882  1716 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:12.671279  2847 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:12.791586  2816 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:12.625670  2819 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.744s	user 0.005s	sys 0.012s
W20260709 13:01:13.057682  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.038s	sys 0.037s
W20260709 13:01:13.058492  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.039s	sys 0.038s
W20260709 13:01:13.060670  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 886670us
I20260709 13:01:13.063822  1614 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:13.054129  2816 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.055390  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 772) took 1259 ms. Trace:
I20260709 13:01:13.010185  2819 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.072408  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:13.074895  2378 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 2: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:13.075906  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:13.077646  2379 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:13.072983  1614 connection.cc:256] RPC call timeout handler was delayed by -1.2957476030000521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
I20260709 13:01:13.066097  2856 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.067306  2904 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:13.085255  1614 connection.cc:256] RPC call timeout handler was delayed by -1.0690442070000472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
W20260709 13:01:13.088894  1614 connection.cc:256] RPC call timeout handler was delayed by -1.0597130479999919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
W20260709 13:01:13.089469  1614 connection.cc:256] RPC call timeout handler was delayed by -1.0242301890000363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
I20260709 13:01:13.082989  2836 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.094590  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.088881  2904 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.097291  2753 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.093248  2836 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.107779  2753 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.114871  2753 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.111775  2836 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.071457  1571 rpcz_store.cc:276] 0709 13:01:11.691324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.694134 (+  2810us) service_pool.cc:224] Handling call
0709 13:01:11.694623 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.719131 (+ 24508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.725727 (+  6596us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.823113 (+ 97386us) log.cc:844] Serialized 26 byte log entry
0709 13:01:12.392520 (+569407us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:12.408058 (+ 15538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.612266 (+204208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.612388 (+   122us) raft_consensus.cc:1685] finished
0709 13:01:12.612405 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.950753 (+338348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18762,"raft.queue_time_us":24559,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":16015,"thread_start_us":392026,"threads_started":1,"wal-append.queue_time_us":615987}
I20260709 13:01:13.120047  2819 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.121059  2379 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:13.122663  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:01:13.123521  2824 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.005s
I20260709 13:01:13.124017  2824 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.125216  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:13.126137  2836 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.126518  2824 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.102964  2904 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.126904  2836 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.130460  2753 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.133291  2753 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.137140  2904 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.139622  2904 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:13.140971  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:13.141608  2379 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.150120  2379 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:13.152156  1614 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:13.153402  2904 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.154065  2904 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.155107  2904 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.197322  2376 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.199555  2816 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.200788  2904 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.201356  2904 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.203053  2904 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.204523  2904 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:13.221680  2894 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 1.485s	user 0.017s	sys 0.051s
I20260709 13:01:13.228783  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 173) took 2477 ms. Trace:
I20260709 13:01:13.230108  2373 rpcz_store.cc:276] 0709 13:01:10.751121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.757802 (+  6681us) service_pool.cc:224] Handling call
0709 13:01:10.782368 (+ 24566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.932660 (+1150292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.228336 (+1295676us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54005d25d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:13.228656 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115359616}
I20260709 13:01:13.229393  2895 raft_consensus.cc:1064] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:13.234038  1645 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.234597  2895 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.279512  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 774) took 1585 ms. Trace:
I20260709 13:01:13.280088  1569 rpcz_store.cc:276] 0709 13:01:11.694429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.694574 (+   145us) service_pool.cc:224] Handling call
0709 13:01:11.694873 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.717272 (+ 22399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.720569 (+  3297us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.768043 (+ 47474us) log.cc:844] Serialized 26 byte log entry
0709 13:01:12.105775 (+337732us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:12.115172 (+  9397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.163509 (+ 48337us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.279119 (+1115610us) raft_consensus.cc:1685] finished
0709 13:01:13.279148 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.279417 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23369,"raft.queue_time_us":24257,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":3886,"thread_start_us":286451,"threads_started":1,"wal-append.queue_time_us":398455}
I20260709 13:01:13.307067  2901 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.307889  2901 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.311143  2901 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.314456  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 37) took 1200 ms. Trace:
I20260709 13:01:13.315071  1716 rpcz_store.cc:276] 0709 13:01:12.113787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.116914 (+  3127us) service_pool.cc:224] Handling call
0709 13:01:12.166790 (+ 49876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:12.743692 (+576902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.314013 (+570321us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54003dc2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:13.314348 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315245824}
I20260709 13:01:13.319064  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:13.320478  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.321863  1717 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:13.324927  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.325767  1718 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.325915  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.326613  1717 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.327570  2903 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:13.329259  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.371s	user 0.025s	sys 0.064s
W20260709 13:01:13.329775  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.372s	user 0.025s	sys 0.064s
I20260709 13:01:13.319244  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:13.331961  2856 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.332867  2856 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.333329  2856 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.335740  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.331070  2921 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:13.337308  1720 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:13.337312  2921 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.342166  2895 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:13.342787  2895 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.344738  1722 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.348848  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.349967  1722 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:13.353808  1541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.289s	user 0.000s	sys 0.018s
W20260709 13:01:13.361135  1541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.000s	sys 0.020s
I20260709 13:01:13.361850  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.362658  1716 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:13.363721  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.297s	user 0.014s	sys 0.038s
W20260709 13:01:13.364493  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.298s	user 0.014s	sys 0.039s
W20260709 13:01:13.364848  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.29919e+06us
I20260709 13:01:13.365971  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.356830  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.366792  1723 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:13.367022  1720 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:13.368041  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.374289  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.376848  1720 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:13.379844  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.371192  1717 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:13.368809  2921 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.385614  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.387046  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:13.380712  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.393464  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1337) took 1804 ms (client timeout 1536 ms). Trace:
W20260709 13:01:13.395128  1724 rpcz_store.cc:269] 0709 13:01:11.589316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.589528 (+   212us) service_pool.cc:224] Handling call
0709 13:01:13.393244 (+1803716us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400088254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:13.393380 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236928896}
I20260709 13:01:13.396993  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.397130  1716 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:13.367944  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.398824  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.397992  1724 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.399659  1725 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:13.359505  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 239) took 1769 ms. Trace:
I20260709 13:01:13.400971  2372 rpcz_store.cc:276] 0709 13:01:11.590138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.606535 (+ 16397us) service_pool.cc:224] Handling call
0709 13:01:11.995620 (+389085us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:12.108843 (+113223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.624024 (+515181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.359124 (+735100us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54005e1ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:13.359400 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181897216}
I20260709 13:01:13.385597  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.402820  1717 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.399040  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.405478  1716 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:13.409828  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 768) took 2555 ms. Trace:
I20260709 13:01:13.410403  1645 rpcz_store.cc:276] 0709 13:01:10.854680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.854914 (+   234us) service_pool.cc:224] Handling call
0709 13:01:10.855207 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.235073 (+2379866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.409386 (+174313us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54001ba3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:13.409731 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5470686592}
W20260709 13:01:13.412878  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.013s	sys 0.022s
W20260709 13:01:13.413347  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.013s	sys 0.022s
I20260709 13:01:13.414129  1724 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:13.414196  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 769) took 1724 ms. Trace:
I20260709 13:01:13.394275  2903 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:01:13.414803  1573 rpcz_store.cc:276] 0709 13:01:11.689620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.710046 (+ 20426us) service_pool.cc:224] Handling call
0709 13:01:11.710474 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.710711 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:12.240815 (+530104us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.413803 (+1172988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.413862 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:13.413915 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.413992 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.414132 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"thread_start_us":1172458,"threads_started":1,"wal-append.queue_time_us":325345}
I20260709 13:01:13.416814  2901 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.417270  2901 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:13.393971  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.419735  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.422077  1644 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:13.422755  1718 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:13.423556  1721 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:13.420779  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.418280  2901 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.427201  1573 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:13.427937  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.427804  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.428618  1719 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.430133  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:13.433487  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:13.433703  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:13.434329  1571 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.434314  1567 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.438762  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:13.409946  2895 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.440661  1569 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:13.444252  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:13.445233  1573 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.447243  1719 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:13.420826  1717 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:13.457506  2866 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.458298  1571 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:13.458372  2866 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:13.458348  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65295.1us
I20260709 13:01:13.471326  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.471392  2842 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.472396  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.472649  2819 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.473117  2819 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.473502  2819 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.473408  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:13.474745  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.476701  2824 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.477082  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.477494  2842 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.462512  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:13.478556  2842 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.478598  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.479707  1569 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.358714  1642 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:13.467258  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:13.467859  1573 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:13.486790  2816 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:01:13.489208  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.552s (SENT)
I20260709 13:01:13.489806  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:13.464138  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 98652.6us
I20260709 13:01:13.477123  2824 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.464710  1567 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:13.492938  2824 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:13.496075  1540 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.519s	user 0.020s	sys 0.018s
I20260709 13:01:13.388206  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.496867  1720 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:13.501582  1540 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.525s	user 0.021s	sys 0.018s
W20260709 13:01:13.504012  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.097s	user 0.018s	sys 0.028s
W20260709 13:01:13.504802  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.019s	sys 0.028s
W20260709 13:01:13.505188  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 1.0987e+06us
I20260709 13:01:13.505318  2929 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.505941  2929 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.500110  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:13.511615  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 975830us
I20260709 13:01:13.512225  2917 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:13.513689  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:13.514032  2917 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.514552  2917 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:13.484616  2842 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:13.516500  2929 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.516731  2897 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.518054  2897 raft_consensus.cc:2764] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:13.519168  2897 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.519634  2897 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:13.515519  2917 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.516551  2914 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:01:13.519423  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.527580  2866 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.527980  2927 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.528522  2927 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:13.529059  2927 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.529354  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:13.530114  1572 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:13.538825  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 219690us
I20260709 13:01:13.542722  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:13.540102  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:13.544698  2845 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.520434  2842 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.544720  2917 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.522832  2897 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.553275  2897 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.554343  2822 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.553833  2824 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.554970  2822 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.556067  2822 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:13.566257  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52701.7us
I20260709 13:01:13.567356  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:13.572997  2901 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.576282  2929 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.426290   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:13.581260  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75011.9us
I20260709 13:01:13.582113  1616 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:13.579999  2929 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.586267  1472 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 203) took 2747 ms. Trace:
W20260709 13:01:13.588404  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45755.8us
I20260709 13:01:13.589532  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:13.591326  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:13.594305  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.595623  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.597605  2845 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.602183  2845 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:13.602968  2845 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.597819  2926 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:13.597811  2921 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:13.596760  2895 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:13.608863  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68831.7us
I20260709 13:01:13.609092  1572 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:13.611984  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:13.613147  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 123993us
I20260709 13:01:13.609571  2921 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:13.616565  2921 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.617197  2933 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.621728  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:13.625053  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:13.557237  2824 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.639748  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:13.587971  1472 rpcz_store.cc:276] 0709 13:01:10.838473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.839182 (+   709us) service_pool.cc:224] Handling call
0709 13:01:13.586154 (+2746972us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:11.474669 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:11.525366 (+ 50697us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:11.525409 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:01:11.525432 (+    23us) tablet.cc:662] Decoding operations
0709 13:01:11.525857 (+   425us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:11.525956 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:11.526003 (+    47us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:11.526225 (+   222us) tablet.cc:701] Row locks acquired
0709 13:01:11.526248 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:01:11.526438 (+   190us) write_op.cc:270] Start()
0709 13:01:11.526575 (+   137us) write_op.cc:276] Timestamp: P: 1783602071526387 usec, L: 0
0709 13:01:11.526603 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:01:11.599195 (+ 72592us) log.cc:844] Serialized 599 byte log entry
0709 13:01:13.221454 (+1622259us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b20000058f8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc83391cb91 00007fc83393ea19 00007fc8339098c3 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169 00007fc833912131 00007fc83391142d 000000000038f354
0709 13:01:13.242650 (+ 21196us) op_driver.cc:464] REPLICATION: finished
0709 13:01:13.249985 (+  7335us) write_op.cc:301] APPLY: starting
0709 13:01:13.250062 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:13.250153 (+    91us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:13.250167 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:13.250427 (+   260us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:13.250564 (+   137us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:13.250598 (+    34us) write_op.cc:312] APPLY: finished
0709 13:01:13.251068 (+   470us) log.cc:844] Serialized 23 byte log entry
0709 13:01:13.251309 (+   241us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:13.251430 (+   121us) write_op.cc:454] Released schema lock
0709 13:01:13.251473 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:13.251837 (+   364us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:13.251865 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":9984,"child_traces":[["op",{"apply.queue_time_us":7028,"apply.run_cpu_time_us":1840,"apply.run_wall_time_us":1838,"num_ops":1,"prepare.queue_time_us":298724,"prepare.run_cpu_time_us":14928,"prepare.run_wall_time_us":192806,"raft.queue_time_us":12333,"raft.run_cpu_time_us":6902,"raft.run_wall_time_us":7408,"replication_time_us":1715838,"spinlock_wait_cycles":9789184,"thread_start_us":275386,"threads_started":4,"wal-append.queue_time_us":128726}]]}
I20260709 13:01:13.608577  2926 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:13.612788  2822 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:13.641988  2904 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:13.652794  2816 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:13.675318  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 175360us
I20260709 13:01:13.677825  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.654455  2926 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:13.680462  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68543.7us
I20260709 13:01:13.658485  2822 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.689481  2819 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.656157  2904 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.693289  1720 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
W20260709 13:01:13.695779  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56141.3us
I20260709 13:01:13.697045  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:13.656464  2931 raft_consensus.cc:2764] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:13.679543  2932 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:13.712404  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.713248  2929 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.714005  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:13.714167  2753 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.714253  2895 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.706761  2904 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.726369  2933 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:13.693089  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:13.710862  1569 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:13.728483  2919 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:13.734593  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97331.3us
I20260709 13:01:13.724148  2819 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.730283  2931 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:13.740513  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.740655  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.669255  2816 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.735818  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:13.741604  1717 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:13.723747  2932 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:13.745690  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.749792  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.757999  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.762885  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.763610  1722 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:13.734771  2919 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.766230  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.767530  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:13.767408  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.724798  2935 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.811828  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:13.817447  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50044.8us
I20260709 13:01:13.816515  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.807745  1724 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.819025  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:13.819551  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 83861.6us
I20260709 13:01:13.811020  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.792737  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:13.821483  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 96565.8us
I20260709 13:01:13.822219  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:13.823376  1722 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:13.769407  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.824834  1717 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:13.745107  2895 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:13.741793  2929 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.745186  2933 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:13.745029  2932 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.837139  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.823118  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:13.831774  1539 connection.cc:256] RPC call timeout handler was delayed by -3.26442253700003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:01:13.840047  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.737823  2939 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.854121  1717 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.858749  2935 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:13.854907  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:13.859928  2935 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.808537  1646 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.863659  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:13.864708  2941 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.866384  1717 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:13.811444  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.806401  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.837074  2895 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:13.811962  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.810586  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.871524  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:13.873315  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54377.3us
I20260709 13:01:13.806901  2842 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.874375  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:13.837263  2931 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:13.806876  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.852178  2933 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:13.908809  2945 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.806907  2927 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.855465  2819 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:13.883766  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.884438  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.872262  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.352283  2856 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.902338  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.905330  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:13.906390  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:13.925226  2378 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.911836  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:13.912117  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:13.909610  2933 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.913448  2919 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.914017  2842 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.891079  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:13.915720  1719 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.882645  2873 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:13.827587  2940 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:01:13.885612  2939 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.871344  2938 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.925230  2379 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:13.916836  2377 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:13.936496  2856 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:13.903825  2931 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:13.944674  2372 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 2: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:13.945307  2376 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:13.956125  2939 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:13.937690  2933 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:13.933027  2819 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:13.953521  2931 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:13.961968  2927 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.964217  2856 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.964944  2856 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:13.968099  2856 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:13.939882  2895 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:14.032007  1539 connection.cc:256] RPC call timeout handler was delayed by -2.029232890000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:01:13.914991  1642 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:14.071058  1539 connection.cc:256] RPC call timeout handler was delayed by -1.8393328899999801s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:01:14.071676  1539 connection.cc:256] RPC call timeout handler was delayed by -1.7151328900000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 13:01:14.072096  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5936328899999808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
I20260709 13:01:13.915221  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:14.036283  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.075918  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:14.081888  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.082365  1648 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:14.082239  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:14.082680  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.951514  2929 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:14.088793  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:13.986373  2941 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:13.982144  2945 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:13.969496  2948 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:14.121119  2945 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.122095  2948 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:14.122913  2948 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.046610  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:14.053817  2951 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:01:14.065834  2819 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:14.072577  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5792328900000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2017s for a response
W20260709 13:01:14.132848  1539 connection.cc:256] RPC call timeout handler was delayed by -1.533332889999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
W20260709 13:01:14.133409  1539 connection.cc:256] RPC call timeout handler was delayed by -1.521632889999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
W20260709 13:01:14.133945  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4640328899999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260709 13:01:14.138185  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.141641  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.101565  1651 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:14.102138  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:14.098560  1648 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.107347  2840 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:14.125110  2822 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 13:01:14.128675  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.072252  2895 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.152189  2901 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.153060  2901 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:14.156765  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 589386us
I20260709 13:01:14.157013  2926 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:14.159366  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 744611us
I20260709 13:01:14.163261  2901 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.165764  2884 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.167076  2884 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:14.170061  2380 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.172933  2840 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.173789  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:14.174939  2371 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.160151  2926 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:14.182740  2921 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.012s
I20260709 13:01:14.177294  2840 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.183869  2921 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.186156  1716 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:14.202423  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44394.6us
I20260709 13:01:14.202557  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:14.208521  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.503s (SENT)
I20260709 13:01:14.209659  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:14.227109  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.230618  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.230881  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:14.231899  1573 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:14.234653  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.235922  1571 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:14.236279  1474 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 214) took 3126 ms. Trace:
I20260709 13:01:14.168099  2884 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.236975  1474 rpcz_store.cc:276] 0709 13:01:11.109521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.109763 (+   242us) service_pool.cc:224] Handling call
0709 13:01:14.235723 (+3125960us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153224,"reported_tablets":1,"spinlock_wait_cycles":136064}
I20260709 13:01:14.242303  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.204109  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:14.252766  1644 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
W20260709 13:01:14.220530  2845 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.015s
I20260709 13:01:14.245388  2921 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.264698  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:14.268661  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 101734us
I20260709 13:01:14.256247  2884 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:14.280233  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.281453  1569 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:14.286180  2901 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.287338  2824 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:14.290015  2824 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.288532  2956 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:14.293154  2956 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.295189  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.289043  2919 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.296698  1573 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:14.280527  2932 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.307008  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 111) took 2765 ms. Trace:
I20260709 13:01:14.307641  1471 rpcz_store.cc:276] 0709 13:01:11.541783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.541984 (+   201us) service_pool.cc:224] Handling call
0709 13:01:11.662988 (+121004us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:01:14.306750 (+2643762us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":638562,"reported_tablets":1,"spinlock_wait_cycles":236714880}
I20260709 13:01:14.311291  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:14.312244  1720 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.312711  1725 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.306828  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:14.307164  1647 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.325548  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:14.328584  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 146323us
I20260709 13:01:14.330333  1723 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.330804  2373 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:14.311194  2856 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:14.332371  1721 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.334095  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.335362  1573 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:14.334012  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.338331  1473 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 201) took 2663 ms. Trace:
I20260709 13:01:14.338956  1473 rpcz_store.cc:276] 0709 13:01:11.674665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.675158 (+   493us) service_pool.cc:224] Handling call
0709 13:01:14.338082 (+2662924us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":826646,"reported_tablets":2}
W20260709 13:01:14.340531  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 136708us
I20260709 13:01:14.341774  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.343358  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:14.344303  1643 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.346150  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.346463  2380 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.348184  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.349313  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:14.350298  1647 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:14.350574  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.351657  1569 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:14.332273  2856 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:14.355540  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151691us
I20260709 13:01:14.362183  2837 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:01:14.363442  2856 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.283710  2931 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.371549  2900 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:14.373869  2824 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.376371  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.359678  2956 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.275533  2845 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.388692  1720 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:14.376644  2856 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:14.390417  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:14.398155  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.398087  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.398703  2816 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:14.397982  1719 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.402585  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.392905  2900 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:14.402647  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.553011  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.419087  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.521992  1643 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
W20260709 13:01:14.555797  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.477946  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.430092  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:14.424674  1647 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:14.469691  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50634.7us
I20260709 13:01:14.429689  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:14.532536  2378 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.506s	user 0.014s	sys 0.005s
I20260709 13:01:14.565696  1573 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:14.566097  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.566962  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.532569  2374 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.552134  2835 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:14.559230  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 95897.3us
W20260709 13:01:14.573669  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 178965us
I20260709 13:01:14.483140  2932 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.571874  1569 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:14.574913  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.576900  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.575430  1721 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.577778  2931 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.578593  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.486377  2371 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:14.588318  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.592299  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.580418  2835 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:14.594460  1570 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:14.425868  1746 debug-util.cc:398] Leaking SignalData structure 0x7b08001285c0 after lost signal to thread 434
W20260709 13:01:14.574198  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 144737us
I20260709 13:01:14.575663  2378 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:14.556464  2900 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.584295  1716 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:14.584738  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.584939  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.585446  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.585886  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.586714  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.598146  2948 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.620668  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.599745  2929 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.595628  2835 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.606705  2856 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:14.606843  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 206867us
I20260709 13:01:14.616958  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:14.617828  2904 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.841s	user 0.004s	sys 0.019s
I20260709 13:01:14.625451  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.434594  2816 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.598856  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:14.633311  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70386.9us
I20260709 13:01:14.639386  1572 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:14.646096  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.644703  2856 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.621433  2845 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.650159  1573 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.651712  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:14.627550  1571 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
W20260709 13:01:14.669865  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.669615  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:14.650470  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:14.673928  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:14.678588  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79888.4us
I20260709 13:01:14.650482  2933 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.648209  2929 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:14.681633  1574 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:14.685577  1746 debug-util.cc:398] Leaking SignalData structure 0x7b0800137c20 after lost signal to thread 1508
I20260709 13:01:14.696112  1574 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:14.686084  2935 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.664865  2945 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:01:14.687203  2914 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.689230  1573 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:14.689839  1643 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.690796  2948 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:14.692196  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.693264  2929 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.693925  2946 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:14.694020  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77355.6us
I20260709 13:01:14.667027  2856 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:14.681681  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:14.708310  2935 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:14.713968  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:14.714098  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:14.698189  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.714519  2935 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.692226  1567 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.703883  2941 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:14.681890  2837 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.647903  2904 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:14.707072  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.564s (SENT)
I20260709 13:01:14.693341  2835 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:14.710575  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.687417  2816 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.711586  2931 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:14.712700  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87456.8us
I20260709 13:01:14.702813  1574 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.716917  2862 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.717849  2914 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.720165  1567 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:14.722860  2837 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:14.732388  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.735903  2837 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.737053  1567 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.738405  1567 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.743208  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:14.744724  1574 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.745596  2904 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.729912  2862 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:14.747558  2862 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.748435  2914 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:14.749837  2914 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:14.754122  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72959.7us
I20260709 13:01:14.759238  1573 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:14.758774  2955 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.762355  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:14.767707  1746 debug-util.cc:398] Leaking SignalData structure 0x7b080035d660 after lost signal to thread 1600
W20260709 13:01:14.772792  1746 debug-util.cc:398] Leaking SignalData structure 0x7b080035d640 after lost signal to thread 1601
W20260709 13:01:14.774567  1746 debug-util.cc:398] Leaking SignalData structure 0x7b080035bc20 after lost signal to thread 1675
I20260709 13:01:14.768043  1575 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.777868  1575 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:14.786866  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.788457  2933 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:14.789750  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.790602  2901 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:14.809448  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:14.810707  1716 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.790932  2957 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:14.812613  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.813794  1568 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.815789  1568 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:14.816483  2901 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.816979  2901 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:14.817620  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.719s (SENT)
I20260709 13:01:14.817102  1568 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.818354  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:14.818570  2901 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.792858  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:14.820508  1746 debug-util.cc:398] Leaking SignalData structure 0x7b0800395120 after lost signal to thread 1749
I20260709 13:01:14.821278  2957 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.818602  1568 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.821761  2957 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:14.822319  1746 debug-util.cc:398] Leaking SignalData structure 0x7b0800395a00 after lost signal to thread 2404
I20260709 13:01:14.822795  2919 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:14.822942  2957 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.808097  2884 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:14.813761  1723 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.825001  1569 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.825726  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:14.798935  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73365.9us
W20260709 13:01:14.819944  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.892s (SENT)
I20260709 13:01:14.821306  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.795349  1567 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:14.834172  1573 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:14.820029  2901 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:14.844779  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 124836us
I20260709 13:01:14.835850  2933 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:14.858455  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.844s (SENT)
I20260709 13:01:14.859077  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:14.860562  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:14.860579  2931 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.860589  2884 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:14.863664  2884 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.864636  2875 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:14.865223  2875 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:14.866245  2875 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.843639  2919 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:14.840775  1569 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:14.827978  2816 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:01:14.879052  1573 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:14.885152  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:14.889909  1574 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:14.872489  2957 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.885336  1571 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:14.877296  2835 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.879616  1575 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.910360  1722 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:14.849395  2941 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:14.914856  1570 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:14.913582  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.919s (SENT)
I20260709 13:01:14.919327  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:14.922240  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 101092us
I20260709 13:01:14.923450  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.924327  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:14.925026  2919 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.925940  2760 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:14.928684  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 76714us
I20260709 13:01:14.928932  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:14.930029  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:14.931375  2955 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:14.878762  2822 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:01:14.933703  2822 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:14.934125  2822 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:14.935122  2822 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.936764  2955 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:14.937459  2955 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.938266  1723 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:14.938985  1569 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.940466  1569 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:14.940482  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.944829  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.945200  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:14.943616  1725 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:14.962667  1575 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:14.940100  2955 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:14.974870  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51620.4us
I20260709 13:01:14.982167  1720 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:14.928196  2856 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:14.987401  1725 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:14.975867  2941 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:14.989573  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:14.976523  2929 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:14.993966  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99686.6us
W20260709 13:01:14.994998  2344 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.014s (SENT)
I20260709 13:01:14.995574  2344 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:14.995796  1573 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:15.008791  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.030047  2822 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:15.031726  2967 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:14.998467  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:15.037635  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.043901  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.045013  2946 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:15.044911  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74144us
W20260709 13:01:15.046156  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51274.2us
W20260709 13:01:15.047434  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.055s (SENT)
I20260709 13:01:15.047806  2378 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Advancing to term 2
W20260709 13:01:15.048484  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90206.1us
I20260709 13:01:15.049952  2760 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:15.053436  2966 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:01:15.054751  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65693.8us
I20260709 13:01:15.058422  2378 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 2 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:15.060827  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:15.061685  1647 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.064731  1568 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:15.055876  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.072501  2929 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.074539  2956 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.075516  2929 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:15.075274  2956 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:14.996531  2933 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:14.984869  2955 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:15.079387  2955 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.081892  2955 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:15.025693  2900 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.084896  2900 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:15.085731  1571 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:15.086350  2956 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:14.996155  2953 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.053510  2856 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:15.087193  2953 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.089624  2953 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:15.094085  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.094640  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.095304  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.096925  2837 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.047935  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.105358  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:15.106573  1716 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:15.108124  1570 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.113289  2917 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.114981  1569 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:01:15.115556  1569 pending_rounds.cc:85] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 2
I20260709 13:01:15.115903  1569 pending_rounds.cc:107] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
W20260709 13:01:15.119493  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.121876  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:15.112260  2835 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:01:15.132226  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:15.122294  1569 raft_consensus.cc:2917] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:01:15.131911  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:15.151055  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:15.151073  1570 raft_consensus.cc:606] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.151973  1725 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:15.154229  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:15.145176  2948 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:15.130278  2875 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:01:15.162313  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:15.158341  1647 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:15.165753  1575 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:15.166728  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 111002us
I20260709 13:01:15.168506  2875 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:15.169162  2875 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:15.170565  2875 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.172224  2875 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.175213  2941 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.176718  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:15.178138  2946 raft_consensus.cc:2764] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:15.129786  2760 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:01:15.156574  1569 raft_consensus.cc:2889] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.242229  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:15.250244  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.250746  2953 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.252893  2953 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:15.251245  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.253667  2953 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.267472  2931 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:15.273439  2845 raft_consensus.cc:993] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: : Instructing follower 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:15.274248  2845 raft_consensus.cc:1081] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Signalling peer 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:15.274341  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:15.275791  2845 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:15.164208  2862 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 13:01:15.290047  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:15.289752  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47632.9us
W20260709 13:01:15.290577  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 155567us
W20260709 13:01:15.286172  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46665.1us
I20260709 13:01:15.280581  2931 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:15.277698  2845 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:15.309307  2900 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.304630  2875 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.317493  2845 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.331295  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:15.332417  2975 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:01:15.332867  2973 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:15.333647  2973 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.334084  2973 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.319412  2948 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:15.308899  2901 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 13:01:15.344619  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 113044us
I20260709 13:01:15.344995  2981 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.344791  2948 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.345829  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.345620  2981 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.316726  2822 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:01:15.320590  2904 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:01:15.354928  2900 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.324240  2824 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:01:15.324497  2875 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.372102  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:15.380795  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70952.5us
I20260709 13:01:15.350353  2953 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:15.401567  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 354202us
I20260709 13:01:15.376096  2914 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:15.420785  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 93723.9us
W20260709 13:01:15.421695  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.093s (SENT)
I20260709 13:01:15.348706  2981 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:15.429541  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.429621  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.390712  2929 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:15.428720  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.412377  2914 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:15.435909  2979 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:01:15.435645  2914 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.437201  2914 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.438199  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:15.441912  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69940.6us
I20260709 13:01:15.446614  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:15.452982  2884 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.453529  2884 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:15.380692  2931 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.453003  1717 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:15.458114  2980 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:15.459059  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56825.3us
I20260709 13:01:15.459215  2973 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:15.464548  2976 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.220s	user 0.000s	sys 0.002s
I20260709 13:01:15.446189  2929 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.468984  2982 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.469230  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.469321  1643 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:15.469475  2982 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:15.463730  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.471472  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.473246  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.478580  2904 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.479140  2982 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.480377  1616 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.481082  2982 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.481627  2824 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:15.482295  2824 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:15.482868  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.484105  2822 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.485373  1571 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:15.488163  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70714.1us
W20260709 13:01:15.490671  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.492050  1575 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:15.493432  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.456279  2981 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.468154  2929 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:15.477947  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:15.498255  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.502286  2900 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:15.502485  2835 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.502967  2900 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.502321  2862 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:15.493788  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:15.494294  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.232s (SENT)
I20260709 13:01:15.507283  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:15.510468  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 100631us
I20260709 13:01:15.486842  2953 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:15.512451  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.483485  2824 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:15.514812  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.454631  2884 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.530350  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:15.532372  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:15.533499  1570 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:15.505486  2900 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:15.537113  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:15.539043  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.535552  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:15.541646  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47393.9us
W20260709 13:01:15.543330  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54491.4us
I20260709 13:01:15.516607  2946 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:15.538440  2862 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:15.501327  2988 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:01:15.546487  1571 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:15.567194  2862 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:15.583942  2862 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.584547  2973 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:15.584708  2793 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:15.568426  2900 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.585804  2900 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.564685  2981 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.588980  2981 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.591720  2981 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:15.585186  2793 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:15.551297  2946 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:15.544118  2845 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.625551  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:15.638825  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 101171us
I20260709 13:01:15.641775  2884 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.642047  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:15.643014  1647 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:15.634711  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57965.2us
I20260709 13:01:15.584098  2983 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.569401  2875 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:15.637838  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45270.8us
I20260709 13:01:15.607048  2946 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.592298  2956 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.610090  1471 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:15.626606  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:15.653873  1471 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:15.653852  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.659211  1471 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:15.663194  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.657968  2793 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.668291  2793 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.668484  3001 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:15.654814  2900 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:15.672483  2980 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.663923  2374 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:15.677754  2845 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:15.725281  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54979.9us
I20260709 13:01:15.728096  1725 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:15.726984  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:15.730500  1717 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.701083  2981 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:15.739727  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.749042  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:15.743156  2981 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.721210  3002 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:01:15.773841  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.775987  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.757601  2901 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:01:15.794162  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:15.757460  2793 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:01:15.805944  2793 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.807396  2793 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:15.808239  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52074.4us
I20260709 13:01:15.808522  2793 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.809430  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:15.810036  2793 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.811885  2884 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.813304  2371 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:15.824158  1717 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:15.803661  2824 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.826963  2835 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:15.828688  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.830694  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:15.801972  3000 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.783954  2955 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:01:15.820010  2862 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:15.840591  2998 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:01:15.854921  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78980.6us
I20260709 13:01:15.856498  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:15.850601  2955 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:15.858614  2955 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:15.859884  2955 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.861318  2955 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:15.864595  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.867739  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:15.876073  3004 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:01:15.890945  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.704s (SENT)
I20260709 13:01:15.895759  2862 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:15.898175  3008 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:15.910440  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.910213  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:15.892527  2973 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.924695  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:15.932678  2374 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:01:15.935742  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68439.8us
I20260709 13:01:15.935743  2371 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:15.939821  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:15.925721  2956 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:15.921288  2933 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:15.931305  2946 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:15.947110  3005 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:15.947640  3005 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:15.948052  3005 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:15.944939  2824 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:15.925062  2973 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:15.958520  1688 connection.cc:256] RPC call timeout handler was delayed by -4.1567891509999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 13:01:15.958849  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:01:15.959372  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.960455  1688 connection.cc:256] RPC call timeout handler was delayed by -3.9635043479999581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:01:15.960827  1688 connection.cc:256] RPC call timeout handler was delayed by -3.8237891509999713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
W20260709 13:01:15.961144  1688 connection.cc:256] RPC call timeout handler was delayed by -3.82288915099997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
W20260709 13:01:15.961683  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.958002  2933 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:15.948705  2371 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:15.969027  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:15.965358  2378 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:15.943921  3001 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.975986  2380 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:15.948918  2946 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:15.981743  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:15.979694  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.980044  2374 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:01:15.963548  2901 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.986304  2374 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 4
I20260709 13:01:15.997126  1720 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:15.967697  2824 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:15.999140  1688 connection.cc:256] RPC call timeout handler was delayed by -3.6392891509999572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
W20260709 13:01:15.999823  1688 connection.cc:256] RPC call timeout handler was delayed by -3.5712881689999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 13:01:15.997380  2374 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:01:16.000811  2374 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:16.002066  2933 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:16.000197  1688 connection.cc:256] RPC call timeout handler was delayed by -3.384625179000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:01:16.005159  1688 connection.cc:256] RPC call timeout handler was delayed by -3.1393269760000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 13:01:16.028842  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.994966  2956 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:15.990377  2935 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:16.050706  3010 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:16.016196  2933 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
W20260709 13:01:16.053009  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72581.3us
W20260709 13:01:16.056972  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.084935  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.085839  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.087828  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.074671  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 218272us
W20260709 13:01:16.094244  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110183us
W20260709 13:01:16.103443  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.117880  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:16.126986  1570 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.129441  3019 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:16.140904  2835 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:16.141654  2835 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.143600  2973 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:16.107864  2824 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.144661  1571 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.130259  3019 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.162801  2955 raft_consensus.cc:1064] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:16.164047  3019 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:16.171979  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 135538us
I20260709 13:01:16.165876  2955 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 0, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:16.182050  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.178853  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:16.185694  2380 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:16.189005  3019 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:16.189726  3019 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.200330  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:16.200047  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:16.202942  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:16.210527  2946 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.211525  2946 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:16.212692  2946 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.218595  3017 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:16.219920  3019 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:16.227901  2835 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.240521  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:16.241652  3019 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.242379  3019 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:16.243373  3019 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:16.249958  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.249852  2380 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.252727  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:01:16.253518  1724 raft_consensus.cc:606] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:16.253895  1724 raft_consensus.cc:614] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:16.256752  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:16.260607  2380 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.268778  1717 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:16.264063  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 168887us
I20260709 13:01:16.258062  1649 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.279860  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:16.281580  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:16.281502  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:16.283428  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:16.294111  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.265900  3000 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.284464  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:16.301275  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:16.301730  1646 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.302556  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:16.304824  1648 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.306704  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:16.307868  1648 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:16.307974  1650 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.309443  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:16.310230  1650 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:16.311385  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:16.313378  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:16.317245  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:16.322834  1645 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.323026  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:16.294448  2973 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.326632  1649 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.327999  2933 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.328722  2933 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:16.329963  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.330938  2862 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.331133  2933 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.292855  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:16.331568  2862 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.335124  3005 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:16.332331  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 220170us
I20260709 13:01:16.289501  3023 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.336916  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:16.338449  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:16.338696  2914 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:16.332523  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 205049us
I20260709 13:01:16.332630  1644 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:16.333074  2793 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.342121  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:16.342937  1723 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:16.298345  1539 connection.cc:256] RPC call timeout handler was delayed by -1.130148131999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
I20260709 13:01:16.338510  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:16.343612  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.343600  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1040481320000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15580000000000002s for a response
W20260709 13:01:16.344179  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0851481319999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:01:16.344630  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0491481319999707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:01:16.344918  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0365481320000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:01:16.345176  1539 connection.cc:256] RPC call timeout handler was delayed by -1.030248131999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 13:01:16.345443  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0104481320000218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
I20260709 13:01:16.340031  1719 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.312302  1650 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:16.349592  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66269.9us
I20260709 13:01:16.357858  1720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:55262
I20260709 13:01:16.358469  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:16.359254  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:16.358642  1720 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.360718  1720 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.361238  1724 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:16.353724  1717 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:16.354581  2933 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.365322  2933 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.365981  2862 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:16.366869  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:16.367671  2933 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.361774  3002 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:16.370509  3005 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:16.367588  1644 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:16.374823  1722 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.299909  2835 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:16.395221  1721 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:16.402063  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.418890  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.421351  1721 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.392467  2862 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.392114  3005 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:16.427297  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.428022  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 89647us
I20260709 13:01:16.394563  3023 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:16.380674  2982 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:01:16.362114  2914 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:01:16.434134  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:16.437671  1724 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.423717  3017 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.454703  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:01:16.456357  3029 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:01:16.456808  1717 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.457543  2914 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.458015  2914 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:16.467288  2988 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.467626  1720 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.481191  1642 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:16.482298  2946 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.435670  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:16.481865  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 45060.8us
I20260709 13:01:16.482936  2946 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:16.483986  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:16.486984  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:16.487882  2793 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.489405  2793 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:16.489838  2793 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:16.490209  2793 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.493186  2835 raft_consensus.cc:1064] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:16.493085  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:16.451628  3019 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.425850  3002 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:16.497665  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:16.507565  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:16.512213  1723 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
W20260709 13:01:16.506258  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 164248us
I20260709 13:01:16.517007  1647 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.520485  1645 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
W20260709 13:01:16.525563  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.856s (SENT)
I20260709 13:01:16.526678  1720 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:16.527486  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 84059us
W20260709 13:01:16.521360  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 155883us
I20260709 13:01:16.529168  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:16.529109  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:16.529942  1575 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.529906  1570 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:16.534121  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.530651  1570 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:16.459367  2914 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.468237  2988 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.504060  2956 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:16.539265  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.442204  3005 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:16.549845  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.504426  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:16.523921  2884 raft_consensus.cc:2764] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:16.525887  1719 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.553574  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.442625  3023 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.557615  2988 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.558317  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:16.552781  1570 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.539611  2981 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:16.574651  3017 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.575630  1570 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.576507  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:16.596952  1651 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:16.605259  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.746s (SENT) [suppressed 15 similar messages]
W20260709 13:01:16.607661  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 45760.3us
I20260709 13:01:16.613525  1575 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:16.615156  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:16.620227  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 43818.1us
I20260709 13:01:16.621363  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.501338  2835 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.560685  2956 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.636516  2862 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:16.639845  3033 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.587134  3017 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.580148  1718 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:16.580899  3020 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:01:16.624853  2955 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:16.650614  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 115603us
I20260709 13:01:16.661316  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.661640  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:16.583207  2929 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:01:16.673662  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 139708us
I20260709 13:01:16.674921  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:16.641501  3034 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:16.690233  2955 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:16.693320  3030 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:01:16.693629  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:16.695940  3032 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:01:16.697942  3027 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
I20260709 13:01:16.629869  3000 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:01:16.641193  2884 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:01:16.663084  3033 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.665860  2946 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:16.708892  3015 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.592s	user 0.001s	sys 0.005s
W20260709 13:01:16.723081  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 61527us
I20260709 13:01:16.720562  1570 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:16.681339  3025 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:16.750021  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.066s (SENT)
I20260709 13:01:16.754576  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.630705  3002 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.651779  3017 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:16.811991  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 88057.2us
I20260709 13:01:16.827247  3042 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:16.724825  3033 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.808759  3012 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
W20260709 13:01:16.810027  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60063us
I20260709 13:01:16.811599  3001 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.820365  2981 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.829864  2824 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:16.747839  2822 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 13:01:16.893782  3023 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.894644  3044 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.178s
I20260709 13:01:16.815599  3025 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:16.875221  3005 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.883630  2824 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.900573  3000 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:16.934237  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 809) took 1400 ms. Trace:
W20260709 13:01:16.941831  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 280944us
I20260709 13:01:16.927013  2824 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:16.936386  1573 rpcz_store.cc:276] 0709 13:01:15.528109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.534872 (+  6763us) service_pool.cc:224] Handling call
0709 13:01:15.550039 (+ 15167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:15.564901 (+ 14862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:15.564933 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:15.644819 (+ 79886us) log.cc:844] Serialized 26 byte log entry
0709 13:01:15.914983 (+270164us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:15.915048 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.915145 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:16.702313 (+787168us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:16.919708 (+217395us) raft_consensus.cc:1685] finished
0709 13:01:16.919744 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.928074 (+  8330us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13299,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":24408,"spinlock_wait_cycles":15635328,"thread_start_us":231251,"threads_started":1,"wal-append.queue_time_us":347854}
I20260709 13:01:16.957008  3017 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.960961  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:16.962852  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:16.967623  3037 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:01:16.940042  2941 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.967499  3005 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:16.968741  2941 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.969775  3017 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:16.973244  3040 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:01:16.974768  2941 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.966995  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:16.978114  3033 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:16.982932  2793 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.884112  3001 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:16.939096  3000 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.001573  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:16.984952  3033 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:01:17.012588  3033 consensus_queue.cc:1243] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:17.013337  3033 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.013734  3033 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:17.014900  3033 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.016449  3005 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:17.017375  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 388933us
I20260709 13:01:17.025343  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:17.017016  3005 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:17.035805  3005 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.017791  3033 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.042594  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:17.040518  2374 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:17.055972  3051 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:17.056620  3051 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:17.057004  3051 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.058050  2793 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.026521  2955 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:17.064267  2955 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.064791  2955 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:17.065794  2955 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.067050  2955 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.022313  3008 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:01:17.068715  3008 consensus_queue.cc:1243] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:17.069319  3008 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.069622  3008 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:17.069996  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.070495  3008 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.088160  3049 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:01:17.089551  2904 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:17.074463  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.090348  2904 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.075192  3008 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.126283  1650 raft_consensus.cc:606] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:17.127254  2904 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.128618  3047 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.129287  3047 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.141659  3000 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.142377  3000 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.143121  3000 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:17.124372  3025 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.149077  3001 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.154237  3047 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.155016  3047 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.155587  3047 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.162719  3000 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:17.165664  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.185441  1721 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:17.186793  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.210975  1721 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.219101  3001 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.219861  3001 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.221462  1646 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:17.221644  3033 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.222396  3001 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.222265  3033 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.225802  2982 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:17.226349  2982 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:17.227963  2956 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.227715  2982 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.229346  2982 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.233220  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.245370  1718 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:17.234123  3005 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.236326  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:17.238309  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:17.247426  2373 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:17.238735  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:17.250653  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:17.236246  3051 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.238376  2380 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:17.240754  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.243489  2372 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:17.244134  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:17.264317  1573 raft_consensus.cc:606] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:17.233568  3047 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.253028  1721 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:17.271387  1649 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:17.267191  2988 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:01:17.274449  2989 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:17.276654  2929 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:01:17.277179  2824 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.283771  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.311543  1721 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:17.265741  3017 raft_consensus.cc:2764] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:17.304445  2901 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:17.317497  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:17.299466  2981 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.315521  2989 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.329631  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.314539  3033 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.304138  2956 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:01:17.322641  3058 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.339856  3016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.004s	sys 0.019s
I20260709 13:01:17.351208  2953 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:17.351447  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 308947us
I20260709 13:01:17.370131  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:17.338132  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.371694  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.371547  1717 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:17.373723  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.329839  2856 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:17.370702  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:17.400146  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.402352  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.349323  2989 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.421473  2953 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:17.422060  3060 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:17.421988  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51383.8us
I20260709 13:01:17.422924  3060 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:17.423998  3060 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.424815  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:17.425547  1573 raft_consensus.cc:606] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:17.426753  1573 raft_consensus.cc:614] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:17.428206  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:17.429257  1570 raft_consensus.cc:2376] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 2. Re-sending same reply.
I20260709 13:01:17.431937  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.386082  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:17.439363  1570 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.462576  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:17.464071  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:17.465793  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.466594  1575 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:17.329718  2981 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:17.374869  3016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.004s	sys 0.019s
I20260709 13:01:17.382383  2856 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:17.479512  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 93597.5us
I20260709 13:01:17.505951  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.503320  2835 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.481871  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.515363  1571 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.515364  2835 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:17.515949  1571 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:17.516100  1568 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:17.517104  2835 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.481871  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.513411  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:17.513141  2793 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.520925  2793 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.523655  2793 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.524812  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.525705  1572 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:17.438097  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.414589  3047 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.458002  3023 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:17.482111  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.541323  1574 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:17.513882  2856 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:17.554852  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.563831  1572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:17.566636  1572 raft_consensus.cc:606] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:17.416410  3042 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:01:17.567924  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.575984  2380 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.567924  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.569553  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:17.570679  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.571420  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:17.571736  2914 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.379093  3058 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.477079  2929 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:17.586468  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:17.589133  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.516999  1571 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.601898  1571 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.602530  1644 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:17.605393  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.595670  1643 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:17.600800  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.605654  2973 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.591149  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43518.5us
I20260709 13:01:17.561836  3033 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.560575  1718 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:17.638947  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.581785  3005 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.642831  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.643728  2380 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 LEARNER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:17.644506  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:17.592621  3025 raft_consensus.cc:1064] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:17.654268  2824 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.561455  3047 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.649688  3033 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.589277  3051 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.683364  1643 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:17.657728  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.649340  2973 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.669102  2981 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.669241  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.656901  3058 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.589790  2373 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.668171  1645 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:17.677453  1541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.007s	sys 0.038s
I20260709 13:01:17.656831  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:17.696771  1541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.007s	sys 0.038s
W20260709 13:01:17.698493  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.587495  3023 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:17.681804  2862 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:17.706247  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:17.710950  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.700850  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:17.715035  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.712373  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.681226  2375 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
W20260709 13:01:17.695194  3049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.033s	sys 0.028s
W20260709 13:01:17.716409  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.716894  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:17.701328  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.021s	sys 0.023s
I20260709 13:01:17.727211  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.713294  1642 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:17.677366  3034 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.017s	sys 0.065s
W20260709 13:01:17.733844  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.021s	sys 0.023s
I20260709 13:01:17.734596  1650 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:17.700973  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.738034  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.740398  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.742511  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.747871  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.748708  1645 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:17.719849  2371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.007s	sys 0.022s
W20260709 13:01:17.719651  3036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.017s	sys 0.025s
W20260709 13:01:17.759354  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.759204  1570 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
W20260709 13:01:17.763242  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.025s	sys 0.026s
W20260709 13:01:17.731333  3049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.034s	sys 0.029s
I20260709 13:01:17.766047  3059 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:17.766681  2374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.030s	sys 0.029s
W20260709 13:01:17.771338  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.025s	sys 0.026s
W20260709 13:01:17.749092  3034 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.018s	sys 0.066s
W20260709 13:01:17.850281  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 488201us
I20260709 13:01:17.817010  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:17.851604  1647 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.818266  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:17.761502  3002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.014s	sys 0.035s
W20260709 13:01:17.770824  2371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.007s	sys 0.022s
I20260709 13:01:17.758073  2845 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.758303  2941 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:17.717423  2862 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:17.873374  1724 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.005s	sys 0.024s
I20260709 13:01:17.849531  3008 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.853320  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.824612  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:17.850602  3034 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:17.770720  3036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.018s	sys 0.025s
I20260709 13:01:17.859622  2929 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.863548  3049 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:17.850994  2824 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.860229  3002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.015s	sys 0.036s
I20260709 13:01:17.871449  3051 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.903937  1647 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:17.886411  1724 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.006s	sys 0.025s
I20260709 13:01:17.840035  1645 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:17.835476  2374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.030s	sys 0.030s
I20260709 13:01:17.863237  3042 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:17.863559  2904 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.887498  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.863703  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.871989  1649 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:17.870397  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.818212  3059 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:17.899345  2919 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.021s	sys 0.061s
I20260709 13:01:17.887979  1567 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:17.919526  2793 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.925034  3025 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.928252  2793 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.930126  3059 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.930835  3059 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.897501  3008 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.933207  3047 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.933511  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.934139  3047 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:17.934360  1569 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:17.934887  2793 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:17.934819  3047 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.935767  2793 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.874756  2845 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.935649  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.914651  1649 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:17.908512  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:17.875959  2989 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:17.920608  2919 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.021s	sys 0.061s
I20260709 13:01:17.925580  3051 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.911018  2981 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.925138  2862 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:17.889194  2941 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.920203  3049 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:17.916218  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 267) took 1408 ms. Trace:
I20260709 13:01:17.936769  1644 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:17.943411  2793 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.906850  3065 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.931711  3059 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.926313  2904 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.956089  1718 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:17.910527  2824 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.963852  2981 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:17.967813  2941 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.967935  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.969942  2989 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:17.970602  3061 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:01:17.972255  1642 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:17.904549  2929 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.957427  1646 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:17.997789  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:17.976641  3034 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.980592  3008 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:17.982520  2344 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.010s	sys 0.016s
I20260709 13:01:17.913980  3042 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:17.996784  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:17.936653  3033 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:17.998330  1722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.025s	sys 0.004s
W20260709 13:01:18.000698  1717 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.010s	sys 0.030s
I20260709 13:01:17.982478  2973 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:17.968326  1648 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:18.029696  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:18.029938  2941 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.029975  1644 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:18.001425  3060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.014s	sys 0.022s
I20260709 13:01:18.255002  1573 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:18.256609  1722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.026s	sys 0.019s
W20260709 13:01:18.193982  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.258581  1717 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.012s	sys 0.042s
I20260709 13:01:18.032377  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:18.229178  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.268517  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.262809  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.263203  2344 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.016s	sys 0.017s
W20260709 13:01:18.269972  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 839023us
W20260709 13:01:18.159150  1725 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.006s	sys 0.047s
W20260709 13:01:18.270924  1725 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.010s	sys 0.049s
I20260709 13:01:18.272836  1649 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:18.133801  1720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.031s	sys 0.020s
W20260709 13:01:18.274600  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.037s	sys 0.021s
I20260709 13:01:18.086311  1721 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
W20260709 13:01:18.275309  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278645us
W20260709 13:01:18.269999  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.077190  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.092911  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:18.098994  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:18.288198  1575 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:18.282912  1647 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:18.067963  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.023s	sys 0.030s
W20260709 13:01:18.259629  1571 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.005s
W20260709 13:01:18.086791  2856 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.005s
I20260709 13:01:18.301950  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:18.160079  2914 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.030292  3051 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.032594  2845 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:18.286851  2373 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.017s
I20260709 13:01:18.298213  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1428) took 1101 ms. Trace:
I20260709 13:01:18.300181  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:18.148168  3023 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.309068  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:18.284576  3063 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.313s	user 0.000s	sys 0.002s
I20260709 13:01:18.273613  2956 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.316831  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:18.099493  2824 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.294421  3059 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:18.351428  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46494.6us
I20260709 13:01:18.393697  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:18.285764  3033 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.283448  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:18.297055  2835 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.007s	sys 0.011s
I20260709 13:01:18.354684  2981 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:18.272835  3060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.015s	sys 0.027s
I20260709 13:01:18.361099  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:18.348201  1720 rpcz_store.cc:276] 0709 13:01:17.174726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.174854 (+   128us) service_pool.cc:224] Handling call
0709 13:01:17.175154 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.175417 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:17.368906 (+193489us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.275235 (+906329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:18.275324 (+    89us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:18.275391 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.275495 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.275700 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":592100,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":3448,"raft.queue_time_us":8925,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":23195,"thread_start_us":1096775,"threads_started":2}
W20260709 13:01:18.300489  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.023s	sys 0.039s
I20260709 13:01:18.076871  2953 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.284148  3005 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.426188  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:18.316586  3043 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:01:18.286482  1719 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:18.000236  3065 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.394186  1571 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:18.309110  2973 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:18.307988  2919 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:18.332494  2856 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:18.284119  3021 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:18.399356  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.015s	sys 0.030s
I20260709 13:01:18.353823  2373 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:18.280588  2941 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:18.355417  3049 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.442597  1573 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:18.373884  2946 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.349499  2914 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.303755  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:18.351814  3078 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.175s
W20260709 13:01:18.409415  3062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.029s	sys 0.043s
W20260709 13:01:18.463725  2343 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.019s	sys 0.025s
I20260709 13:01:18.701754  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:18.830912  2343 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.050s	sys 0.095s
I20260709 13:01:18.831148  1570 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:18.831323  3078 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:18.831772  3078 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.409484  2904 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:18.631331  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.037s	sys 0.060s
W20260709 13:01:18.832636  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.050s	sys 0.101s
I20260709 13:01:17.984325  2371 rpcz_store.cc:276] 0709 13:01:16.502135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.504196 (+  2061us) service_pool.cc:224] Handling call
0709 13:01:16.563877 (+ 59681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:16.579008 (+ 15131us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:17.055812 (+476804us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.873053 (+817241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.876441 (+  3388us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:17.888621 (+ 12180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.898430 (+  9809us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:17.906840 (+  8410us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":125101,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":17065,"raft.queue_time_us":15139,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":9789,"spinlock_wait_cycles":9512704,"thread_start_us":1190120,"threads_started":2,"wal-append.queue_time_us":908662}
I20260709 13:01:18.831182  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 823) took 1172 ms. Trace:
W20260709 13:01:18.683780  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.833918  1649 rpcz_store.cc:276] 0709 13:01:17.659041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.956310 (+297269us) service_pool.cc:224] Handling call
0709 13:01:18.259559 (+303249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.273844 (+ 14285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.830960 (+557116us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540008ff50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:18.831097 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696071936}
W20260709 13:01:18.835054  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.835006  3059 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.829921  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1560) took 1184 ms. Trace:
W20260709 13:01:18.835623  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.835718  3059 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.761924  3065 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.837113  3065 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:18.823004  3070 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.832s	user 0.000s	sys 0.004s
I20260709 13:01:18.833135  1471 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 112) took 3682 ms. Trace:
I20260709 13:01:18.697773  2901 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.838270  3059 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:18.839123  3059 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:18.839550  2981 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.838047  3071 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.836025  1645 rpcz_store.cc:276] 0709 13:01:17.645798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.910498 (+264700us) service_pool.cc:224] Handling call
0709 13:01:17.911139 (+   641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.328705 (+417566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.328755 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:18.328820 (+    65us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:18.328868 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.829583 (+500715us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54001bbcd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:18.829648 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.829859 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959546752}
I20260709 13:01:18.840152  2981 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:18.839830  3059 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:18.840776  3071 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.842526  2981 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:18.842814  3071 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:18.843034  3059 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:18.843505  2981 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.708990  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:18.397891  3035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.021s	sys 0.027s
I20260709 13:01:18.844487  1720 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:18.844719  3071 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:18.445269  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.000s	sys 0.024s
W20260709 13:01:18.846257  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.000s	sys 0.036s
I20260709 13:01:18.845559  3071 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.846392  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 289) took 1289 ms. Trace:
I20260709 13:01:18.846858  2373 rpcz_store.cc:276] 0709 13:01:17.556834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.559914 (+  3080us) service_pool.cc:224] Handling call
0709 13:01:18.846299 (+1286385us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:18.623443  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.011s	sys 0.015s
I20260709 13:01:18.848280  3071 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:18.848960  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.012s	sys 0.018s
I20260709 13:01:18.376109  3023 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:18.849298  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 937470us
I20260709 13:01:18.849475  3008 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.849913  3054 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.850332  3008 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.850420  3054 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.851825  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:18.693552  3025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.011s	sys 0.010s
I20260709 13:01:18.764032  3042 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.852696  3054 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:18.444953  2837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.003s	sys 0.021s
I20260709 13:01:18.844082  3059 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:18.853223  2837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.009s	sys 0.027s
I20260709 13:01:18.844991  2929 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:18.445148  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.000s	sys 0.025s
W20260709 13:01:18.854275  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.004s	sys 0.033s
I20260709 13:01:18.439728  3033 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:18.853601  3059 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:18.823740  1724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.002s	sys 0.023s
W20260709 13:01:18.856575  1724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.002s	sys 0.023s
I20260709 13:01:18.699023  1651 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:18.835557  2914 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:18.865163  2378 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:18.684664  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 193) took 1618 ms. Trace:
W20260709 13:01:18.811172  1725 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.011s	sys 0.005s
I20260709 13:01:19.347021  2989 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:18.835783  2835 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.010s	sys 0.025s
I20260709 13:01:18.768057  3047 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:18.927529  1717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.007s	sys 0.016s
I20260709 13:01:18.844074  2981 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:19.349076  1717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.027s	sys 0.025s
I20260709 13:01:19.349131  2856 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.453210  3005 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:18.940519  3059 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:19.349870  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 498112us
I20260709 13:01:18.864979  2953 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.350756  1575 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:19.351289  2953 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.351783  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:18.765206  2793 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.352579  3005 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.353575  2953 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:19.347998  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.476s	user 0.025s	sys 0.071s
W20260709 13:01:18.942408  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.571557  3051 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.195583  3049 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.953332  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 65) took 1627 ms. Trace:
I20260709 13:01:18.453902  1646 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:19.364913  1576 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:19.377799  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 828) took 1600 ms (client timeout 1818 ms). Trace:
W20260709 13:01:18.852663  3025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.011s	sys 0.010s
I20260709 13:01:18.965991  2862 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:18.830102  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1432) took 1077 ms. Trace:
I20260709 13:01:18.700531  2956 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:18.830118  3062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.065s	sys 0.112s
I20260709 13:01:19.355623  2941 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.396216  2956 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:19.141662  1722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.006s	sys 0.008s
W20260709 13:01:19.397019  1722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.009s	sys 0.009s
I20260709 13:01:19.397758  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1430) took 2220 ms. Trace:
I20260709 13:01:19.398007  1722 rpcz_store.cc:276] 0709 13:01:17.177128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.177692 (+   564us) service_pool.cc:224] Handling call
0709 13:01:17.178162 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.178422 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:18.264369 (+1085947us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.397455 (+1133086us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.397510 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:19.397537 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.397612 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.397714 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":644595,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":2202117,"threads_started":2}
I20260709 13:01:19.398509  2862 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.399546  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:18.838007  1471 rpcz_store.cc:276] 0709 13:01:15.151030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.151272 (+   242us) service_pool.cc:224] Handling call
0709 13:01:18.833072 (+3681800us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:16.138145 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:16.148391 (+ 10246us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:16.148451 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:01:16.148488 (+    37us) tablet.cc:662] Decoding operations
0709 13:01:16.149284 (+   796us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:16.149403 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:16.149449 (+    46us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:01:16.150500 (+  1051us) tablet.cc:701] Row locks acquired
0709 13:01:16.150571 (+    71us) write_op.cc:260] PREPARE: finished
0709 13:01:16.150845 (+   274us) write_op.cc:270] Start()
0709 13:01:16.151041 (+   196us) write_op.cc:276] Timestamp: P: 1783602076150758 usec, L: 0
0709 13:01:16.151092 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:01:16.177034 (+ 25942us) log.cc:844] Serialized 1302 byte log entry
0709 13:01:17.062098 (+885064us) op_driver.cc:464] REPLICATION: finished
0709 13:01:17.748258 (+686160us) write_op.cc:301] APPLY: starting
0709 13:01:17.774117 (+ 25859us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:17.864166 (+ 90049us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:17.898975 (+ 34809us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:17.927852 (+ 28877us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:17.972696 (+ 44844us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:01:17.972803 (+   107us) write_op.cc:312] APPLY: finished
0709 13:01:17.973421 (+   618us) log.cc:844] Serialized 35 byte log entry
0709 13:01:17.973725 (+   304us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:17.973826 (+   101us) write_op.cc:454] Released schema lock
0709 13:01:17.973890 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:17.974466 (+   576us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:17.974503 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":291712,"child_traces":[["op",{"apply.queue_time_us":635799,"apply.run_cpu_time_us":6566,"apply.run_wall_time_us":252532,"num_ops":3,"prepare.queue_time_us":284018,"prepare.run_cpu_time_us":20858,"prepare.run_wall_time_us":170419,"raft.queue_time_us":362131,"raft.run_cpu_time_us":32670,"raft.run_wall_time_us":255176,"replication_time_us":910795,"thread_start_us":856146,"threads_started":4,"wal-append.queue_time_us":244354,"wal-append.run_cpu_time_us":15576,"wal-append.run_wall_time_us":2577288}]]}
I20260709 13:01:19.400705  2862 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.401476  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 832) took 1712 ms. Trace:
I20260709 13:01:19.355273  2973 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:18.597565  2845 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.401928  1646 rpcz_store.cc:276] 0709 13:01:17.688724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.317131 (+628407us) service_pool.cc:224] Handling call
0709 13:01:18.409316 (+ 92185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.381153 (+971837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.401389 (+ 20236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.402504  2845 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.403450  2973 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:18.823127  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.404595  2845 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.767105  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 60) took 1573 ms. Trace:
I20260709 13:01:19.155833  3086 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:19.405160  1716 rpcz_store.cc:276] 0709 13:01:17.194062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.196217 (+  2155us) service_pool.cc:224] Handling call
0709 13:01:17.196846 (+   629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:17.212455 (+ 15609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.212513 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:17.213396 (+   883us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.453506 (+1240110us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:18.465847 (+ 12341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.600503 (+134656us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:18.766892 (+166389us) raft_consensus.cc:1685] finished
0709 13:01:18.766916 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.767046 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":240,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":1309,"thread_start_us":1223012,"threads_started":1,"wal-append.queue_time_us":1427572}
I20260709 13:01:19.406018  3086 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.407173  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.407469  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 63) took 1129 ms. Trace:
I20260709 13:01:19.407815  1716 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:18.852913  3071 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:18.930470  2919 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.408561  3071 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.410311  2919 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.410581  3071 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:18.765100  3080 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.272s	user 0.000s	sys 0.000s
I20260709 13:01:18.833052  2946 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.414191  2946 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:18.575537  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1435) took 1118 ms. Trace:
I20260709 13:01:19.416133  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52714 (request call id 8) took 1889 ms. Trace:
I20260709 13:01:19.416245  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1554) took 1858 ms. Trace:
I20260709 13:01:19.418437  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 835) took 1529 ms. Trace:
I20260709 13:01:18.831792  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 827) took 1062 ms. Trace:
W20260709 13:01:19.347517  1725 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.022s	sys 0.030s
W20260709 13:01:18.774695  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.241531  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.832492  1718 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:18.698387  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 812) took 1226 ms (client timeout 1567 ms). Trace:
I20260709 13:01:19.418905  1575 rpcz_store.cc:276] 0709 13:01:17.889431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.301567 (+412136us) service_pool.cc:224] Handling call
0709 13:01:19.418283 (+1116716us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002a71d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:19.418385 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408180480}
I20260709 13:01:18.823593  1574 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.365950  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:19.419063  1570 rpcz_store.cc:276] 0709 13:01:17.769042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.779646 (+ 10604us) service_pool.cc:224] Handling call
0709 13:01:18.701507 (+921861us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:18.831714 (+130207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107011456}
I20260709 13:01:19.116837  1723 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:19.367172  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1439) took 2001 ms. Trace:
I20260709 13:01:19.353770  3005 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:18.838833  2901 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:19.423838  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57958.5us
I20260709 13:01:19.418691  1651 rpcz_store.cc:276] 0709 13:01:17.557759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.557980 (+   221us) service_pool.cc:224] Handling call
0709 13:01:18.698558 (+1140578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.372444 (+673886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.416062 (+ 43618us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001bb7d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:19.416191 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2594853120}
I20260709 13:01:18.957131  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:18.768455  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:19.349288  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1431) took 2171 ms. Trace:
W20260709 13:01:19.354357  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.746s (SENT)
I20260709 13:01:19.427505  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 64) took 1148 ms. Trace:
I20260709 13:01:19.364226  1721 rpcz_store.cc:276] 0709 13:01:17.326163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.326315 (+   152us) service_pool.cc:224] Handling call
0709 13:01:18.953035 (+1626720us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b540004ef54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:18.953223 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705366400}
I20260709 13:01:19.423417  1717 rpcz_store.cc:276] 0709 13:01:17.365753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.467251 (+101498us) service_pool.cc:224] Handling call
0709 13:01:17.467638 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.467884 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:18.266589 (+798705us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.366664 (+1100075us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.366715 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:19.366754 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.366980 (+   226us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.367112 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16974,"prepare.queue_time_us":355958,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":18100,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":19762,"thread_start_us":1880024,"threads_started":2,"wal-append.queue_time_us":1078099}
I20260709 13:01:19.422060  1572 rpcz_store.cc:276] 0709 13:01:17.526385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.527027 (+   642us) service_pool.cc:224] Handling call
0709 13:01:19.415767 (+1888740us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400255e20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ceb520 00007fc833d68652 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:19.416059 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656462080}
W20260709 13:01:19.378224  1576 rpcz_store.cc:269] 0709 13:01:17.777374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.821541 (+ 44167us) service_pool.cc:224] Handling call
0709 13:01:18.360836 (+539295us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:19.377733 (+1016897us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5610752}
W20260709 13:01:18.844959  3035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.413s	user 0.026s	sys 0.035s
I20260709 13:01:18.943845  3054 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:18.941745  3079 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:01:19.384876  1573 rpcz_store.cc:276] 0709 13:01:17.752314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.425703 (+673389us) service_pool.cc:224] Handling call
0709 13:01:18.830041 (+404338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.436496  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.436584  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.437078  1573 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.437182  1721 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:19.437592  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1462) took 1463 ms. Trace:
I20260709 13:01:19.437956  1573 rpcz_store.cc:276] 0709 13:01:17.974632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.436246 (+1461614us) service_pool.cc:224] Handling call
0709 13:01:19.437493 (+  1247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.387145  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.388561  3099 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.440605  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.440773  3099 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.441429  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:19.441974  1643 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.443611  3099 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:19.389776  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:19.417059  3059 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.391736  1474 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 2, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:19.444497  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 825) took 1704 ms (client timeout 1906 ms). Trace:
I20260709 13:01:19.444572  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:19.095463  2955 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.445298  2376 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.445582  2955 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.441445  2371 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:19.447888  2955 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.394060  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:18.495704  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.448884  1567 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:18.845141  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1446) took 1168 ms. Trace:
I20260709 13:01:19.416452  3019 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:19.365216  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.453866  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.454463  1721 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.454893  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1455) took 1463 ms. Trace:
I20260709 13:01:19.455184  1721 rpcz_store.cc:276] 0709 13:01:17.992089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.453641 (+1461552us) service_pool.cc:224] Handling call
0709 13:01:19.454842 (+  1201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:18.820685  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 520765us
I20260709 13:01:18.832185  3078 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.139217  3033 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:19.350749  2981 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.347069  2374 rpcz_store.cc:276] 0709 13:01:17.066027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.066206 (+   179us) service_pool.cc:224] Handling call
0709 13:01:17.066610 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:17.067015 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:17.067053 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:17.067592 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:01:17.915612 (+848020us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:17.921213 (+  5601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.948464 (+ 27251us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:18.599983 (+651519us) raft_consensus.cc:1685] finished
0709 13:01:18.624914 (+ 24931us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.654667 (+ 29753us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1253,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"thread_start_us":837533,"threads_started":1,"wal-append.queue_time_us":903380}
I20260709 13:01:18.852626  3008 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.429299  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.462690  1575 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:19.463136  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1457) took 1541 ms. Trace:
I20260709 13:01:19.463444  1575 rpcz_store.cc:276] 0709 13:01:17.922196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.429041 (+1506845us) service_pool.cc:224] Handling call
0709 13:01:19.463083 (+ 34042us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:19.427950  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73672.3us
I20260709 13:01:19.431180  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:19.465440  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.850s (SENT)
I20260709 13:01:19.466902  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.467654  1570 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.468137  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1438) took 1713 ms. Trace:
I20260709 13:01:19.468199  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.357285  3051 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.473701  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:19.427929  1647 rpcz_store.cc:276] 0709 13:01:18.278723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.302536 (+ 23813us) service_pool.cc:224] Handling call
0709 13:01:19.427447 (+1124911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.474287  1721 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.475198  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:18.838009  3065 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.435122  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:19.435175  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:19.482352  2380 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.438818  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:19.439570  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52544.8us
I20260709 13:01:19.482913  1573 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.483672  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:19.435372  2941 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.449576  1720 rpcz_store.cc:276] 0709 13:01:17.676675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.630781 (+954106us) service_pool.cc:224] Handling call
0709 13:01:18.845082 (+214301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.401382  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.450515  1717 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
W20260709 13:01:19.450595  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 56631.8us
I20260709 13:01:19.496932  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.400265  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1443) took 1896 ms. Trace:
I20260709 13:01:19.451781  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 833) took 1579 ms. Trace:
W20260709 13:01:19.444947  1568 rpcz_store.cc:269] 0709 13:01:17.739772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.747342 (+  7570us) service_pool.cc:224] Handling call
0709 13:01:19.389618 (+1642276us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:19.444433 (+ 54815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37248256}
I20260709 13:01:19.421283  3102 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:01:19.445518  1474 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:19.452476  3099 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.452816  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.504395  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:19.466214  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 62) took 1188 ms. Trace:
I20260709 13:01:19.407907  1644 rpcz_store.cc:276] 0709 13:01:18.278379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.370276 (+ 91897us) service_pool.cc:224] Handling call
0709 13:01:18.828765 (+458489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.405661 (+576896us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.405695 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.405746 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:19.405779 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.407222 (+  1443us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54001badd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:19.407274 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.407409 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326324992}
I20260709 13:01:19.422435  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 66) took 1951 ms. Trace:
W20260709 13:01:19.422514  1650 rpcz_store.cc:269] 0709 13:01:17.472524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.472744 (+   220us) service_pool.cc:224] Handling call
0709 13:01:18.698066 (+1225322us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54001bb7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:18.698261 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493689088}
W20260709 13:01:19.460935  1616 connection.cc:256] RPC call timeout handler was delayed by -1.389879690999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:01:19.594573  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.416390  1719 rpcz_store.cc:276] 0709 13:01:17.364412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.368292 (+  3880us) service_pool.cc:224] Handling call
0709 13:01:18.029497 (+661205us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:18.454263 (+424766us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33752960}
I20260709 13:01:19.468528  1570 rpcz_store.cc:276] 0709 13:01:17.754463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.432774 (+1678311us) service_pool.cc:224] Handling call
0709 13:01:19.468088 (+ 35314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.468936  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.595909  1573 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:19.595463  1616 connection.cc:256] RPC call timeout handler was delayed by -1.060479690999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 13:01:19.596062  1575 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:19.596517  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1458) took 1674 ms. Trace:
I20260709 13:01:19.596837  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:19.597154  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.597925  1575 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:19.598196  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.598843  2380 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:19.598922  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.599326  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.599565  1575 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:19.599921  1719 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
W20260709 13:01:19.432483  1574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1430) took 1672 ms (client timeout 1517 ms). Trace:
I20260709 13:01:19.498564  1567 rpcz_store.cc:276] 0709 13:01:17.869855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.433773 (+563918us) service_pool.cc:224] Handling call
0709 13:01:19.449400 (+1015627us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:19.600492  1574 rpcz_store.cc:269] 0709 13:01:17.751453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.751587 (+   134us) service_pool.cc:224] Handling call
0709 13:01:18.156737 (+405150us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:19.423667 (+1266930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18526336}
I20260709 13:01:19.601027  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.601513  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.601591  1567 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.602555  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.427295  1724 rpcz_store.cc:276] 0709 13:01:17.177709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.178193 (+   484us) service_pool.cc:224] Handling call
0709 13:01:17.179185 (+   992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.179481 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:17.909067 (+729586us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.348780 (+1439713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.348869 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:19.348919 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.349037 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.349154 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8279,"prepare.queue_time_us":643874,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":34998,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":62230,"thread_start_us":1661344,"threads_started":2,"wal-append.queue_time_us":1166340}
I20260709 13:01:19.604115  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:19.604197  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.604912  1574 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.605530  1616 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:19.605646  1574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:19.606418  1574 raft_consensus.cc:606] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:19.606809  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:19.607803  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:19.606842  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.608721  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:19.608935  1575 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:19.472972  1643 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
W20260709 13:01:19.434077  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1592) took 1773 ms (client timeout 1666 ms). Trace:
I20260709 13:01:19.479338  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1422) took 2312 ms. Trace:
I20260709 13:01:19.611152  3025 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.612443  2919 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.612962  2919 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.612475  3023 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.611093  2904 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.493584  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1429) took 2317 ms. Trace:
I20260709 13:01:19.613996  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:19.614005  2904 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:19.449957  2946 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.614636  1719 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.614847  2904 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:19.613871  1718 rpcz_store.cc:269] 0709 13:01:17.658727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.154077 (+495350us) service_pool.cc:224] Handling call
0709 13:01:19.431716 (+1277639us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.616030  2919 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.614411  1723 rpcz_store.cc:276] 0709 13:01:17.175969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.176120 (+   151us) service_pool.cc:224] Handling call
0709 13:01:17.176449 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.423457 (+2247008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.493360 (+ 69903us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54001a86d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:19.493526 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4462374656}
I20260709 13:01:19.617388  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.617995  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.618625  1718 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.497624  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:19.497746  1722 rpcz_store.cc:276] 0709 13:01:17.504232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.399307 (+1895075us) service_pool.cc:224] Handling call
0709 13:01:19.400193 (+   886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.453474  3019 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.499471  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.357446  3049 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.500183  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:19.501400  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1590) took 1843 ms. Trace:
I20260709 13:01:19.623721  1717 rpcz_store.cc:276] 0709 13:01:17.658139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.440386 (+1782247us) service_pool.cc:224] Handling call
0709 13:01:19.501244 (+ 60858us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:19.501348 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20825472}
I20260709 13:01:19.502157  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.477020  3092 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.408193  3086 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:19.628407  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 129013us
I20260709 13:01:19.629037  3086 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.628702  2901 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.629516  3086 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.630427  3086 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.631047  3049 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.506491  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.632696  1720 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:19.632807  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1441) took 2266 ms. Trace:
I20260709 13:01:19.633286  1722 rpcz_store.cc:276] 0709 13:01:17.366400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.631810 (+2265410us) service_pool.cc:224] Handling call
0709 13:01:19.632135 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.632452 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.632494 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.632537 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.632566 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.632625 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.632749 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.634020  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1444) took 2129 ms. Trace:
I20260709 13:01:19.634138  3019 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.634405  1720 rpcz_store.cc:276] 0709 13:01:17.504589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.633315 (+2128726us) service_pool.cc:224] Handling call
0709 13:01:19.633589 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.633752 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.633775 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.633809 (+    34us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.633837 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.633883 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.633973 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.634965  1722 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:19.635345  3019 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.473495  3034 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.635694  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1587) took 2083 ms. Trace:
I20260709 13:01:19.635735  1720 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.636178  3034 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.637070  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1589) took 1979 ms. Trace:
I20260709 13:01:19.637387  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1442) took 2270 ms. Trace:
I20260709 13:01:19.637516  1720 rpcz_store.cc:276] 0709 13:01:17.657729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.635314 (+1977585us) service_pool.cc:224] Handling call
0709 13:01:19.635571 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.636814 (+  1243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.637015 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.637877  1717 rpcz_store.cc:276] 0709 13:01:17.366695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.636603 (+2269908us) service_pool.cc:224] Handling call
0709 13:01:19.636901 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.637072 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.637104 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.637148 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:19.637179 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.637237 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.637333 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.638691  3034 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:19.638984  1720 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.639648  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1445) took 1980 ms. Trace:
I20260709 13:01:19.502851  3017 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.477020  3054 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.612664  1725 rpcz_store.cc:276] 0709 13:01:17.166966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.167125 (+   159us) service_pool.cc:224] Handling call
0709 13:01:17.168235 (+  1110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.193294 (+ 25059us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:18.271508 (+1078214us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.419478 (+1147970us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.419532 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:19.419572 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.479097 (+ 59525us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b54003dd1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:19.479140 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.479280 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5718,"prepare.queue_time_us":630323,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":1115,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":14960,"spinlock_wait_cycles":54807936,"thread_start_us":2217567,"threads_started":2,"wal-append.queue_time_us":1102019}
I20260709 13:01:19.628495  3092 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.640618  3054 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.642918  3054 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.643000  3092 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.477077  3071 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.513550  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.495899  2941 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.644137  2941 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.639744  3023 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.644948  3023 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:19.645454  3023 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.509183  1721 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:19.512501  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.629446  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.649114  1716 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:19.649375  3017 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.649869  3017 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.650080  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.651036  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1449) took 1914 ms. Trace:
I20260709 13:01:19.652056  3005 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:19.550544  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:19.652726  3005 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.652556  3033 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:19.569909  3098 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:19.592589  1576 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:19.516261  3047 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:01:19.653644  3005 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.654224  2845 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.498986  3021 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:01:19.652769  1723 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:01:19.655429  3021 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.655926  3021 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:19.657265  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.657111  3021 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.658243  3049 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.658717  3049 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.658787  3021 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.659421  2941 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.659646  3049 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.514381  1569 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:19.477140  3000 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:19.660758  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1452) took 1801 ms (client timeout 2035 ms). Trace:
I20260709 13:01:19.660705  3000 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:19.661281  1569 rpcz_store.cc:269] 0709 13:01:17.859151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.088516 (+229365us) service_pool.cc:224] Handling call
0709 13:01:19.401254 (+1312738us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49e9c 00007fc830c49a5b 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:19.660677 (+259423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3175296}
I20260709 13:01:19.648548  1720 rpcz_store.cc:276] 0709 13:01:17.659354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.638535 (+1979181us) service_pool.cc:224] Handling call
0709 13:01:19.638810 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.639410 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.639598 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.662855  3000 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.663050  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1437) took 1909 ms. Trace:
I20260709 13:01:19.513670  2884 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.663450  1569 rpcz_store.cc:276] 0709 13:01:17.754071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.662143 (+1908072us) service_pool.cc:224] Handling call
0709 13:01:19.662502 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.662688 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.662728 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.662770 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.662808 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.662869 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.662978 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.602505  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.611263  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.665023  1724 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:19.654805  2845 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.665346  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.631047  2901 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:19.666487  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1451) took 1815 ms. Trace:
I20260709 13:01:19.611693  2862 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.633459  2376 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:19.507014  1648 rpcz_store.cc:276] 0709 13:01:18.277988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.348386 (+ 70398us) service_pool.cc:224] Handling call
0709 13:01:18.354933 (+  6547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.396665 (+ 41732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.409183 (+ 12518us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.758019 (+348836us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.073591 (+315572us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.073659 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.073793 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.465964 (+392171us) raft_consensus.cc:1685] finished
0709 13:01:19.465998 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.466145 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25979,"raft.queue_time_us":19270,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":39068,"thread_start_us":314969,"threads_started":1,"wal-append.queue_time_us":706378}
I20260709 13:01:19.593775  1571 rpcz_store.cc:276] 0709 13:01:17.470057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.474354 (+  4297us) service_pool.cc:224] Handling call
0709 13:01:17.515189 (+ 40835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.364495 (+1849306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.420877 (+ 56382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.667865  1724 rpcz_store.cc:276] 0709 13:01:17.850695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.665668 (+1814973us) service_pool.cc:224] Handling call
0709 13:01:19.665942 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.666195 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.666233 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.666269 (+    36us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:19.666295 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.666348 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.666443 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":936}
I20260709 13:01:19.611867  2953 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:19.669092  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1439) took 1914 ms. Trace:
I20260709 13:01:19.669512  1571 rpcz_store.cc:276] 0709 13:01:17.755041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.668159 (+1913118us) service_pool.cc:224] Handling call
0709 13:01:19.668476 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.668732 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.668760 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.668806 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.668844 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.668917 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.669034 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.670641  1567 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 1.
I20260709 13:01:19.671314  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1440) took 1915 ms. Trace:
I20260709 13:01:19.606783  1574 raft_consensus.cc:614] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:19.670037  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1453) took 1726 ms. Trace:
I20260709 13:01:19.672552  3099 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.673372  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1442) took 1917 ms. Trace:
I20260709 13:01:19.673503  3065 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.673786  1569 rpcz_store.cc:276] 0709 13:01:17.756310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.672505 (+1916195us) service_pool.cc:224] Handling call
0709 13:01:19.672846 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.673012 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.673042 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.673114 (+    72us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.673152 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.673214 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.673318 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.674600   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:01:19.665073  1572 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:19.675529  3051 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.676153  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1435) took 1922 ms. Trace:
I20260709 13:01:19.677012  1724 rpcz_store.cc:276] 0709 13:01:17.943928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.668818 (+1724890us) service_pool.cc:224] Handling call
0709 13:01:19.669081 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.669756 (+   675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.669781 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.669818 (+    37us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.669846 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.669896 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.669982 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.678215  1572 rpcz_store.cc:276] 0709 13:01:17.753403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.628123 (+1874720us) service_pool.cc:224] Handling call
0709 13:01:19.664821 (+ 36698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.675948 (+ 11127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.676107 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6912}
I20260709 13:01:19.636125  1722 rpcz_store.cc:276] 0709 13:01:17.552887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.634347 (+2081460us) service_pool.cc:224] Handling call
0709 13:01:19.634630 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.635410 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.635639 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.679555  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:19.451367  2835 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.612819  3025 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.510689  1474 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 2, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.643872  3071 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.691118  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1448) took 1991 ms. Trace:
W20260709 13:01:19.514722  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47374.5us
I20260709 13:01:19.526162  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:19.650771  3017 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.513054  3062 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.666050  2845 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.614955  2946 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:19.667073  2862 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.611395  3042 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.651520  1716 rpcz_store.cc:276] 0709 13:01:17.736642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.649892 (+1913250us) service_pool.cc:224] Handling call
0709 13:01:19.650315 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.650558 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.650598 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.650667 (+    69us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.650728 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.650822 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.650964 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.675565  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1444) took 1903 ms. Trace:
I20260709 13:01:19.673036  3099 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.673640  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1443) took 1916 ms. Trace:
I20260709 13:01:19.719995  1574 rpcz_store.cc:276] 0709 13:01:17.757097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.672946 (+1915849us) service_pool.cc:224] Handling call
0709 13:01:19.673194 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.673355 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.673380 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.673419 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.673447 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.673499 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.673589 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.721246  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.721792  1574 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:19.723076  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1445) took 1950 ms. Trace:
I20260709 13:01:19.723413  1574 rpcz_store.cc:276] 0709 13:01:17.772226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.722394 (+1950168us) service_pool.cc:224] Handling call
0709 13:01:19.722647 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.722796 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.722820 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.722861 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.722889 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.722938 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.723030 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.676805   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:19.677141  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.724887  1570 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:19.677596  1567 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.596860  1573 rpcz_store.cc:276] 0709 13:01:17.922546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.438563 (+1516017us) service_pool.cc:224] Handling call
0709 13:01:19.596461 (+157898us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.713972  3078 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.474251  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.639684  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1447) took 1959 ms. Trace:
I20260709 13:01:19.729687  1717 rpcz_store.cc:276] 0709 13:01:17.680203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.638961 (+1958758us) service_pool.cc:224] Handling call
0709 13:01:19.639225 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.639403 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.639432 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.639475 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.639504 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.639553 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.639640 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.671706  1571 rpcz_store.cc:276] 0709 13:01:17.755661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.670552 (+1914891us) service_pool.cc:224] Handling call
0709 13:01:19.670823 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.670979 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.671017 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.671063 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.671097 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.671152 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.671251 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.681959  1650 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:19.476372  2956 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.682677  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.715042  1719 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:19.688246  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1456) took 1688 ms. Trace:
I20260709 13:01:19.690497  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:19.690970  3025 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.715492  3042 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:19.691511  1725 rpcz_store.cc:276] 0709 13:01:17.700084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.640977 (+1940893us) service_pool.cc:224] Handling call
0709 13:01:19.641280 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.664267 (+ 22987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.664295 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.664351 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.664379 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.690891 (+ 26512us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54001702d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:19.690934 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.691063 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52430720}
I20260709 13:01:19.619815  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:19.699407  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1454) took 1746 ms. Trace:
I20260709 13:01:19.740900  1724 rpcz_store.cc:276] 0709 13:01:17.953124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.678673 (+1725549us) service_pool.cc:224] Handling call
0709 13:01:19.698774 (+ 20101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.699004 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.699042 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.699099 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.699143 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.699214 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.699353 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.710649  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:19.692653  3060 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:19.712657  2845 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.663585  2884 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:19.726452  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 117802us
I20260709 13:01:19.727258  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1449) took 1952 ms. Trace:
I20260709 13:01:19.727655  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:19.681231  1575 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:19.765803  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1433) took 2013 ms. Trace:
I20260709 13:01:19.766412  1575 rpcz_store.cc:276] 0709 13:01:17.752718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.680554 (+1927836us) service_pool.cc:224] Handling call
0709 13:01:19.680895 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.765268 (+ 84373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.765716 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:19.731428  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 257281us
I20260709 13:01:19.768959  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.734287  1722 rpcz_store.cc:276] 0709 13:01:18.000435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.687193 (+1686758us) service_pool.cc:224] Handling call
0709 13:01:19.687592 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.687819 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.687859 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.687918 (+    59us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.687962 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.688032 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.688175 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.771828  1721 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 13:01:19.772475  1721 pending_rounds.cc:85] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 3
I20260709 13:01:19.772818  1721 pending_rounds.cc:107] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 13:01:19.773173  1721 raft_consensus.cc:2917] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
W20260709 13:01:19.735631  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53039.5us
I20260709 13:01:19.718307  3062 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:19.625533  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 125442us
I20260709 13:01:19.727802  3071 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:19.776192  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:19.764107  1573 rpcz_store.cc:276] 0709 13:01:17.774616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.726550 (+1951934us) service_pool.cc:224] Handling call
0709 13:01:19.726826 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.726976 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.727000 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.727040 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.727068 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.727114 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.727205 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.788900  1721 raft_consensus.cc:2889] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:19.735276  2941 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.739195  3092 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.760917  1571 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.715075  2946 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.732563  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 830) took 2071 ms. Trace:
I20260709 13:01:19.776119  3101 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:19.780103  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:19.787017  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.746197  3005 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.745704  3060 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.746649  3042 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.827859  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:55254 (request call id 71) took 5664 ms. Trace:
I20260709 13:01:19.829150  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:19.832023  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:19.757349  3111 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:01:19.826970  3000 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:19.758674  2862 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:19.763777  2884 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.823732  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 830) took 1997 ms. Trace:
I20260709 13:01:19.764237  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1441) took 1994 ms. Trace:
I20260709 13:01:19.868615  1575 rpcz_store.cc:276] 0709 13:01:17.826718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.767467 (+1940749us) service_pool.cc:224] Handling call
0709 13:01:19.768005 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.818425 (+ 50420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.823020 (+  4595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.823119 (+    99us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.823196 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.823303 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.823591 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.787858  3035 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.847390  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 67483.1us
I20260709 13:01:19.804827  2953 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:19.839344  1573 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:19.876065  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 831) took 2014 ms. Trace:
I20260709 13:01:19.876477  1573 rpcz_store.cc:276] 0709 13:01:17.862094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.792674 (+1930580us) service_pool.cc:224] Handling call
0709 13:01:19.826932 (+ 34258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.875780 (+ 48848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.876017 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.838357  3092 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.848289  2373 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 2 LEARNER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:19.847536  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:19.886898  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.718117  1569 rpcz_store.cc:276] 0709 13:01:17.771882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.674835 (+1902953us) service_pool.cc:224] Handling call
0709 13:01:19.675106 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.675266 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.675291 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.675331 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.675360 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.675417 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.675516 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.851704  1721 rpcz_store.cc:276] 0709 13:01:14.157899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.158282 (+   383us) service_pool.cc:224] Handling call
0709 13:01:14.340391 (+182109us) log.cc:844] Serialized 501 byte log entry
0709 13:01:19.789918 (+5449527us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13454,"raft.queue_time_us":12179,"raft.run_cpu_time_us":1546,"raft.run_wall_time_us":1546,"spinlock_wait_cycles":273536,"thread_start_us":56250,"threads_started":1,"wal-append.queue_time_us":168280,"wal-append.run_cpu_time_us":3478,"wal-append.run_wall_time_us":1017217}
I20260709 13:01:19.874533  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:19.880050  1575 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:19.890975  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1453) took 2028 ms. Trace:
I20260709 13:01:19.891363  1575 rpcz_store.cc:276] 0709 13:01:17.862892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.879468 (+2016576us) service_pool.cc:224] Handling call
0709 13:01:19.879835 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.890692 (+ 10857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.890917 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.891868  1573 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:19.865832  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78925.4us
I20260709 13:01:19.892771  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 832) took 2027 ms. Trace:
I20260709 13:01:19.854611  3060 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:19.887641  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.857072  3109 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:19.852245  3005 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.893142  1573 rpcz_store.cc:276] 0709 13:01:17.865961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.880563 (+2014602us) service_pool.cc:224] Handling call
0709 13:01:19.891539 (+ 10976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.892516 (+   977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.892714 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.873106  2835 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.893080  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:19.845468  3051 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:01:19.863137  3000 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.860771  3101 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:19.907640  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 836) took 2009 ms. Trace:
I20260709 13:01:19.881565  3035 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.904414  3005 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.908593  2793 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:19.909152  2793 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.909739  2793 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.910064  3035 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.911984  3035 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.875264  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 829) took 2055 ms. Trace:
I20260709 13:01:19.874755  2953 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:19.880695  2946 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.901177  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.908172  1573 rpcz_store.cc:276] 0709 13:01:17.898001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.906483 (+2008482us) service_pool.cc:224] Handling call
0709 13:01:19.906952 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.907204 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.907236 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.907287 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.907324 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.907392 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.907566 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.926877  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.927845  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.907256  3101 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:19.839880  3086 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:01:19.874379  1567 rpcz_store.cc:276] 0709 13:01:17.756017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.671342 (+1915325us) service_pool.cc:224] Handling call
0709 13:01:19.677426 (+  6084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.731338 (+ 53912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.750225 (+ 18887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.884609  3092 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.844642  1642 rpcz_store.cc:276] 0709 13:01:17.661246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.078290 (+417044us) service_pool.cc:224] Handling call
0709 13:01:18.843387 (+765097us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:18.843563 (+   176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.415359 (+571796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.415393 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.415445 (+    52us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.415478 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.717845 (+302367us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54000f55d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:19.725810 (+  7965us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.731425 (+  5615us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339112960}
I20260709 13:01:19.894654  2904 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:19.892422  2955 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:19.923350  3035 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.928174  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1456) took 2008 ms. Trace:
I20260709 13:01:19.957644  1573 rpcz_store.cc:276] 0709 13:01:17.919502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.927234 (+2007732us) service_pool.cc:224] Handling call
0709 13:01:19.927591 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.927784 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.927809 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.927859 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.927923 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.927986 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.928111 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.932659  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.915462  3000 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:19.940382  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.898854  3109 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:19.914557  2956 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:19.905313  2835 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:19.968612  2835 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.964897  3054 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.969419  3054 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.969817  3054 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.970428  1568 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.971313  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1459) took 2048 ms. Trace:
I20260709 13:01:19.972072  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1436) took 2218 ms. Trace:
I20260709 13:01:19.893167  2901 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.972818  2901 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.973308  2901 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.925712  2953 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:19.923359  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1460) took 1067 ms. Trace:
I20260709 13:01:19.908283  3033 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:01:19.961156  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.938385  2862 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.942612  3047 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:19.937592  3071 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:19.964811  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 13:01:19.971711  1576 rpcz_store.cc:276] 0709 13:01:17.922799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.970538 (+2047739us) service_pool.cc:224] Handling call
0709 13:01:19.970876 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.971030 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.971053 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.971095 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.971123 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.971173 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.971262 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.002753  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1452) took 2136 ms. Trace:
I20260709 13:01:20.019378  1721 rpcz_store.cc:276] 0709 13:01:17.866383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.667907 (+1801524us) service_pool.cc:224] Handling call
0709 13:01:19.771194 (+103287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.918161 (+146967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.002482 (+ 84321us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400232b50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:20.002679 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236795264}
I20260709 13:01:20.009445  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.021086  1725 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:19.922894  1571 rpcz_store.cc:276] 0709 13:01:17.820218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.747980 (+1927762us) service_pool.cc:224] Handling call
0709 13:01:19.760746 (+ 12766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.844389 (+ 83643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.867072 (+ 22683us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.014958  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:20.022758  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:20.015944  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:19.960326  3035 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.929639  3062 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:20.036579  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 146936us
W20260709 13:01:20.032696  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 526503us
I20260709 13:01:20.030474  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:20.034533  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:20.038206  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:20.037998  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.030979  1458 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 69128.3us
I20260709 13:01:19.975301  3109 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:20.041255  3086 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:20.041961  1722 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.044432  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:20.049778  3086 raft_consensus.cc:2764] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:20.051247  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:20.054222  1576 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:20.058475  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1461) took 2135 ms. Trace:
I20260709 13:01:20.059087  1576 rpcz_store.cc:276] 0709 13:01:17.923282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.019171 (+2095889us) service_pool.cc:224] Handling call
0709 13:01:20.053808 (+ 34637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.058025 (+  4217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.058387 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.019498  3047 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.016791  2862 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:20.020665  3034 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:20.018507  3071 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:20.034204  3102 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:19.972563  1568 rpcz_store.cc:276] 0709 13:01:17.753736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.644315 (+1890579us) service_pool.cc:224] Handling call
0709 13:01:19.970231 (+325916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.970916 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.971886 (+   970us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54003638d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:19.972019 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748284032}
I20260709 13:01:20.064024  1722 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:20.052450  2856 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:20.051182  3059 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:20.104110  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 229729us
I20260709 13:01:20.072288  3034 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:20.113988  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 284943us
I20260709 13:01:20.069726  2862 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:20.146452  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 116062us
W20260709 13:01:20.118650  3112 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.331s	user 0.002s	sys 0.000s
I20260709 13:01:20.106772  3071 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:20.105510  3114 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:01:20.066654  3047 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:20.170531  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 145677us
I20260709 13:01:20.140990  1576 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:20.158638  3106 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.987746  1716 rpcz_store.cc:276] 0709 13:01:18.852664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.748269 (+895605us) service_pool.cc:224] Handling call
0709 13:01:19.853906 (+105637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.879236 (+ 25330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.882463 (+  3227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.888315 (+  5852us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:19.890542 (+  2227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.903475 (+ 12933us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.919591 (+ 16116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.200456  3059 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:20.236122  3115 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:01:20.300678  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1450) took 2525 ms. Trace:
I20260709 13:01:20.301087  3059 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.198791  2904 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:20.239396  3118 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.314256  3118 consensus_queue.cc:1243] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:20.314776  3118 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:20.315083  3118 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:20.281816  3131 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:20.207427  1571 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:20.241025  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:20.245115  1568 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:01:20.212353  2919 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.596s	user 0.004s	sys 0.007s
I20260709 13:01:20.286258  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:20.211459  3034 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.233201  2856 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:20.301316  1720 rpcz_store.cc:276] 0709 13:01:17.775716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.663099 (+1887383us) service_pool.cc:224] Handling call
0709 13:01:19.663400 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.663717 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.663744 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.664125 (+   381us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.922733 (+258608us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.922788 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.922877 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.300260 (+377383us) raft_consensus.cc:1685] finished
0709 13:01:20.300288 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.300600 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":305,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":244600,"threads_started":1,"wal-append.queue_time_us":310988}
I20260709 13:01:20.315994  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1451) took 2489 ms. Trace:
W20260709 13:01:20.318094  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77329.6us
I20260709 13:01:20.319414  3124 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:20.319583  3118 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:20.343140  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57086.3us
I20260709 13:01:20.341560  1572 rpcz_store.cc:276] 0709 13:01:17.826195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.747704 (+1921509us) service_pool.cc:224] Handling call
0709 13:01:19.748089 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.756405 (+  8316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.756450 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.757032 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.869177 (+112145us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:19.967001 (+ 97824us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b4800002dd8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:01:20.060081 (+ 93080us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.065490 (+  5409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.065579 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.302766 (+237187us) raft_consensus.cc:1685] finished
0709 13:01:20.315738 (+ 12972us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.315918 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":201653,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":9732,"raft.run_cpu_time_us":1533,"raft.run_wall_time_us":206248,"spinlock_wait_cycles":40665344,"thread_start_us":295528,"threads_started":2,"wal-append.queue_time_us":178385}
W20260709 13:01:20.351178  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 313095us
I20260709 13:01:20.351531  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:20.315485  3008 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:20.353154  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 148176us
I20260709 13:01:20.347448  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:20.356274  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.302454  3106 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:20.315672  3131 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:20.383948  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:20.381673  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1450) took 2567 ms. Trace:
I20260709 13:01:20.384631  1570 rpcz_store.cc:276] 0709 13:01:17.814663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.734174 (+1919511us) service_pool.cc:224] Handling call
0709 13:01:19.761329 (+ 27155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.761682 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.761717 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.762157 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.944689 (+182532us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:20.070527 (+125838us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.380885 (+310358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.380984 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.381436 (+   452us) raft_consensus.cc:1685] finished
0709 13:01:20.381455 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.381600 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":216677,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":93934,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":37801,"thread_start_us":604055,"threads_started":3,"wal-append.queue_time_us":267238}
I20260709 13:01:20.346650  3124 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:20.386103  1574 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.011s	sys 0.014s
W20260709 13:01:20.388242  1574 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.012s	sys 0.014s
I20260709 13:01:20.350869  3089 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:20.360092  2956 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:20.361542  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1464) took 2368 ms. Trace:
I20260709 13:01:20.355860  3008 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:20.341777  2919 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:20.406931  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 834) took 2533 ms. Trace:
I20260709 13:01:20.411756  1575 rpcz_store.cc:276] 0709 13:01:17.873926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.893517 (+2019591us) service_pool.cc:224] Handling call
0709 13:01:19.893832 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.894076 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:20.031677 (+137601us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.405241 (+373564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.405310 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:20.406079 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.406160 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.406868 (+   708us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":224741,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2376,"raft.queue_time_us":280,"thread_start_us":491326,"threads_started":2}
I20260709 13:01:20.387655  3118 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:20.413436  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 206745us
I20260709 13:01:20.387735  3131 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.396127  3089 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.391952  3124 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.401481  2956 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:20.446164  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 388635us
I20260709 13:01:20.450090  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:20.454008  1720 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:20.461609  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:20.462911  1720 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:20.433462  3089 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.472124  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.431241  3129 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:20.448576  2956 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:20.480615  2793 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.570s	user 0.016s	sys 0.008s
I20260709 13:01:20.404251  1571 rpcz_store.cc:276] 0709 13:01:17.986080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.061669 (+2075589us) service_pool.cc:224] Handling call
0709 13:01:20.121546 (+ 59877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.318719 (+197173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.354779 (+ 36060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.505607  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.508565  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 843) took 1011 ms. Trace:
I20260709 13:01:20.518134  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.522816  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 127187us
I20260709 13:01:20.434303  2856 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:20.587965  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 235273us
I20260709 13:01:20.524853  3129 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:20.522675  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:20.555372  3005 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.646s	user 0.017s	sys 0.005s
W20260709 13:01:20.609615  2835 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.013s
I20260709 13:01:20.558401  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.591472  2941 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:20.624056  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 198671us
W20260709 13:01:20.626744  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.505318  2793 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:20.638078  3138 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.143s	user 0.000s	sys 0.002s
I20260709 13:01:20.552584  3130 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:20.564211  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:20.678433  2793 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:20.633322  3021 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.974s	user 0.008s	sys 0.019s
I20260709 13:01:20.680220  3021 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.509362  3008 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:20.638670  3125 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:01:20.624532  3023 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.978s	user 0.018s	sys 0.008s
I20260709 13:01:20.676182  3145 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
W20260709 13:01:20.633060  3054 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.654s	user 0.014s	sys 0.009s
W20260709 13:01:20.640621  2845 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.877s	user 0.020s	sys 0.012s
I20260709 13:01:20.474259  3106 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:20.743691  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.745307  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.746667  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 363664us
I20260709 13:01:20.647732  2371 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:20.747965  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:20.731043  3142 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:01:20.733182  1716 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:20.765252  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1468) took 1297 ms. Trace:
I20260709 13:01:20.765812  1576 rpcz_store.cc:276] 0709 13:01:19.467576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.314001 (+846425us) service_pool.cc:224] Handling call
0709 13:01:20.314382 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.314737 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.314766 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.315258 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.389413 (+ 74155us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:20.751192 (+361779us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.751484 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.751583 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.765035 (+ 13452us) raft_consensus.cc:1685] finished
0709 13:01:20.765054 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.765187 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37216,"prepare.queue_time_us":78991,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":1655,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":7185,"thread_start_us":358578,"threads_started":2,"wal-append.queue_time_us":159205}
I20260709 13:01:20.686244  2793 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:01:20.644311  2835 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:20.773533  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:20.744901  3130 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:20.776574  2380 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:01:20.723786  3008 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:20.750427  2919 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:20.774996  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 167579us
I20260709 13:01:20.648298  3005 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:20.777330  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4657766150000384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:01:20.785029  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4558766150000224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:01:20.668140  2941 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:20.775930  2973 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.000s	sys 0.018s
W20260709 13:01:20.785799  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3973766150000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 13:01:20.786186  1539 connection.cc:256] RPC call timeout handler was delayed by -1.388376615000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 13:01:20.786480  1539 connection.cc:256] RPC call timeout handler was delayed by -1.245276615000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 13:01:20.774549  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 124983us
W20260709 13:01:20.787837  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0085766150000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:01:20.789798  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.790477  1575 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:20.741094  3143 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:01:20.752914  3023 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:20.773267  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:20.787369  2973 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.000s	sys 0.018s
I20260709 13:01:20.799371  1572 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:20.778416  2955 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.750968  3054 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:20.802416  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.759662  2981 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:01:20.805328  1575 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:20.803781  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:20.797760  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.823642  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:20.824203  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:20.877192  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.827229  3141 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:20.827431  3147 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:20.891386  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 100879us
I20260709 13:01:20.892309  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:20.892738  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.894253  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.829348  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.509301  1570 rpcz_store.cc:276] 0709 13:01:19.496862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.386736 (+889874us) service_pool.cc:224] Handling call
0709 13:01:20.389025 (+  2289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.389267 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:20.434090 (+ 44823us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.508072 (+ 73982us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.508137 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:20.508195 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.508284 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.508475 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33393,"prepare.queue_time_us":87316,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":121,"thread_start_us":73437,"threads_started":1}
I20260709 13:01:20.835320  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.844164  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.887s (SENT)
I20260709 13:01:20.901530  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:20.846300  2904 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:20.850412  3065 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.860183  3155 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:20.865773  3108 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.176s	user 0.007s	sys 0.000s
I20260709 13:01:20.867755  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.868470  3021 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:20.868930  3128 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.457s	user 0.003s	sys 0.000s
W20260709 13:01:20.870028  3000 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.905s	user 0.011s	sys 0.004s
I20260709 13:01:20.870563  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:20.871696  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1470) took 1403 ms. Trace:
I20260709 13:01:20.878695  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1471) took 1410 ms. Trace:
I20260709 13:01:20.952937  1572 rpcz_store.cc:276] 0709 13:01:19.468418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.368324 (+899906us) service_pool.cc:224] Handling call
0709 13:01:20.369204 (+   880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.801336 (+432132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.878355 (+ 77019us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b5400026cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:20.878609 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988909440}
I20260709 13:01:20.881954  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.882077  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.873443  3140 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:01:20.884690  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.883891  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60332.1us
I20260709 13:01:20.887877  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.889114  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.917142  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:20.956960  1723 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:20.919653  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75573.3us
I20260709 13:01:20.957561  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:20.772812  3129 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:20.922125  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.874491  3136 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:01:20.809111  3134 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.921286  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.891849  2946 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.955s	user 0.013s	sys 0.017s
I20260709 13:01:20.940330  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:20.940440  3042 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.066s	user 0.013s	sys 0.017s
I20260709 13:01:20.995270  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:20.896317  3078 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:20.954217  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:20.968289  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97838us
I20260709 13:01:20.810958  2955 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:20.933871  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:20.929430  3017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.008s	sys 0.004s
I20260709 13:01:20.915269  2904 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:20.989542  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.991441  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:20.991693  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.049579  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:21.002782  1572 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:20.952469  3119 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:20.952688  1568 rpcz_store.cc:276] 0709 13:01:19.468089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.320430 (+852341us) service_pool.cc:224] Handling call
0709 13:01:20.383851 (+ 63421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.399154 (+ 15303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.402496 (+  3342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.406587 (+  4091us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:20.412555 (+  5968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.425361 (+ 12806us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.871497 (+446136us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b5400041ad0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:20.871633 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140241280}
I20260709 13:01:21.053442  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:21.043458  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.952307  3000 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:20.969725  3141 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:21.045794  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.930423  2919 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:01:20.988531  2973 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:21.054628  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:20.992908  2845 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:21.051820  3023 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:20.934206  3065 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.086875  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:21.087581  3151 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:21.064414  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:21.074038  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:21.042984  3017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.009s	sys 0.005s
I20260709 13:01:21.076690  2793 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:21.093760  3065 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.094403  1573 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:21.093880  2793 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:21.095922  1573 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:21.055964  2904 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.102645  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 195) took 1668 ms. Trace:
W20260709 13:01:21.101517  3146 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:01:21.004897  3051 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:01:21.151437  1720 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:21.022090  3042 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:20.967406  3130 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.108603  2973 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:21.094480  2941 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.144537  2835 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:21.105502  2378 rpcz_store.cc:276] 0709 13:01:19.434140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.435601 (+  1461us) service_pool.cc:224] Handling call
0709 13:01:19.442716 (+  7115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.457123 (+ 14407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.457151 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.457586 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.690056 (+232470us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:19.690094 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.690157 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.864884 (+1174727us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:21.092213 (+227329us) raft_consensus.cc:1685] finished
0709 13:01:21.092242 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.102559 (+ 10317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8274,"raft.queue_time_us":18705,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":90,"spinlock_wait_cycles":8575744,"thread_start_us":188521,"threads_started":1,"wal-append.queue_time_us":209947}
I20260709 13:01:21.153033  1718 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:01:21.153681  3136 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:01:21.179235  3157 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:21.108225  3062 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:01:21.175282  3017 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:21.184281   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:21.109928  2845 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:21.204846  2953 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.183s	user 0.008s	sys 0.017s
W20260709 13:01:21.206463  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.169226  3005 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.240446  3017 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.242969  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:21.193061  2901 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.219s	user 0.014s	sys 0.018s
I20260709 13:01:21.200320  3149 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:01:21.172580  3065 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:21.197463  3118 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.003s
I20260709 13:01:21.175876  2973 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:21.177441  2919 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:21.190336  3134 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:21.180974  3054 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:21.226294  2953 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:21.227035  3099 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:21.227155  3047 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.925s	user 0.009s	sys 0.004s
I20260709 13:01:21.229322  2793 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.231256  1721 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:21.232227  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.234359  3151 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:21.235188  2862 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.030s	user 0.018s	sys 0.017s
I20260709 13:01:21.236152  2845 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.237550  2955 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:21.234380  1724 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:21.277161  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:21.231305  3089 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.730s	user 0.007s	sys 0.003s
W20260709 13:01:21.311106  2956 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.803s	user 0.011s	sys 0.000s
W20260709 13:01:21.314541  3071 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.025s	user 0.011s	sys 0.005s
I20260709 13:01:21.315569  2793 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:21.318989  3035 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.262s	user 0.010s	sys 0.011s
W20260709 13:01:21.323446  3060 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.410s	user 0.006s	sys 0.021s
W20260709 13:01:21.326748  3025 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.564s	user 0.012s	sys 0.015s
I20260709 13:01:21.328787  1472 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.397003  3025 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.397859  1472 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.399574  3025 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.330508  1474 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 215) took 4774 ms. Trace:
I20260709 13:01:21.399945  1472 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.299854  2919 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.400506  1474 rpcz_store.cc:276] 0709 13:01:16.556403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.556576 (+   173us) service_pool.cc:224] Handling call
0709 13:01:21.328408 (+4771832us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:19.978528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:20.008737 (+ 30209us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:20.008790 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:01:20.008817 (+    27us) tablet.cc:662] Decoding operations
0709 13:01:20.009520 (+   703us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:20.009569 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:20.009596 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:01:20.009884 (+   288us) tablet.cc:701] Row locks acquired
0709 13:01:20.009901 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:01:20.010039 (+   138us) write_op.cc:270] Start()
0709 13:01:20.010144 (+   105us) write_op.cc:276] Timestamp: P: 1783602080009994 usec, L: 0
0709 13:01:20.010185 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:01:20.010930 (+   745us) log.cc:844] Serialized 1307 byte log entry
0709 13:01:20.706104 (+695174us) op_driver.cc:464] REPLICATION: finished
0709 13:01:21.161094 (+454990us) write_op.cc:301] APPLY: starting
0709 13:01:21.171605 (+ 10511us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:21.202554 (+ 30949us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:21.224551 (+ 21997us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:21.305878 (+ 81327us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:21.306050 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:01:21.306095 (+    45us) write_op.cc:312] APPLY: finished
0709 13:01:21.306314 (+   219us) log.cc:844] Serialized 35 byte log entry
0709 13:01:21.306494 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:21.306585 (+    91us) write_op.cc:454] Released schema lock
0709 13:01:21.306623 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:21.306958 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:21.306986 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":441363,"apply.run_cpu_time_us":2631,"apply.run_wall_time_us":159173,"num_ops":3,"prepare.queue_time_us":247118,"prepare.run_cpu_time_us":18784,"prepare.run_wall_time_us":116699,"raft.queue_time_us":314204,"raft.run_cpu_time_us":39013,"raft.run_wall_time_us":297819,"replication_time_us":662641,"thread_start_us":611236,"threads_started":4,"wal-append.queue_time_us":236608}]]}
I20260709 13:01:21.243924  3017 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.338104  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1447) took 3564 ms. Trace:
I20260709 13:01:21.471055  1472 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.472010  1574 rpcz_store.cc:276] 0709 13:01:17.773915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.724231 (+1950316us) service_pool.cc:224] Handling call
0709 13:01:19.728843 (+  4612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.729785 (+   942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.729813 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.732765 (+  2952us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.388757 (+655992us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:20.423480 (+ 34723us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.424286 (+   806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.424522 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.337845 (+913323us) raft_consensus.cc:1685] finished
0709 13:01:21.337871 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.338033 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4951,"prepare.queue_time_us":79101,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":47,"raft.queue_time_us":5933,"raft.run_cpu_time_us":566,"raft.run_wall_time_us":74898,"thread_start_us":601720,"threads_started":1,"wal-append.queue_time_us":658969}
W20260709 13:01:21.292600  3109 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.253s	user 0.022s	sys 0.009s
I20260709 13:01:21.081549  3141 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:21.368556  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.249420  1571 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
W20260709 13:01:21.482056  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.293234  3051 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.485909  3051 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.296464  3033 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.488986  3033 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.491571  3033 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.342053  2862 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:21.494505  2862 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:21.495918  3051 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.244048  3042 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:21.356518  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:21.394471  3008 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.395004  3035 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.396019  3060 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:21.446586  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.304032  2901 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.245195  3000 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:21.344520  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:21.475412  1472 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.260689  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:21.332502  3157 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:21.293963  3065 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.293491  3047 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:21.300539  3092 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.344s	user 0.014s	sys 0.011s
I20260709 13:01:21.501346  2953 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:21.510469  1458 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 154019us
W20260709 13:01:21.353211  3152 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.453s	user 0.002s	sys 0.000s
W20260709 13:01:21.520061  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.077s	sys 0.155s
W20260709 13:01:21.540946  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.077s	sys 0.155s
W20260709 13:01:21.520371  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.366380  3089 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:21.523089  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 262508us
I20260709 13:01:21.371100  2956 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:21.383898  3071 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.526466  1472 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:21.516280  3124 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.058s	user 0.012s	sys 0.002s
W20260709 13:01:21.534785  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.084s	sys 0.084s
W20260709 13:01:21.550081  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.084s	sys 0.084s
W20260709 13:01:21.535084  1617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.029s	sys 0.058s
W20260709 13:01:21.550674  1617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.029s	sys 0.058s
I20260709 13:01:21.544889  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:21.551373  1570 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.552161  1572 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:21.550467  1472 catalog_manager.cc:5697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.502478  3109 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:21.526031  3033 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:21.515244  3008 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.566294  1642 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:21.567461  3109 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:21.540771  1567 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:21.556712  1573 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:21.510807  2973 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.558414  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:21.559147  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:21.530918  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1454) took 3634 ms. Trace:
I20260709 13:01:21.515794  2904 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.536199  3025 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.599431  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:21.571632  1570 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:21.554280  1472 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.581878  3145 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:21.585605  3160 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:21.589310  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:21.554278  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 660350us
I20260709 13:01:21.595297  1650 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:21.590624  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.595957  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.568565  1718 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:21.367911  3134 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.357458  3151 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:21.541481  3092 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.558360  3124 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:21.606475  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.006s	sys 0.061s
I20260709 13:01:21.629333  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:21.639279  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.008s	sys 0.062s
W20260709 13:01:21.639693  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.11554e+06us
I20260709 13:01:21.640942  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:21.606657  2344 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.015s	sys 0.028s
I20260709 13:01:21.605515  1472 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.642813  3065 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.642762  3157 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:21.644263  3124 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.643400  1472 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.628166  1651 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:21.605803  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:21.563532  3005 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:01:21.650002  1575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.005s	sys 0.005s
I20260709 13:01:21.641633  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:21.650365  1575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.005s	sys 0.005s
I20260709 13:01:21.650831  1650 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:21.644513  2973 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.563589  3033 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:21.628720  2955 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:21.605844  2374 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:21.572830  2862 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:21.599244  3164 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:21.633507  2884 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.749s	user 0.016s	sys 0.011s
I20260709 13:01:21.656837  1647 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 3. Ops: []
W20260709 13:01:21.647826  3059 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.346s	user 0.007s	sys 0.010s
I20260709 13:01:21.594573  3147 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
W20260709 13:01:21.678735  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73037.7us
I20260709 13:01:21.646667  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.679606  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.671134  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:21.680265  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.006s	sys 0.013s
W20260709 13:01:21.681782  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.006s	sys 0.013s
W20260709 13:01:21.682149  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 678049us
I20260709 13:01:21.653234  3033 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.636354  3119 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:01:21.687994  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.693696  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.698704  1650 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
W20260709 13:01:21.693897  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.705801  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:21.711675  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73849.2us
W20260709 13:01:21.647348  2344 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.017s	sys 0.030s
I20260709 13:01:21.632901  3025 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:21.669399  3129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.010s	sys 0.011s
I20260709 13:01:21.719936  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:21.720489  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 736484us
W20260709 13:01:21.690125  2856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.006s	sys 0.026s
I20260709 13:01:21.685735  2955 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:21.726300  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.684379  3150 raft_consensus.cc:2764] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:21.740094  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.737881  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.705097  2956 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:21.723342  2919 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:21.728783  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46477.5us
I20260709 13:01:21.745034  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.689633  3166 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:21.746660  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:21.686614  3164 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.756079  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:21.757742  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:21.758621  3164 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.759883  3164 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:01:21.728552  2375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.008s	sys 0.012s
W20260709 13:01:21.761648  2375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.009s	sys 0.012s
I20260709 13:01:21.766105  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:21.767009  1570 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:21.687369  2884 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.748917  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.780311  1649 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:21.676141  2862 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.750358  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:21.747896  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 110742us
I20260709 13:01:21.722908  2941 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:21.736403  3131 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.285s	user 0.013s	sys 0.003s
I20260709 13:01:21.739151  1646 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:21.733234  2856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.007s	sys 0.026s
W20260709 13:01:21.730901  3129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.010s	sys 0.012s
I20260709 13:01:21.781965  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:21.783423  1570 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:21.691561  3059 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:21.733817  3005 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:21.736476  2955 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:21.740831  2901 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.806017  2941 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:21.775008  2845 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:01:21.749872  3106 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
W20260709 13:01:21.934875  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.704787  3155 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:01:21.918450  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:21.594388  1569 rpcz_store.cc:276] 0709 13:01:17.894392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.914784 (+2020392us) service_pool.cc:224] Handling call
0709 13:01:19.945682 (+ 30898us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.017652 (+ 71970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.021451 (+  3799us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.146822 (+125371us) log.cc:844] Serialized 26 byte log entry
0709 13:01:20.385512 (+238690us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:20.385917 (+   405us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.397419 (+ 11502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.406332 (+  8913us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.250184 (+843852us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b1800244b40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:21.478310 (+228126us) raft_consensus.cc:1685] finished
0709 13:01:21.518772 (+ 40462us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.528946 (+ 10174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78649,"prepare.queue_time_us":230,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":1355,"raft.queue_time_us":56718,"raft.run_cpu_time_us":585,"raft.run_wall_time_us":40791,"spinlock_wait_cycles":9780096,"thread_start_us":208315,"threads_started":1,"wal-append.queue_time_us":248411}
I20260709 13:01:21.983248  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:21.902771  1647 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:21.862689  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 171691us
I20260709 13:01:21.976050  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:21.976265  1649 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:21.839339  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:21.907325  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:21.974438  1650 raft_consensus.cc:2410] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:21.774868  3060 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:21.798542  3047 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:21.844370  1648 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:21.860370  2904 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.006898  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:21.891937  3089 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.008642  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:21.943009  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 299) took 1168 ms. Trace:
I20260709 13:01:21.944597  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 74) took 1155 ms. Trace:
I20260709 13:01:21.896106  3008 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:22.044137  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 301206us
I20260709 13:01:21.799072  2919 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.262s
I20260709 13:01:21.819095  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:22.087857  1614 connection.cc:256] RPC call timeout handler was delayed by -1.8973123029999783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2469s for a response
W20260709 13:01:21.945886  2793 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
I20260709 13:01:21.894950  3147 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.838889  1722 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:22.052237  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:21.821278  3131 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.057397  3185 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:22.101042  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.058998  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.061587  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.107124  1649 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:22.107349  1720 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:22.072284  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96360us
I20260709 13:01:21.774412  3156 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.273s
I20260709 13:01:21.978605  3005 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:21.821758  3141 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.079452  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.130411  1721 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:22.079044  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:22.079428  1570 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
W20260709 13:01:22.088407  1614 connection.cc:256] RPC call timeout handler was delayed by -1.1102173269999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33640000000000003s for a response
I20260709 13:01:22.088168  1573 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.096101  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.048810  3092 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.100003  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:22.009840  2373 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:22.107972  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.147706  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.118045  1649 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:22.117703  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:22.120549  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 68382.3us
I20260709 13:01:22.059554  2845 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.126284  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 298) took 1357 ms. Trace:
I20260709 13:01:22.081609  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:22.081780  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.078389  3182 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:22.166508  1717 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:22.145081  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.150552  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.151224  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.142671  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.169320  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.154057  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.134691  3151 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.143826  3099 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:22.143920  1645 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:22.141144  3106 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:22.163044  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.179208  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.152916  3134 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:22.195837  3172 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:01:22.196301  2941 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:22.196475  3169 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:01:22.082154  1571 rpcz_store.cc:276] 0709 13:01:20.789124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.789471 (+   347us) service_pool.cc:224] Handling call
0709 13:01:21.944375 (+1154904us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000262d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:21.944521 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833316224}
I20260709 13:01:22.189988  1645 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:22.143215  2793 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.192117  1570 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.161109  3025 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:22.193830  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.567s (SENT) [suppressed 8 similar messages]
I20260709 13:01:22.157266  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.200019  1649 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:22.189422  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.142856  2862 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.197347  3008 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:22.201313  3175 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:01:22.137929  3191 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:22.082082  2374 rpcz_store.cc:276] 0709 13:01:20.774568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.775865 (+  1297us) service_pool.cc:224] Handling call
0709 13:01:20.799882 (+ 24017us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.664796 (+864914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.942554 (+277758us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54006fe0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:21.942901 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774707968}
I20260709 13:01:22.200845  1725 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:22.089963  2955 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:22.193986  1720 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:22.211753  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.207057  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.122051  3185 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.199714  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:22.213305  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.216562  1725 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:22.194485  3065 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:22.115134  3147 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.193233  2884 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.170538  2845 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.223205  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:22.229974  3178 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 13:01:22.199004  2941 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:22.242277  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.252771  1721 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.137794  3159 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:01:22.230851  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.244982  3005 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:22.223054  3034 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.870s	user 0.005s	sys 0.011s
I20260709 13:01:22.212821  2793 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:22.258126  1720 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:22.222071  3191 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:22.234045  2955 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.254192  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.245136  3134 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:22.261157  3017 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:22.262318  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
W20260709 13:01:22.260775  3170 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:01:22.266044  1573 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:22.268280  3179 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:01:22.250312  3059 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.270429  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.276196  1725 raft_consensus.cc:2410] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:22.270944  3131 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.271356  2845 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.263780  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.258229  2941 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:22.260128  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:22.269428  3033 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 13:01:22.257671  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:22.295202  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.178488  3182 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.297508  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:22.299607  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.300984  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.291312  2793 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:22.321024  3177 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:01:22.297030  3033 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:22.150455  2375 rpcz_store.cc:276] 0709 13:01:20.769052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.772573 (+  3521us) service_pool.cc:224] Handling call
0709 13:01:20.874647 (+102074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.054898 (+180251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.061441 (+  6543us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.074254 (+ 12813us) log.cc:844] Serialized 564 byte log entry
0709 13:01:21.762499 (+688245us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:21.762566 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.762680 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.125984 (+363304us) raft_consensus.cc:1685] finished
0709 13:01:22.126011 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.126202 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":687730,"threads_started":1,"wal-append.queue_time_us":899828}
I20260709 13:01:22.301878  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:22.326716  2973 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.003s
I20260709 13:01:22.327088  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.289968  1721 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:22.298252  3134 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:22.327572  1651 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:22.296731  1719 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:22.328600  3182 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.328662  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.299927  3017 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:22.262832  3034 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.331589  3033 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:22.332281  1576 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:22.333050  3034 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.308831  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 77) took 1518 ms. Trace:
I20260709 13:01:22.333653  1575 rpcz_store.cc:276] 0709 13:01:20.795112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.088200 (+293088us) service_pool.cc:224] Handling call
0709 13:01:21.100472 (+ 12272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.110185 (+  9713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.110219 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.147340 (+ 37121us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.650729 (+503389us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:21.761382 (+110653us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.761578 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.761651 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.308574 (+546923us) raft_consensus.cc:1685] finished
0709 13:01:22.308601 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.308758 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9238,"prepare.queue_time_us":155003,"prepare.run_cpu_time_us":449,"prepare.run_wall_time_us":2615,"raft.queue_time_us":56059,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":22649,"thread_start_us":589099,"threads_started":2,"wal-append.queue_time_us":626891}
I20260709 13:01:22.310812  1570 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:22.311699  2835 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:22.335568  2835 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:22.335695  3008 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:22.336015  1716 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:22.336956  2835 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.338487  2835 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.342526  3042 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:22.343071  3042 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:22.346638  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.296730  2845 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:22.318504  2941 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.318717  3181 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:22.363272  3185 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:22.363691  3179 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:22.325026  1725 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:22.330451  2955 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:22.330868  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:22.318032  3176 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.381s	user 0.001s	sys 0.000s
I20260709 13:01:22.352854  1574 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:01:22.360750  3174 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.774s	user 0.005s	sys 0.000s
I20260709 13:01:22.361178  3134 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:22.359645  3005 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:22.359311  2845 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:22.360780  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:22.382355  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.387951  1719 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:22.390159  3071 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.386458  2845 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.394735  3051 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:22.395172  3051 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:22.291250  3191 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.386940  3147 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:22.398763  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:22.401048  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:22.366195  3099 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:01:22.405794  3005 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:22.406497  2955 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:22.406867  2955 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.407204  2955 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.408363  2845 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:22.410436  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1608) took 1529 ms (client timeout 1899 ms). Trace:
I20260709 13:01:22.410609  1724 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:22.410838  1717 rpcz_store.cc:269] 0709 13:01:20.880804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.880975 (+   171us) service_pool.cc:224] Handling call
0709 13:01:22.410231 (+1529256us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540010e0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:22.410380 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704643456}
I20260709 13:01:22.344097  3042 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:22.413296  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.414033  1717 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:22.411897  3181 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:22.415670  3181 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:22.415994  3181 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:22.416566  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1616) took 1509 ms. Trace:
I20260709 13:01:22.416921  1723 rpcz_store.cc:276] 0709 13:01:20.906778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.043213 (+136435us) service_pool.cc:224] Handling call
0709 13:01:21.047798 (+  4585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.048278 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.048329 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.081164 (+ 32835us) log.cc:844] Serialized 564 byte log entry
0709 13:01:21.584448 (+503284us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:21.584490 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.584561 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.351439 (+766878us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:22.416284 (+ 64845us) raft_consensus.cc:1685] finished
0709 13:01:22.416303 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.416508 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38782720,"thread_start_us":425533,"threads_started":1,"wal-append.queue_time_us":473973}
I20260709 13:01:22.424880  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.427637  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 293) took 1966 ms. Trace:
I20260709 13:01:22.428063  2373 rpcz_store.cc:276] 0709 13:01:20.460633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.462689 (+  2056us) service_pool.cc:224] Handling call
0709 13:01:20.481766 (+ 19077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.160894 (+1679128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.416263 (+255369us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54005e2754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:22.426703 (+ 10440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3476050304}
I20260709 13:01:22.417968  3134 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:22.457957  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 351391us
I20260709 13:01:22.417896  2845 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.480993  1725 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.481218  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.494692  1570 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.482772  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.495982  1573 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.487952  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.497179  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.497946  1645 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:22.488245  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.396035  3051 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:22.433852  3166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.044s	sys 0.092s
I20260709 13:01:22.520107  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:22.516824  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:22.520044  3166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.044s	sys 0.092s
I20260709 13:01:22.547777  3166 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:22.547642  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.550907  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.552444  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.417335  3181 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:22.556164  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.497000  3042 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.565145  1650 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:22.437088  3005 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 13:01:22.595256  2955 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.460073  3099 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:01:22.600889  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.607193  1570 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:22.612287  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 478611us
W20260709 13:01:22.613847  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 92371us
I20260709 13:01:22.614943  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:22.615072  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:22.615965  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:22.617071  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 423276us
I20260709 13:01:22.622742  1573 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:22.624488  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:22.625480  1573 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:22.626526  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.627250  1573 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.629179  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.556192  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.566329  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:22.596479  3150 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.003s	sys 0.010s
I20260709 13:01:22.640774  1650 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:22.645996  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:22.663380  1570 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:22.598263  3051 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:22.640987  1688 connection.cc:256] RPC call timeout handler was delayed by -2.6503252800000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:01:22.669435  1688 connection.cc:256] RPC call timeout handler was delayed by -2.4091252799999552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:01:22.669890  1688 connection.cc:256] RPC call timeout handler was delayed by -2.3083252799999627s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260709 13:01:22.632313  1575 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:22.631358  3088 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.008s	sys 0.026s
W20260709 13:01:22.665373  2376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.013s	sys 0.009s
I20260709 13:01:22.663808  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:22.657724  3071 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.678745  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:22.631722  3189 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:22.681653  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:22.676010  2793 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.716814  1573 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:22.632128  2904 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:01:22.686878  1650 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:22.680365  3182 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.729141  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.725924  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.730865  1570 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.731372  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:22.731647  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:22.684581  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.017s	sys 0.020s
W20260709 13:01:22.732533  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.020s
W20260709 13:01:22.732903  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 608883us
I20260709 13:01:22.722980  2374 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.727063  3147 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:22.740384  3035 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:22.740770  3147 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:22.681228  3088 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.008s	sys 0.029s
W20260709 13:01:22.683491  2376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.013s	sys 0.009s
I20260709 13:01:22.741987  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:22.680770  3156 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.008s	sys 0.016s
I20260709 13:01:22.742455  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:22.658464  3166 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:22.747885  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:22.899259  3156 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.013s	sys 0.033s
I20260709 13:01:22.723512  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1460) took 4763 ms. Trace:
W20260709 13:01:22.769980  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.032s	user 0.021s	sys 0.048s
W20260709 13:01:22.901634  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.166s	user 0.028s	sys 0.059s
I20260709 13:01:22.784488  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:22.818804  1570 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:22.904641  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.908574  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:22.914188  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:22.915792  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:22.848205  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1614) took 1965 ms. Trace:
I20260709 13:01:22.919445  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.749073  2793 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:22.965191  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.753355  3147 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:22.782320  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:22.727648  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:22.850030  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 85) took 1102 ms. Trace:
I20260709 13:01:22.721712  3071 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:22.729265  3189 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:22.848948  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1584) took 1095 ms. Trace:
I20260709 13:01:22.980530  1651 rpcz_store.cc:276] 0709 13:01:21.753372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.753677 (+   305us) service_pool.cc:224] Handling call
0709 13:01:21.754247 (+   570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.327986 (+573739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.848685 (+520699us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540031bad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:22.848882 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317879040}
W20260709 13:01:22.939397  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.940920  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 874) took 1131 ms. Trace:
I20260709 13:01:22.942319  1722 rpcz_store.cc:276] 0709 13:01:20.890182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.930811 (+ 40629us) service_pool.cc:224] Handling call
0709 13:01:21.262828 (+332017us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:21.263162 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.094001 (+830839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.847945 (+753944us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400213754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:22.848141 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335629824}
I20260709 13:01:22.982821  1574 rpcz_store.cc:276] 0709 13:01:21.809188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.274645 (+465457us) service_pool.cc:224] Handling call
0709 13:01:22.275049 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.373461 (+ 98412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.940576 (+567115us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b54002a67d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:22.940832 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178255232}
I20260709 13:01:22.983376  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:22.944064  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:22.963832  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:22.727942  3017 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:22.967530  3192 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
I20260709 13:01:22.968277  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:22.971698  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 229325us
I20260709 13:01:22.973637  2378 raft_consensus.cc:2410] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:22.756688  3149 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:22.976189  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.811684  3204 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:22.974824  1576 rpcz_store.cc:276] 0709 13:01:21.747642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.748527 (+   885us) service_pool.cc:224] Handling call
0709 13:01:22.849799 (+1101272us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54000cba54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:22.849961 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258593536}
I20260709 13:01:22.795393  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.777256  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:22.973899  3202 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.028384  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.984151  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.999150  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:22.935967  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:23.002209  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58265.6us
I20260709 13:01:23.001677  2380 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:23.010165  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:22.964319  2955 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:23.011246  1725 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.001s
I20260709 13:01:23.120965  1725 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
W20260709 13:01:23.121115  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 218966us
I20260709 13:01:22.936746  1567 rpcz_store.cc:276] 0709 13:01:17.923039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.972502 (+2049463us) service_pool.cc:224] Handling call
0709 13:01:19.973260 (+   758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.578154 (+1604894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.679178 (+1101024us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:22.686460 (+  7282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3576542080}
I20260709 13:01:23.020877  2376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 303) took 1013 ms. Trace:
I20260709 13:01:23.063694  1569 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.126636  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:23.084810  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:22.983829  2375 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:23.067994  1651 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:23.140928  3198 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.141651  3198 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:23.142786  3198 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.099407  3202 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.009068  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 73) took 3011 ms. Trace:
I20260709 13:01:23.006150  3035 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.984822  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:23.104842  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 197016us
W20260709 13:01:23.104269  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 132233us
I20260709 13:01:23.027806  3204 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:23.110667  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.101377  1574 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.100615  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:23.254782  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 198237us
I20260709 13:01:23.081202  2919 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.977893  2835 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.085026  2973 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.124312  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.094813  3021 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.081362  3199 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:23.261677  1722 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:23.096925  3156 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.263985  3021 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.264354  2973 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.264415  3156 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.266098  3021 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.266304  2973 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.266705  3156 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.263833  2835 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:23.127619  3203 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.160s	user 0.000s	sys 0.002s
I20260709 13:01:23.125520  2376 rpcz_store.cc:276] 0709 13:01:22.000074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.006176 (+  6102us) service_pool.cc:224] Handling call
0709 13:01:22.071575 (+ 65399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.942854 (+871279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.963234 (+ 20380us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.975456 (+ 12222us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:22.977513 (+  2057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.984670 (+  7157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.013569 (+ 28899us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8625,"raft.queue_time_us":146978,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":5586,"thread_start_us":750678,"threads_started":1}
I20260709 13:01:23.122612  3088 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:23.137563  2374 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:23.139890  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.261256  2919 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.066138  3180 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:01:23.011739  3017 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.236791  1719 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:23.162470  1642 rpcz_store.cc:276] 0709 13:01:19.989897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.004130 (+ 14233us) service_pool.cc:224] Handling call
0709 13:01:20.073230 (+ 69100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.566788 (+1493558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.972651 (+1405863us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001ba3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:22.984316 (+ 11665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328593280}
I20260709 13:01:23.225620  3219 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:01:23.189293  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1618) took 2187 ms. Trace:
I20260709 13:01:23.128753  2793 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.237272  3047 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:23.198536  3206 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 13:01:23.129604  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 864) took 1581 ms. Trace:
I20260709 13:01:23.288118  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.289572  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.288542  1568 rpcz_store.cc:276] 0709 13:01:21.545191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.552632 (+  7441us) service_pool.cc:224] Handling call
0709 13:01:21.594452 (+ 41820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.780924 (+186472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.784612 (+  3688us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.999399 (+214787us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.144104 (+144705us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:22.159214 (+ 15110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.169013 (+  9799us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.103957 (+934944us) raft_consensus.cc:1685] finished
0709 13:01:23.103977 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.126476 (+ 22499us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29485,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":19021,"thread_start_us":217236,"threads_started":2,"wal-append.queue_time_us":437405}
I20260709 13:01:23.290742  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.290807  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.291528  1642 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.291664  1568 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:23.292948  3214 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:01:23.258463  3147 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:23.295259  3042 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
W20260709 13:01:23.296018  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.296839  3147 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:23.231549  3202 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.148729  3189 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:23.297629  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:23.298743  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.299263  1575 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:23.184999  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:23.238081  3051 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:01:23.189452  3035 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.300566  2955 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.300693  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.301152  2955 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.301249  3035 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.301891  1650 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:23.302784  3182 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
I20260709 13:01:23.303179  3182 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.303251  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.303534  2955 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:23.303874  1650 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.191493  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:23.168597  3208 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:01:23.225086  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.086059  3194 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
W20260709 13:01:23.252576  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 130649us
I20260709 13:01:23.307047  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.185915  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.261415  3199 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:23.158468  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:23.257597  3109 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:23.272315  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:23.270804  2835 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.279068  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.281582  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:23.321558  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 135768us
I20260709 13:01:23.284232  1718 rpcz_store.cc:276] 0709 13:01:20.997293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.227436 (+230143us) service_pool.cc:224] Handling call
0709 13:01:21.227804 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.652918 (+425114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.128735 (+1475817us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54003ac354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:23.184738 (+ 56003us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747874688}
I20260709 13:01:23.301353  1725 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:23.285650  3156 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:23.210842  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.300060  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:23.305558  1570 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:23.306099  3195 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:01:23.310057  1576 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:23.296609  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.318421  1723 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:23.319043  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 416476us
I20260709 13:01:23.321964  1651 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:23.329493  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.323231  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.330785  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.255415  3071 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:23.332008  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.333498  1651 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:23.334473  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.335731  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.337146  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.277431  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 871) took 1513 ms. Trace:
I20260709 13:01:23.296939  2919 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.309769  3199 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.338544  1575 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:23.320574  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.322448  1574 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:23.280862  3088 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.349213  1725 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:23.318126  3147 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.305994  3047 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.374616  3047 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:23.337536  1723 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:23.285912  3149 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.347767  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:23.338155  3109 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.357499  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:23.358669  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.383278  1574 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:23.347579  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.362097  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.324196  3156 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:23.338330  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.344441  2955 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.363366  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1625) took 1203 ms. Trace:
I20260709 13:01:23.394940  1725 rpcz_store.cc:276] 0709 13:01:22.160102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.346446 (+1186344us) service_pool.cc:224] Handling call
0709 13:01:23.348770 (+  2324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.363039 (+ 14269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.363290 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.345440  3042 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.367893  3204 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.369040  3196 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:23.370035  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 88) took 1576 ms. Trace:
I20260709 13:01:23.372189  3088 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.372543  3147 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.399593  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:23.402359  1572 rpcz_store.cc:276] 0709 13:01:21.793420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.271847 (+478427us) service_pool.cc:224] Handling call
0709 13:01:22.272173 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.272596 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.272635 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.293696 (+ 21061us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.695066 (+401370us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:22.740248 (+ 45182us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.740476 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.740545 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.369843 (+629298us) raft_consensus.cc:1685] finished
0709 13:01:23.369863 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.369985 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45410,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":95976,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":18730,"thread_start_us":400122,"threads_started":1,"wal-append.queue_time_us":676515}
I20260709 13:01:23.378695  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.378894  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:23.354067  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.405544  1723 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:23.385355  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:23.404881  1575 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.407095  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.388777  1650 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:23.352596  3071 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:23.398764  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.398974  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.410331  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.408589  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:23.408007  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54025.1us
I20260709 13:01:23.409348  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.413578  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.414072  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.415760  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.393935  3047 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:23.419566  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.422556  1651 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:23.420405  1572 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:23.423431  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.420938  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.424194  1723 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:23.425483  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.426611  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.427438  3213 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:23.428092  3213 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:23.427505  3149 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.428514  3213 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.430222  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.431914  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.432571  1723 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:23.433213  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1631) took 1224 ms. Trace:
I20260709 13:01:23.433548  1723 rpcz_store.cc:276] 0709 13:01:22.209268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.424879 (+1215611us) service_pool.cc:224] Handling call
0709 13:01:23.432355 (+  7476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.432978 (+   623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.433165 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.433745  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.398325  3042 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.472213  1576 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:23.440114  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.460171  1723 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:23.474056  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1633) took 1249 ms. Trace:
I20260709 13:01:23.413484  3193 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:23.407142  3196 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:23.467518  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.475986  1650 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:23.474351  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.407941  3051 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.450728  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:23.474332  1723 rpcz_store.cc:276] 0709 13:01:22.225054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.434553 (+1209499us) service_pool.cc:224] Handling call
0709 13:01:23.459942 (+ 25389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.473811 (+ 13869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.474007 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:23.497318  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46722.1us
I20260709 13:01:23.416469  3219 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.477339  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:23.499410  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1485) took 1014 ms. Trace:
I20260709 13:01:23.503144  1576 rpcz_store.cc:276] 0709 13:01:22.484682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.479735 (+995053us) service_pool.cc:224] Handling call
0709 13:01:23.498958 (+ 19223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.499123 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.499146 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.499186 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:23.499213 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.499261 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.499359 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.490263  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.504830  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:23.500303  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 193372us
I20260709 13:01:23.505431  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.505478  1650 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:23.507536  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.511384  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.502125  3229 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.512319  2919 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:23.401978  2955 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.429488  3147 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.460011  3180 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.421474  3210 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.513973  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.430749  3047 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:23.477905  3159 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.435415  3166 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.502386  3071 raft_consensus.cc:2764] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:23.543251  1651 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:23.544102  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1601) took 1053 ms. Trace:
I20260709 13:01:23.544487  1651 rpcz_store.cc:276] 0709 13:01:22.490445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.515542 (+1025097us) service_pool.cc:224] Handling call
0709 13:01:23.542799 (+ 27257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.543774 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.544036 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.434893  2956 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.459908  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.501618  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:23.519148  3236 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.516551  3194 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.512980  1642 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:23.472220  3005 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:23.521909  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:23.558775  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49721.5us
I20260709 13:01:23.559096  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.559427  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:23.478879  3196 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.565740  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:23.565987  1642 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:23.557951  3071 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:23.479790  3193 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:23.546911  1650 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.573627  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:23.567451  3134 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.085s	user 0.008s	sys 0.004s
I20260709 13:01:23.435182  3217 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:23.558295  3005 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.538895  3047 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.550861  3210 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.553236  1574 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.577644  3071 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.554476  1576 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:23.594496  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 88844.2us
I20260709 13:01:23.459753  3191 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.531754  1568 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:01:23.557003  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 126862us
I20260709 13:01:23.596158  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.577813  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 856) took 1075 ms. Trace:
I20260709 13:01:23.529784  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:55254 (request call id 75) took 1295 ms. Trace:
I20260709 13:01:23.574827  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.521363  2919 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:01:23.470920  2973 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:01:23.578382  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 842) took 1091 ms. Trace:
I20260709 13:01:23.563299  2956 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:23.561529  3198 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.520534  3109 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.417338  3230 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.563293  3237 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:23.579293  3231 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:01:23.589524  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:23.547236  3042 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.591912  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 876) took 1041 ms. Trace:
I20260709 13:01:23.610617  1574 rpcz_store.cc:276] 0709 13:01:22.551020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.541928 (+990908us) service_pool.cc:224] Handling call
0709 13:01:23.553047 (+ 11119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.591592 (+ 38545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.591840 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:23.554127  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52585.1us
I20260709 13:01:23.596918  3071 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.614439  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.593770  3166 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:23.597267  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1481) took 1113 ms. Trace:
I20260709 13:01:23.583135  3134 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.560307  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:23.624166  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63943.7us
I20260709 13:01:23.599062  1642 rpcz_store.cc:276] 0709 13:01:22.502652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.565221 (+1062569us) service_pool.cc:224] Handling call
0709 13:01:23.565733 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.566611 (+   878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.577045 (+ 10434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.582144  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1603) took 1088 ms. Trace:
I20260709 13:01:23.600576  1649 rpcz_store.cc:276] 0709 13:01:22.487322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.487430 (+   108us) service_pool.cc:224] Handling call
0709 13:01:23.573416 (+1085986us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:23.578292 (+  4876us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17578752}
I20260709 13:01:23.599915  1723 rpcz_store.cc:276] 0709 13:01:22.234218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.497240 (+1263022us) service_pool.cc:224] Handling call
0709 13:01:23.529648 (+ 32408us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:23.627168  2344 connection.cc:256] RPC call timeout handler was delayed by -1.1156713670000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 13:01:23.343849  1571 rpcz_store.cc:276] 0709 13:01:21.758016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.231473 (+473457us) service_pool.cc:224] Handling call
0709 13:01:22.231996 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.232487 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.232544 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.233168 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.678838 (+445670us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:22.678893 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.679004 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.271570 (+592566us) raft_consensus.cc:1685] finished
0709 13:01:23.271592 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.271743 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1779,"raft.queue_time_us":13271,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3838,"thread_start_us":439935,"threads_started":1,"wal-append.queue_time_us":734173}
I20260709 13:01:23.603579  1722 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 13:01:23.601747  2956 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.605491  2919 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:23.549077  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:23.597465  3005 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.605739  2955 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:23.618229  3157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.552s	user 0.010s	sys 0.008s
I20260709 13:01:23.624483  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:23.625276  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.609056  3240 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.622783  3217 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:23.628680  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:23.630470  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 84) took 1263 ms. Trace:
I20260709 13:01:23.633759  2956 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:23.609618  3237 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:23.631608  2377 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.635537  1574 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:23.649036  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46334.1us
I20260709 13:01:23.649209  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 878) took 1097 ms. Trace:
I20260709 13:01:23.625197  3182 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:01:23.634330  2919 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.599612  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 855) took 1097 ms. Trace:
I20260709 13:01:23.652221  1645 rpcz_store.cc:276] 0709 13:01:22.502354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.525326 (+ 22972us) service_pool.cc:224] Handling call
0709 13:01:23.574647 (+1049321us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:23.599550 (+ 24903us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29116288}
I20260709 13:01:23.641237  1649 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:23.623414  1567 rpcz_store.cc:276] 0709 13:01:22.483595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.554138 (+1070543us) service_pool.cc:224] Handling call
0709 13:01:23.561702 (+  7564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.573354 (+ 11652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.576470 (+  3116us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.578674 (+  2204us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:23.583270 (+  4596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.587278 (+  4008us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.596055 (+  8777us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.596728  3210 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.643262  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1630) took 1439 ms. Trace:
I20260709 13:01:23.644096  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.644965  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:23.627159  1644 rpcz_store.cc:276] 0709 13:01:22.491093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.459451 (+968358us) service_pool.cc:224] Handling call
0709 13:01:23.579170 (+119719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.633185  3242 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.631168  3230 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:23.646564  1723 rpcz_store.cc:276] 0709 13:01:22.366747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.627731 (+1260984us) service_pool.cc:224] Handling call
0709 13:01:23.628089 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.630177 (+  2088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.630213 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.630256 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:23.630284 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.630345 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.630428 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1763,"raft.queue_time_us":1873,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5799}
I20260709 13:01:23.662101  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 86) took 1294 ms. Trace:
I20260709 13:01:23.662446  1723 rpcz_store.cc:276] 0709 13:01:22.367544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.661360 (+1293816us) service_pool.cc:224] Handling call
0709 13:01:23.661662 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.661826 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.661847 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.661889 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:23.661913 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.661963 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.662055 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.663389  1723 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:23.629537  3161 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.649634  1574 rpcz_store.cc:276] 0709 13:01:22.551910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.628813 (+1076903us) service_pool.cc:224] Handling call
0709 13:01:23.635303 (+  6490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.648962 (+ 13659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.649153 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.665937  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 87) took 1298 ms. Trace:
I20260709 13:01:23.666334  1723 rpcz_store.cc:276] 0709 13:01:22.367808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.663096 (+1295288us) service_pool.cc:224] Handling call
0709 13:01:23.663282 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.665718 (+  2436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.665889 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.649994  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:23.643126  3157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.587s	user 0.011s	sys 0.008s
I20260709 13:01:23.668045  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:23.653368  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 862) took 1104 ms. Trace:
I20260709 13:01:23.653646  1645 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:23.633678  3042 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.623085  3166 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.654847  1725 rpcz_store.cc:276] 0709 13:01:22.204392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.419932 (+1215540us) service_pool.cc:224] Handling call
0709 13:01:23.478367 (+ 58435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.478629 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.478669 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.479185 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.577952 (+ 98767us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:23.577986 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.578044 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.643039 (+ 64995us) raft_consensus.cc:1685] finished
0709 13:01:23.643063 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.643206 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":98464,"threads_started":1,"wal-append.queue_time_us":129988}
I20260709 13:01:23.645946  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:23.657601  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:23.646283  3140 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.634716  3238 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.658263  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 881) took 1104 ms. Trace:
I20260709 13:01:23.670805  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 80) took 1546 ms. Trace:
I20260709 13:01:23.651722  3217 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:23.671237  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 866) took 1055 ms. Trace:
I20260709 13:01:23.692086  1645 rpcz_store.cc:276] 0709 13:01:22.615573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.653149 (+1037576us) service_pool.cc:224] Handling call
0709 13:01:23.653452 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.670984 (+ 17532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.671185 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.650695  3237 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.657068  2376 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:23.650084  1569 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:23.671156  3240 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.676947  3210 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.660043  3191 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.679672  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:23.649392  2793 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:01:23.669533  1649 rpcz_store.cc:276] 0709 13:01:22.549442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.637784 (+1088342us) service_pool.cc:224] Handling call
0709 13:01:23.641008 (+  3224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.653144 (+ 12136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.653329 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.683643  1647 rpcz_store.cc:276] 0709 13:01:22.124915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.185209 (+ 60294us) service_pool.cc:224] Handling call
0709 13:01:23.667911 (+1482702us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:23.670738 (+  2827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6072576}
I20260709 13:01:23.685057  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:23.710081  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.688577  3054 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:23.688894  2919 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.633625  3193 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.660409  2973 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:23.697345  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:23.700493  1725 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:23.722882  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1636) took 1240 ms. Trace:
I20260709 13:01:23.723220  1725 rpcz_store.cc:276] 0709 13:01:22.482357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.674973 (+1192616us) service_pool.cc:224] Handling call
0709 13:01:23.700227 (+ 25254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.722630 (+ 22403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.722842 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.700871  2835 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.700006  3182 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:23.702591  3051 raft_consensus.cc:2764] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:23.711649  1567 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:23.720686  3234 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:01:23.708412  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1489) took 1222 ms. Trace:
I20260709 13:01:23.740020  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:23.714931  1644 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.712699  3054 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.721768  3160 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:23.745870  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.724874  3191 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.705739  3157 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:01:23.728081  3195 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:23.690568  3242 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.749931  3054 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.725926  3130 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:01:23.726670  3213 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.765951  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1632) took 1548 ms. Trace:
I20260709 13:01:23.779472  3217 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.742398  2835 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:23.740275  3182 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.728614  3161 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:01:23.728305  2973 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.690575  3230 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.738478  3210 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:01:23.683408  1571 rpcz_store.cc:276] 0709 13:01:22.552773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.637722 (+1084949us) service_pool.cc:224] Handling call
0709 13:01:23.642693 (+  4971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.642884 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.642912 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.642969 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:23.645564 (+  2595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.651547 (+  5983us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.657479 (+  5932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.785601  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:23.780623  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:23.786657  1720 rpcz_store.cc:276] 0709 13:01:22.217180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.222882 (+  5702us) service_pool.cc:224] Handling call
0709 13:01:23.765755 (+1542873us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400372154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:23.765875 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25615232}
I20260709 13:01:23.795013  3130 consensus_queue.cc:1243] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:23.765870  3194 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.758167  2919 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.825116  3194 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.748039  1569 rpcz_store.cc:276] 0709 13:01:22.485948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.756561 (+270613us) service_pool.cc:224] Handling call
0709 13:01:23.705661 (+949100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.823912  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:23.781224  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 83) took 1027 ms. Trace:
I20260709 13:01:23.797819  2835 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.812161  3229 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.804737  3005 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:23.812963  3211 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.727s	user 0.000s	sys 0.001s
I20260709 13:01:23.787369  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:23.825562  3182 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:23.834153  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46883us
I20260709 13:01:23.817665  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.782887  3199 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.837469  3035 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:23.839680  1651 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:23.840896  3199 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.832417  2372 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:23.819752  3210 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.820008  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:23.804781  3166 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:23.782702  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1495) took 1226 ms. Trace:
I20260709 13:01:23.849452  3047 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.850104  1567 rpcz_store.cc:276] 0709 13:01:22.555423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.660874 (+1105451us) service_pool.cc:224] Handling call
0709 13:01:23.707413 (+ 46539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.749965 (+ 42552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.781318 (+ 31353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.851313  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:23.830426  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.852721  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.854710  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:23.845323  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1490) took 1178 ms. Trace:
I20260709 13:01:23.847187  1647 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.847390  1725 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:23.846711  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:23.828011  1644 rpcz_store.cc:276] 0709 13:01:22.739020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.668984 (+929964us) service_pool.cc:224] Handling call
0709 13:01:23.699655 (+ 30671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.749870 (+ 50215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.766280 (+ 16410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.855659  1720 rpcz_store.cc:276] 0709 13:01:22.666676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.844237 (+1177561us) service_pool.cc:224] Handling call
0709 13:01:23.844659 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.844900 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.844939 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.844998 (+    59us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:23.845042 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.845112 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.845253 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.858556  2919 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.827450  3196 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.852133  3047 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:23.871061  3118 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:23.871572  3054 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.872284  3005 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.863945  2372 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:23.873184  2919 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.888510  3230 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:23.847648  3021 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:01:23.846915  3210 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:23.857450  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:23.890774  2919 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.892768  3210 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.894155  3210 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.885454  3054 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:23.869491  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.862792  3194 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.893301  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.903647  1571 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:23.903825  3194 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.905588  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:23.881887  3118 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:23.885881  2956 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.899520  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1569) took 2298 ms. Trace:
I20260709 13:01:23.874521  3017 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.896409  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:23.936559  1642 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:23.933974  3157 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:23.937248  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:23.891616  3005 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:23.934054  3230 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:23.896824  3246 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:23.934151  1644 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:23.932912  1648 rpcz_store.cc:276] 0709 13:01:21.601645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.631586 (+ 29941us) service_pool.cc:224] Handling call
0709 13:01:21.635472 (+  3886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.062776 (+427304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.899262 (+1836486us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400307354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:23.899458 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326805248}
I20260709 13:01:23.937682  3157 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.939399  3229 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:01:23.950423  3230 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:23.959734  3213 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.960428  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:23.964099  3246 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:23.942312  3005 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:23.978374  3230 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.959375  3071 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:23.980582  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 878) took 1142 ms. Trace:
I20260709 13:01:23.997170  1644 rpcz_store.cc:276] 0709 13:01:22.837696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.876322 (+1038626us) service_pool.cc:224] Handling call
0709 13:01:23.895225 (+ 18903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.961428 (+ 66203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.978780 (+ 17352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.008450  1723 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:23.985503  3246 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.984869  2973 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:23.986104  1571 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.023113  3005 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:24.025684  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1622) took 1879 ms (client timeout 1920 ms). Trace:
W20260709 13:01:24.026180  1721 rpcz_store.cc:269] 0709 13:01:22.145969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.153752 (+  7783us) service_pool.cc:224] Handling call
0709 13:01:24.025517 (+1871765us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:24.025631 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189823232}
I20260709 13:01:24.027526  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:24.028256  1721 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:23.979614  3118 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.029750  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:24.042270  1721 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:24.023113  3071 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:24.061630  3005 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:24.062397  3005 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.065936  1721 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.066125  3005 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:24.067644  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.074924  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1493) took 1406 ms. Trace:
I20260709 13:01:24.057420  3118 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.084690  1721 rpcz_store.cc:276] 0709 13:01:22.668975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.065393 (+1396418us) service_pool.cc:224] Handling call
0709 13:01:24.065727 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.074568 (+  8841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.074857 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.058200  3240 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:24.088428  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1494) took 1418 ms. Trace:
I20260709 13:01:24.089155  1721 rpcz_store.cc:276] 0709 13:01:22.669582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.087424 (+1417842us) service_pool.cc:224] Handling call
0709 13:01:24.087732 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.087940 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.088094 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.088138 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:24.088166 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.088227 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.088377 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.091300  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 88) took 1355 ms. Trace:
W20260709 13:01:24.091678  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 351752us
W20260709 13:01:24.091794  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 377603us
I20260709 13:01:24.110713  3157 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:24.092025  1721 rpcz_store.cc:276] 0709 13:01:22.736027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.090303 (+1354276us) service_pool.cc:224] Handling call
0709 13:01:24.090574 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.090938 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.090963 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.091005 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.091045 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.091109 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.091228 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.110872  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1491) took 1443 ms. Trace:
I20260709 13:01:24.058548  3160 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.087278  3017 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:24.153682  2378 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:24.111534  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:24.162689  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 51151.9us
I20260709 13:01:24.164198  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:24.114800  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 89) took 1378 ms. Trace:
I20260709 13:01:24.168022  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:24.168582  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 91) took 1431 ms. Trace:
I20260709 13:01:24.165333  1721 rpcz_store.cc:276] 0709 13:01:22.736389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.113730 (+1377341us) service_pool.cc:224] Handling call
0709 13:01:24.114313 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.114484 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.114515 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.114553 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.114589 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.114640 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.114750 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.173199  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:24.174559  1717 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.175107  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:24.175323  1721 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.176292  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:24.176582  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:24.177102  3210 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:24.177333  2793 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:24.177577  3210 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.177769  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:24.172866  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:24.179175  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:24.167677  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.182365  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:24.182341  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1495) took 1156 ms. Trace:
I20260709 13:01:24.183139  1721 rpcz_store.cc:276] 0709 13:01:23.025883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.173686 (+1147803us) service_pool.cc:224] Handling call
0709 13:01:24.174094 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.176793 (+  2699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.182285 (+  5492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.184791  1721 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.185527  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1497) took 1159 ms. Trace:
I20260709 13:01:24.185953  1721 rpcz_store.cc:276] 0709 13:01:23.026528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.184162 (+1157634us) service_pool.cc:224] Handling call
0709 13:01:24.184556 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.185253 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.185466 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.168043  3260 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:24.171015  3182 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.176069  3071 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.171634  3213 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:24.118326  3160 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.161813  3017 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.178058  3210 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.174958  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.190079  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:24.265301  3156 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.321527  3000 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:24.322443  3156 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:24.322564  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 132620us
I20260709 13:01:24.322737  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.323518  1648 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:24.266485  3252 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 13:01:24.310072  3249 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:01:24.312415  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 875) took 1475 ms. Trace:
I20260709 13:01:24.299746  3106 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:01:24.325678  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1493) took 1771 ms. Trace:
I20260709 13:01:24.303192  3246 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.326886  1567 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:24.326047  1575 rpcz_store.cc:276] 0709 13:01:22.554764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.634673 (+1079909us) service_pool.cc:224] Handling call
0709 13:01:23.634966 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.635278 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.635311 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.638383 (+  3072us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.965343 (+326960us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:24.064590 (+ 99247us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.065776 (+  1186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.066362 (+   586us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.325478 (+259116us) raft_consensus.cc:1685] finished
0709 13:01:24.325499 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.325627 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"prepare.queue_time_us":109426,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":3151,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":6265,"thread_start_us":391305,"threads_started":2,"wal-append.queue_time_us":435059}
I20260709 13:01:24.327980  3246 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:24.186842  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1602) took 3348 ms (client timeout 1580 ms). Trace:
I20260709 13:01:24.327803  3219 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:24.228914  3182 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.255565  1644 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:24.324635  1649 rpcz_store.cc:276] 0709 13:01:22.836700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.738275 (+901575us) service_pool.cc:224] Handling call
0709 13:01:23.751061 (+ 12786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.751814 (+   753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.751844 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.752386 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.131542 (+379156us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:24.247734 (+116192us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.249528 (+  1794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.249644 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.312101 (+ 62457us) raft_consensus.cc:1685] finished
0709 13:01:24.312134 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.312324 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":130481,"prepare.run_cpu_time_us":1283,"prepare.run_wall_time_us":16928,"raft.queue_time_us":2935,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":7377,"thread_start_us":445560,"threads_started":2,"wal-append.queue_time_us":498686}
I20260709 13:01:24.271340  1718 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:24.307539  3248 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.142s	user 0.001s	sys 0.001s
W20260709 13:01:24.332244  3226 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.106s	user 0.003s	sys 0.001s
I20260709 13:01:24.333453  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:24.359802  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:24.304448  3078 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:01:24.360133  2373 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 202) took 1463 ms (client timeout 1504 ms). Trace:
W20260709 13:01:24.360729  2373 rpcz_store.cc:269] 0709 13:01:22.897285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.899699 (+  2414us) service_pool.cc:224] Handling call
0709 13:01:24.333272 (+1433573us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:24.360061 (+ 26789us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43802496}
I20260709 13:01:24.361248  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:24.362051  1718 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:24.362556  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 204) took 1362 ms. Trace:
I20260709 13:01:24.363173  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:24.363106  2379 rpcz_store.cc:276] 0709 13:01:22.999755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.080819 (+ 81064us) service_pool.cc:224] Handling call
0709 13:01:23.124992 (+ 44173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.136951 (+ 11959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.136983 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.138334 (+  1351us) log.cc:844] Serialized 564 byte log entry
0709 13:01:23.188363 (+ 50029us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:23.188424 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.188512 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.329157 (+1140645us) spinlock_profiling.cc:243] Waited 937 us on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:24.362089 (+ 32932us) raft_consensus.cc:1685] finished
0709 13:01:24.362115 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.362366 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155392,"thread_start_us":49493,"threads_started":1,"wal-append.queue_time_us":59457}
W20260709 13:01:24.353708  1716 rpcz_store.cc:269] 0709 13:01:20.832215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.835044 (+  2829us) service_pool.cc:224] Handling call
0709 13:01:24.172355 (+3337311us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54003dd1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:24.178525 (+  6170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3268392832}
I20260709 13:01:24.252235  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:24.336879  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.367084  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.338158  2845 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.339147  3195 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:24.341545  3230 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.342197  3134 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.343835  3017 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:24.329525  3213 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.186367  3141 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:24.251281  3256 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.251362  3253 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:24.182765  3257 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.354075  3210 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.368441  3246 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:24.186596  3237 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.375166  2378 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:24.366133  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:24.375656  2845 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:24.355722  3071 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:01:24.390445  3182 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:24.379419  3134 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:24.390604  2845 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.346001  1571 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:24.443439  3210 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:24.344087  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:24.445354  1718 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.445655  2372 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.446753  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:24.398795  3230 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:24.328130  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:24.346364  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.452504  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 76178.1us
I20260709 13:01:24.449712  3134 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.453449  1575 raft_consensus.cc:2368] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3.
I20260709 13:01:24.454414  3134 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.455098  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:24.457145  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:24.434733  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.458612  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:24.459252  1575 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:24.459465  1644 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:24.435137  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 254742us
I20260709 13:01:24.460238  1571 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:24.358064  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:24.219260  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.464591  1649 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:24.466213  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.353627  1648 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:24.437847  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 92) took 1312 ms. Trace:
I20260709 13:01:24.382858  3213 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:24.403498  3268 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:01:24.393692  3071 consensus_queue.cc:1243] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:24.447257  3257 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.387538  3246 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.476612  3246 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:24.329888  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1496) took 1774 ms. Trace:
I20260709 13:01:24.476774  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:24.477552  1574 rpcz_store.cc:276] 0709 13:01:22.555721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.681356 (+1125635us) service_pool.cc:224] Handling call
0709 13:01:23.700554 (+ 19198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.700850 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.700878 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.720991 (+ 20113us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.084098 (+363107us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:24.084139 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.084224 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.329681 (+245457us) raft_consensus.cc:1685] finished
0709 13:01:24.329700 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.329821 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6798,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8759,"thread_start_us":361907,"threads_started":1,"wal-append.queue_time_us":427629}
I20260709 13:01:24.380827  3260 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:24.479997  3210 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.353825  3258 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 13:01:24.244395  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.479166  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.467743  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.482851  3246 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:24.485358  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.485970  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.486547  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.480667  1721 rpcz_store.cc:276] 0709 13:01:23.125526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.186925 (+1061399us) service_pool.cc:224] Handling call
0709 13:01:24.242079 (+ 55154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.242359 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.242398 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:24.242993 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.378404 (+135411us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:24.378437 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.378508 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.437066 (+ 58558us) raft_consensus.cc:1685] finished
0709 13:01:24.437134 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.437511 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":95138,"threads_started":1,"wal-append.queue_time_us":126326}
I20260709 13:01:24.487608  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1479) took 2004 ms. Trace:
I20260709 13:01:24.488113  1568 rpcz_store.cc:276] 0709 13:01:22.483107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.319534 (+836427us) service_pool.cc:224] Handling call
0709 13:01:23.478372 (+158838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.598479 (+120107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.487156 (+888677us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400125cd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:24.487400 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281399040}
I20260709 13:01:24.494019  1575 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.496155  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 886) took 1901 ms. Trace:
I20260709 13:01:24.518812  1575 rpcz_store.cc:276] 0709 13:01:22.594650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.488261 (+1893611us) service_pool.cc:224] Handling call
0709 13:01:24.488807 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.495489 (+  6682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.496065 (+   576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.520759  1575 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:24.527269  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 892) took 1316 ms. Trace:
I20260709 13:01:24.501907  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 847) took 2001 ms. Trace:
I20260709 13:01:24.170478  1724 rpcz_store.cc:276] 0709 13:01:22.736988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.114984 (+1377996us) service_pool.cc:224] Handling call
0709 13:01:24.115224 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.168214 (+ 52990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.168244 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.168291 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:24.168319 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.168397 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.168529 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52552,"raft.queue_time_us":53707}
I20260709 13:01:24.528656  1650 rpcz_store.cc:276] 0709 13:01:22.499943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.509924 (+1009981us) service_pool.cc:224] Handling call
0709 13:01:23.510196 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.577188 (+ 66992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.501360 (+924172us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b5400411ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.501704 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83989760}
I20260709 13:01:24.530275  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.531122  1650 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:24.503515  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 854) took 2001 ms. Trace:
I20260709 13:01:24.532694  1651 rpcz_store.cc:276] 0709 13:01:22.502095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.567929 (+1065834us) service_pool.cc:224] Handling call
0709 13:01:23.580809 (+ 12880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.840139 (+259330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.502984 (+662845us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54000910d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.503333 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594872704}
I20260709 13:01:24.534559  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.536245  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.538033  2956 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:24.538944  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.540602  3062 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.541217  1575 rpcz_store.cc:276] 0709 13:01:23.210633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.519860 (+1309227us) service_pool.cc:224] Handling call
0709 13:01:24.520459 (+   599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.526986 (+  6527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.527206 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.509989  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1487) took 2388 ms. Trace:
I20260709 13:01:24.510387  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 891) took 1310 ms. Trace:
W20260709 13:01:24.510809  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.548580  1567 rpcz_store.cc:276] 0709 13:01:23.199663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.499845 (+1300182us) service_pool.cc:224] Handling call
0709 13:01:24.500525 (+   680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.509540 (+  9015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.509582 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.509690 (+   108us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:24.509736 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.509855 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.510073 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8102,"raft.queue_time_us":11068,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":155}
I20260709 13:01:24.550886  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:24.513702  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.510073  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:24.553069  1646 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:24.553789  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44117.7us
W20260709 13:01:24.553908  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 841) took 2066 ms (client timeout 1673 ms). Trace:
I20260709 13:01:24.554726  3078 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:24.514719  3160 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:24.516587  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:24.481099  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:24.495030  1648 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:24.522284  1644 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:24.554466  1646 rpcz_store.cc:269] 0709 13:01:22.487005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.489032 (+  2027us) service_pool.cc:224] Handling call
0709 13:01:24.513269 (+2024237us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:24.553808 (+ 40539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14007168}
I20260709 13:01:24.523092  1721 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:24.523913  1571 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.560880  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1497) took 2004 ms. Trace:
I20260709 13:01:24.561585  1571 rpcz_store.cc:276] 0709 13:01:22.555992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.461320 (+1905328us) service_pool.cc:224] Handling call
0709 13:01:24.523572 (+ 62252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.560551 (+ 36979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.560785 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.562975  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:24.560858  1643 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:01:24.518015  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 61110.3us
I20260709 13:01:24.498870  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:24.527491  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1484) took 2447 ms. Trace:
I20260709 13:01:24.537266  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.540657  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.543166  3213 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.544776  1722 rpcz_store.cc:276] 0709 13:01:22.121398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.324476 (+1203078us) service_pool.cc:224] Handling call
0709 13:01:23.325110 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.632078 (+306968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.508947 (+876869us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.509730 (+   783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639529344}
I20260709 13:01:24.545962  3140 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.547475  1575 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.549661  3196 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:24.486020  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 858) took 1982 ms. Trace:
I20260709 13:01:24.504715  3210 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:24.485982  3260 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.513782  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.569455  1717 rpcz_store.cc:276] 0709 13:01:22.079909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.450446 (+370537us) service_pool.cc:224] Handling call
0709 13:01:24.174388 (+1723942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.175823 (+  1435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.527108 (+351285us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54003dd1d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:24.527334 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3854235904}
I20260709 13:01:24.572721  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.584100  1646 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:24.585526  2901 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.585784  1646 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.586145  2901 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:24.587014  3141 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:24.588466  3141 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.588726  3239 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.589891  3141 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.588639  2378 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:24.590929  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1634) took 2111 ms. Trace:
I20260709 13:01:24.591480  1723 rpcz_store.cc:276] 0709 13:01:22.479312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.667062 (+1187750us) service_pool.cc:224] Handling call
0709 13:01:23.667282 (+   220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.009040 (+341758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.590291 (+581251us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400088754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.590701 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783811328}
I20260709 13:01:24.573649  1650 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:24.577019  3181 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.601233  3210 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.578596  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1477) took 2631 ms. Trace:
I20260709 13:01:24.578922  3204 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:24.580375  1644 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:24.581851  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102533us
W20260709 13:01:24.582502  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 83934.6us
I20260709 13:01:24.582916  3165 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:24.586267  1646 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:24.610824  3165 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:24.611197  3239 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:24.611037  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:24.611371  1646 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.612787  1646 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.594267  2835 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:24.613354  2835 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.566800  1648 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:01:24.614766  2835 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.615466  3237 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:24.592895  3193 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.571756  3140 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.607079  3204 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:24.616541  3193 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.617669  3140 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.617946  3204 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.618042  3193 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.619294  3140 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.619894  3204 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.599918  3147 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.620589  3147 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.576013  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:24.597208  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:24.608028  1616 leader_election.cc:400] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3.
I20260709 13:01:24.606448  1719 rpcz_store.cc:276] 0709 13:01:21.946719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.413563 (+466844us) service_pool.cc:224] Handling call
0709 13:01:22.475405 (+ 61842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.280385 (+804980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.577905 (+1297520us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b5400214154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.578348 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1728448384}
I20260709 13:01:24.612360  3165 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.612969  3065 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:24.605566  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 893) took 1250 ms. Trace:
I20260709 13:01:24.616029  2835 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.637202  1575 rpcz_store.cc:276] 0709 13:01:23.325464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.545939 (+1220475us) service_pool.cc:224] Handling call
0709 13:01:24.547186 (+  1247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.573885 (+ 26699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.575392 (+  1507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.616575  1648 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:24.641125  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1498) took 1312 ms. Trace:
I20260709 13:01:24.641475  1648 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 13:01:24.641690  1573 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:24.641620  1575 rpcz_store.cc:276] 0709 13:01:23.328326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.639871 (+1311545us) service_pool.cc:224] Handling call
0709 13:01:24.640363 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.640716 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.640741 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.640815 (+    74us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:24.640866 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.640949 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.641053 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.644023  1575 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 13:01:24.621950  3147 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.627630  1644 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:24.629336  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:24.630829  1572 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:24.631387  1616 leader_election.cc:403] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:24.620271  3193 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.639293  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 84) took 1892 ms. Trace:
I20260709 13:01:24.637081  3065 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.648897  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1499) took 1306 ms. Trace:
I20260709 13:01:24.655385  1575 rpcz_store.cc:276] 0709 13:01:23.342966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.642737 (+1299771us) service_pool.cc:224] Handling call
0709 13:01:24.643493 (+   756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.648423 (+  4930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.648772 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.652513  1651 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.587801  2901 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.651304  1645 rpcz_store.cc:276] 0709 13:01:22.739320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.693181 (+953861us) service_pool.cc:224] Handling call
0709 13:01:23.693569 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.826187 (+132618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.826232 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:23.826288 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:23.826330 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.628076 (+801746us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003fc354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.628138 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.628320 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304471424}
I20260709 13:01:24.653458  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 889) took 1481 ms. Trace:
I20260709 13:01:24.659750  1572 rpcz_store.cc:276] 0709 13:01:23.172456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.629926 (+1457470us) service_pool.cc:224] Handling call
0709 13:01:24.630390 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.651054 (+ 20664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.653379 (+  2325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.658324  3065 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.665266  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.665707  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1501) took 1322 ms. Trace:
I20260709 13:01:24.666129  1572 rpcz_store.cc:276] 0709 13:01:23.343621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.664627 (+1321006us) service_pool.cc:224] Handling call
0709 13:01:24.665026 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.665324 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.665356 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.665429 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.665460 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.665518 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.665634 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.667975  1572 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.668206  1567 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.668552  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1502) took 1324 ms. Trace:
I20260709 13:01:24.669086  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1506) took 1308 ms. Trace:
I20260709 13:01:24.668948  1572 rpcz_store.cc:276] 0709 13:01:23.343913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.667300 (+1323387us) service_pool.cc:224] Handling call
0709 13:01:24.667751 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.668303 (+   552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.668504 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.669675  1567 rpcz_store.cc:276] 0709 13:01:23.360274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.667442 (+1307168us) service_pool.cc:224] Handling call
0709 13:01:24.667973 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.668803 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.669034 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.671079  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1507) took 1310 ms. Trace:
I20260709 13:01:24.671638  1567 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.671509  1572 rpcz_store.cc:276] 0709 13:01:23.360629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.670032 (+1309403us) service_pool.cc:224] Handling call
0709 13:01:24.670430 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.670750 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:24.670777 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.670815 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:24.670841 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.670893 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.671009 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.673084  3065 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.574368  1642 rpcz_store.cc:276] 0709 13:01:22.503264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.626600 (+1123336us) service_pool.cc:224] Handling call
0709 13:01:23.629182 (+  2582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.937013 (+307831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.485426 (+548413us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54001bc6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:24.485817 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700636032}
W20260709 13:01:24.689986  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 82220.5us
I20260709 13:01:24.661365  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 890) took 1465 ms. Trace:
I20260709 13:01:24.695207  1575 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:24.692665  1649 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:24.677834  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.701297  3147 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.703032  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.703864  1642 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:24.704795  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1500) took 1352 ms. Trace:
I20260709 13:01:24.694126  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.705190  1575 rpcz_store.cc:276] 0709 13:01:23.343307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.660575 (+1317268us) service_pool.cc:224] Handling call
0709 13:01:24.695097 (+ 34522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.695575 (+   478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.695705 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.708860  3191 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.709313  3191 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.706514  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:24.708881  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.713011  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.713188  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:24.716231  1642 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.718461  1571 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:24.718868  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.719302  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 896) took 1127 ms. Trace:
W20260709 13:01:24.721004  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:01:24.723875  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.725991  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.726835  1649 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.727845  2377 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:24.713559  3191 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.732331  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:24.696563  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1508) took 1333 ms. Trace:
I20260709 13:01:24.734711  1651 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:24.732388  3191 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:24.736905  3222 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3.
I20260709 13:01:24.734367  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 286) took 1104 ms. Trace:
I20260709 13:01:24.740006  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:24.740831  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.740262  2374 rpcz_store.cc:276] 0709 13:01:23.629694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.632230 (+  2536us) service_pool.cc:224] Handling call
0709 13:01:24.731964 (+1099734us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:24.734274 (+  2310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10707840}
I20260709 13:01:24.719831  1571 rpcz_store.cc:276] 0709 13:01:23.591712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.717470 (+1125758us) service_pool.cc:224] Handling call
0709 13:01:24.717940 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.718957 (+  1017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.719231 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.745179  3237 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:24.746220  1571 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 3. Ops: []
I20260709 13:01:24.694767  1568 rpcz_store.cc:276] 0709 13:01:23.192032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.494010 (+1301978us) service_pool.cc:224] Handling call
0709 13:01:24.494613 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.495116 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:24.495683 (+   567us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.625006 (+129323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.631495 (+  6489us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:24.638669 (+  7174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.648195 (+  9526us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.654855 (+  6660us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":242,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":2843,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":141,"thread_start_us":99972,"threads_started":1,"wal-append.queue_time_us":246800}
I20260709 13:01:24.750682  3189 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:24.748785  3000 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:24.751207  2793 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:24.751626  3000 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:24.752270  3000 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.752408  3219 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.752983  3219 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:24.754091  3219 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:24.755633  3219 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.749446  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:24.751278  1568 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:24.759152  1568 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:24.765431  1568 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:24.769822  2344 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:24.770300  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.771229  3237 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:24.771273  1644 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:24.773769  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:24.778532  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:24.777683  2793 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:24.752146  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:24.788082  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 43927.5us
W20260709 13:01:24.793609  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1483) took 2309 ms (client timeout 2094 ms). Trace:
W20260709 13:01:24.794190  1573 rpcz_store.cc:269] 0709 13:01:22.484080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.747580 (+263500us) service_pool.cc:224] Handling call
0709 13:01:24.793276 (+2045696us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54004209d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:24.793491 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4006690048}
I20260709 13:01:24.795679  1573 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:24.796909  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:24.798799  1644 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:01:24.802286  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1482) took 2318 ms (client timeout 2130 ms). Trace:
W20260709 13:01:24.802816  1570 rpcz_store.cc:269] 0709 13:01:22.483832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.903836 (+420004us) service_pool.cc:224] Handling call
0709 13:01:24.801847 (+1898011us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54001a9ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:24.802110 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919357696}
I20260709 13:01:24.804504  1570 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.819736  1572 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.840394  1646 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.835711  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1510) took 1473 ms. Trace:
I20260709 13:01:24.843093  1572 rpcz_store.cc:276] 0709 13:01:23.362575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.688294 (+1325719us) service_pool.cc:224] Handling call
0709 13:01:24.735924 (+ 47630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.835400 (+ 99476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.835632 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.845331  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:24.846329  2374 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:24.846395  3000 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:24.835310  3213 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:24.735641  1567 rpcz_store.cc:276] 0709 13:01:23.360940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.670870 (+1309930us) service_pool.cc:224] Handling call
0709 13:01:24.671344 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.685516 (+ 14172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.694426 (+  8910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.835510  3189 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:01:24.839335  1571 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.853165  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:24.853351  1567 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.855486  1567 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.855912  1571 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:24.859179  3237 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:24.861882  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.862429  3160 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:24.862893  3160 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:24.866694  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 147153us
I20260709 13:01:24.869623  1649 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:24.872870  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.872910  3065 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:24.874567  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 895) took 1495 ms. Trace:
I20260709 13:01:24.875298  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:24.876559  3109 raft_consensus.cc:2764] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:24.874977  1575 rpcz_store.cc:276] 0709 13:01:23.378906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.707293 (+1328387us) service_pool.cc:224] Handling call
0709 13:01:24.725606 (+ 18313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.728062 (+  2456us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:24.744403 (+ 16341us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.871408 (+127005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.871469 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.871519 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.871649 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.874511 (+  2862us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15794,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":73,"thread_start_us":42740,"threads_started":1}
I20260709 13:01:24.878144  1574 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:24.878851  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 888) took 1973 ms. Trace:
W20260709 13:01:24.879359  3283 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.150s	user 0.000s	sys 0.002s
I20260709 13:01:24.879190  1574 rpcz_store.cc:276] 0709 13:01:22.905159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.808018 (+1902859us) service_pool.cc:224] Handling call
0709 13:01:24.877939 (+ 69921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.878587 (+   648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.878788 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:24.889219  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.863903  3160 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:24.901052  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.885895  3062 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:24.903584  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 133972us
W20260709 13:01:24.913498  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 161463us
I20260709 13:01:24.902259  3160 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:24.920152  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45013.4us
I20260709 13:01:24.904058  3189 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.925539  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:24.935222  1645 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:24.902182  3099 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:01:24.948503  3008 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.949415  3008 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:24.952103  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:24.924068  3260 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:01:24.956936  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:25.001591  3228 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.778s	user 0.009s	sys 0.000s
W20260709 13:01:25.034935  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.036413  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 78476.5us
I20260709 13:01:24.958540  3204 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:24.998598  1646 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:25.003958  3193 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:25.037442  3000 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.053313  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:25.044895  2956 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:25.064262  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 139618us
I20260709 13:01:25.043531  2835 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:01:25.066747  3213 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.067013  3141 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.065825  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:25.055317  3193 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:25.067576  3141 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:25.071224  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:25.074386  1573 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:25.090747  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48961.6us
I20260709 13:01:25.086172  3204 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.094156  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1488) took 2607 ms. Trace:
I20260709 13:01:25.094620  1576 rpcz_store.cc:276] 0709 13:01:22.485695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.504739 (+1019044us) service_pool.cc:224] Handling call
0709 13:01:23.508444 (+  3705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.595017 (+ 86573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.059445 (+1464428us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b54000c2954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:25.092425 (+ 32980us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105433344}
I20260709 13:01:25.097078  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1492) took 2428 ms. Trace:
I20260709 13:01:25.097043  2835 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:25.097711  2835 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:25.097625  1720 rpcz_store.cc:276] 0709 13:01:22.668385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.860194 (+1191809us) service_pool.cc:224] Handling call
0709 13:01:23.860607 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.861030 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.861070 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.861572 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:01:24.883367 (+1021795us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b1800262900. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:01:24.958067 (+ 74700us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:24.959331 (+  1264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.000610 (+ 41279us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.096089 (+ 95479us) raft_consensus.cc:1685] finished
0709 13:01:25.096110 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.096398 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6768,"spinlock_wait_cycles":3082496,"thread_start_us":62753,"threads_started":1,"wal-append.queue_time_us":1227509}
I20260709 13:01:25.097654  3193 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.100404  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:25.100591  2835 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.101507  3141 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.103340  2344 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:25.106156  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 160051us
W20260709 13:01:25.108219  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65460us
I20260709 13:01:25.123488  3190 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:25.120323  3238 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.121459  3140 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:25.124910  3140 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:25.126025  3140 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.124379  3238 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:25.127345  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:25.128599  3099 raft_consensus.cc:2764] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:25.128085  3238 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.129572  3238 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.136833  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.137108  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.137789  1720 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:25.139134  1716 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.142711  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:25.143827  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.144829  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.145498  3242 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.145845  1719 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.149307  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:25.149596  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 294) took 1483 ms. Trace:
I20260709 13:01:25.150322  3242 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:25.150995  3242 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:25.151450  3242 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.150151  2372 rpcz_store.cc:276] 0709 13:01:23.666175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.023173 (+356998us) service_pool.cc:224] Handling call
0709 13:01:24.023580 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.446251 (+422671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.149195 (+702944us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540064f354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:25.149473 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969116544}
I20260709 13:01:25.161554  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.157820  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.168826  3242 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:25.169970  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49355.4us
I20260709 13:01:25.172982  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:25.171581  3242 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:25.172101  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.174783  2845 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.176261  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:25.170697  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67703.6us
I20260709 13:01:25.186074  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:25.186678  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45718.6us
I20260709 13:01:25.152866  3141 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.155272  3193 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:25.189569  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.190442  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.196039  1646 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:25.196144  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.197661  3099 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.199163  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:25.201356  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.201508  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.198654  3242 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:25.203238  3242 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.203770  3242 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.204312  1569 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.187178  3204 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:25.208636  1720 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:25.212985  3190 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:25.213722  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.213402  1716 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.215806  1576 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.218444  1568 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:25.219058  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.219732  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.220536  1571 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:24.114252  1725 rpcz_store.cc:276] 0709 13:01:22.667711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.856163 (+1188452us) service_pool.cc:224] Handling call
0709 13:01:23.869439 (+ 13276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.869632 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:24.043650 (+174018us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.110493 (+ 66843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.110549 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:24.110595 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.110664 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.110802 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":247659,"raft.queue_time_us":121,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":2352,"thread_start_us":234251,"threads_started":2}
I20260709 13:01:25.200832  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:25.207347  2901 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:25.222810  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1598) took 3103 ms (client timeout 3288 ms). Trace:
W20260709 13:01:25.223240  1643 rpcz_store.cc:269] 0709 13:01:22.119391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.184628 (+ 65237us) service_pool.cc:224] Handling call
0709 13:01:23.565584 (+1380956us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:25.211801 (+1646217us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54000906d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:25.222739 (+ 10938us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257190400}
I20260709 13:01:25.225435  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:25.227617  1573 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.205353  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 887) took 2436 ms. Trace:
W20260709 13:01:25.231293  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58474.7us
I20260709 13:01:25.207095  1719 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:25.231297  1569 rpcz_store.cc:276] 0709 13:01:22.769534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.203403 (+2433869us) service_pool.cc:224] Handling call
0709 13:01:25.204029 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.204984 (+   955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.205289 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:25.230197  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:25.235826  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:25.236289  2956 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:25.236771  1575 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:25.237792  3156 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:25.238140  2956 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:25.238384  3156 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.238624  3144 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.239010  3144 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.239367  2956 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.239822  2956 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.240273  3144 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.241815  3144 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.240450  3156 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.239153  3109 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.243618  3180 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.243976  3180 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.244691  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:25.245020  3180 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.245702  1568 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:25.246423  3180 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.249023  2835 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.251236  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:25.251507  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:25.252238  3281 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.527s	user 0.003s	sys 0.005s
W20260709 13:01:25.250068  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.254108  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:25.256845  3242 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.257205  3118 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:25.258528  3279 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.529s	user 0.001s	sys 0.003s
I20260709 13:01:25.260638  3106 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.261394  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.250118  3190 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:25.278584  3023 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.279814  3109 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:25.280676  3023 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.281317  3188 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.281785  3188 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.282080  3118 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:25.282199  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 72124us
I20260709 13:01:25.282649  3118 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:25.277400  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.283044  3188 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.281064  3229 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.284312  3193 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.284602  2378 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.285342  3229 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.285778  3229 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.286902  3229 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.271495  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:25.288192  3160 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.288455  3000 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.289134  3160 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.289611  3188 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.290194  3000 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:25.290540  3106 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.291750  3189 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:25.292847  3106 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.292838  3062 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.293457  3062 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:25.293991  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 871) took 2515 ms. Trace:
I20260709 13:01:25.294435  3147 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.294485  3062 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:25.274011  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.279717  3173 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:25.283804  3288 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:01:25.280501  3221 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:25.300973  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.283829  3118 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.302042  3118 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.296454  3229 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.296145  3164 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:01:25.297286  3005 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:01:25.304374  3289 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.379s	user 0.002s	sys 0.000s
I20260709 13:01:25.294369  1647 rpcz_store.cc:276] 0709 13:01:22.778229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.708109 (+929880us) service_pool.cc:224] Handling call
0709 13:01:23.708449 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.855773 (+147324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.293549 (+1437776us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400133b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:25.293813 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309833984}
I20260709 13:01:25.281870  3008 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.305336  3000 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.306475  3109 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.300987  3221 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:25.306700  3000 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.305667  3005 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:25.309635  3005 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.310007  3222 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:25.311028  3165 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.311452  3165 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.312506  3147 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:25.312999  3199 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.314160  1645 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:25.315168  3242 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.308465  2374 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.312070  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:25.299002  1650 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.316097  3165 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.316738  2956 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.319669  3195 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:25.320214  2956 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.320544  3195 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:25.321228  3195 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:25.322098  3195 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.320632  3165 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.323365  3195 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.266940  1573 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:25.325636  3286 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 13:01:25.302433  3173 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.306419  3008 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.329015  3222 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:25.329362  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.296113  3062 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.329773  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.330741  1473 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 3, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.332206  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:25.316174  1648 raft_consensus.cc:2368] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3.
I20260709 13:01:25.339854  1725 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:25.326843  3173 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.350687  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.351140  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:25.352942  3173 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.353788  2793 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.352945  1717 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.317579  1719 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:01:25.354462  2793 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.354487  3173 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.369714   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:25.367681  1650 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:25.336393  3221 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.373231  3000 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:25.373984  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46711.5us
I20260709 13:01:25.351063  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:25.367331  2374 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:25.380438  3242 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.381381  1717 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:25.364537  1719 pending_rounds.cc:85] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 1
I20260709 13:01:25.384859  1722 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.386049  1716 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:25.368125  2371 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:25.402053  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.400799  1473 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.405915  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:25.383663  2793 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.408054  3000 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:25.413554  1720 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:25.421473  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 70471.9us
W20260709 13:01:25.433341  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.433854  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.404598  3165 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:01:25.439337  1573 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.400787  3238 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:01:25.411669  1719 pending_rounds.cc:107] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:25.444111  1719 raft_consensus.cc:2889] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.451639  1722 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:25.458237  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.431967  3106 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.439895  3156 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:25.434777  2845 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.445008  3000 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.462191  1725 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:25.468240  1651 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:25.450337  3188 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.473027  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:25.476823  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.477517  3160 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.478137  3160 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:25.479781  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 228660us
I20260709 13:01:25.480960  2793 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:25.481678  2793 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.483263  3221 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.483922  2793 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.484980  1648 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:25.481063  1647 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:25.494848  1645 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:25.498158  1647 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:25.506436  1472 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 202) took 6682 ms. Trace:
I20260709 13:01:25.511790  3141 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.512504  3199 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.513312  3109 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.515156  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:01:25.515777  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47150.5us
I20260709 13:01:25.516476  2835 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.517477  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:25.518321  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:25.517478  1651 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.519217  1650 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.496556  3000 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.488055  3180 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.543496  1688 leader_election.cc:400] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3.
I20260709 13:01:25.569679  1688 leader_election.cc:403] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:25.544623  3297 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 13:01:25.545060  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.545042  1649 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 13:01:25.555855  1473 catalog_manager.cc:5697] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.470510  3106 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.505378  3023 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.496403  3188 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.527616  3156 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.545169  3099 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:25.572389  1649 pending_rounds.cc:85] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 3
I20260709 13:01:25.590195  1649 pending_rounds.cc:107] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
W20260709 13:01:25.572424  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 124154us
I20260709 13:01:25.547281  3160 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.499061  2845 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:25.588176  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44878.9us
I20260709 13:01:25.557904  3260 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.591931  1473 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 3, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.600253  1649 raft_consensus.cc:2917] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 13:01:25.606278  1649 raft_consensus.cc:2889] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:25.608549  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.589103  2956 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.557632  3141 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:25.618294  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 103227us
I20260709 13:01:25.588116  3062 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.593845  3147 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:25.623657  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:25.606806  2901 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.583634  3180 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.506865  1472 rpcz_store.cc:276] 0709 13:01:18.823726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.823981 (+   255us) service_pool.cc:224] Handling call
0709 13:01:25.506194 (+6682213us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:22.046975 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:22.185828 (+138853us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:22.187751 (+  1923us) write_op.cc:435] Acquired schema lock
0709 13:01:22.190524 (+  2773us) tablet.cc:662] Decoding operations
0709 13:01:22.311781 (+121257us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:22.311838 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:22.311865 (+    27us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:01:22.312656 (+   791us) tablet.cc:701] Row locks acquired
0709 13:01:22.312675 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:01:22.312819 (+   144us) write_op.cc:270] Start()
0709 13:01:22.312931 (+   112us) write_op.cc:276] Timestamp: P: 1783602082312773 usec, L: 0
0709 13:01:22.312954 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:01:22.313604 (+   650us) log.cc:844] Serialized 4145 byte log entry
0709 13:01:23.136670 (+823066us) op_driver.cc:464] REPLICATION: finished
0709 13:01:23.255261 (+118591us) write_op.cc:301] APPLY: starting
0709 13:01:23.257856 (+  2595us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:23.304958 (+ 47102us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:23.342422 (+ 37464us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:23.349497 (+  7075us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:23.349723 (+   226us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:01:23.349768 (+    45us) write_op.cc:312] APPLY: finished
0709 13:01:23.399330 (+ 49562us) log.cc:844] Serialized 77 byte log entry
0709 13:01:23.414198 (+ 14868us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:23.431433 (+ 17235us) write_op.cc:454] Released schema lock
0709 13:01:23.469441 (+ 38008us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:23.521070 (+ 51629us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:23.540280 (+ 19210us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21839,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":87422,"apply.run_cpu_time_us":4171,"apply.run_wall_time_us":271303,"mutex_wait_us":6775,"num_ops":10,"prepare.queue_time_us":271479,"prepare.run_cpu_time_us":26641,"prepare.run_wall_time_us":692317,"raft.queue_time_us":165670,"raft.run_cpu_time_us":14276,"raft.run_wall_time_us":120234,"replication_time_us":823576,"thread_start_us":616113,"threads_started":4,"wal-append.queue_time_us":654075,"wal-append.run_cpu_time_us":13153,"wal-append.run_wall_time_us":1492564}]]}
I20260709 13:01:25.628021  3191 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:01:25.631464  3140 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.631671  3301 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:01:25.638458  3160 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:25.641181  2845 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.641796  2845 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:25.642572  2845 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:25.642894  2845 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:25.645917  1645 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.599679  3099 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.649128  3099 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.656759  2380 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.590888  3023 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.669420  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:25.617519  1646 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:25.671797  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:25.681437  1643 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:25.656533  3173 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.622516  3260 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.704177  2845 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:25.705035  3304 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:25.705781  2845 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.706172  3305 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:01:25.709595  3000 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.710474  3156 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.712745  3141 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.713333  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:25.716825  3144 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:25.717242  3144 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:25.718034  3144 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.719183  3144 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.719017  3229 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:25.656734  3204 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:01:25.720466  3008 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:25.721081  3204 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.721560  3204 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.721928  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:25.722270  3008 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:25.722853  1572 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:25.679858  3118 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:01:25.723961  3204 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.661792  3219 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:01:25.681963  2793 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:25.639195  3147 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.685814  2956 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.723006  3008 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.724315  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:25.725069  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:25.733002  1725 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:25.725679  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43900.4us
W20260709 13:01:25.758585  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 80280.5us
W20260709 13:01:25.739547  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 110251us
I20260709 13:01:25.733968  1721 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:25.761337  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:25.742326  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.763471  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:25.669941  2901 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.766929  1720 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:25.726648  3299 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:01:25.762053  1643 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:25.730772  3173 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:25.775377  3173 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.775966  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:25.738402  3191 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.778138  3191 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.741387  3165 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.725497  3023 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.762887  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.734392  3189 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.726321  3188 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.752431  3204 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:25.718043  3180 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.726285  3311 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.729687  2374 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:25.781579  1722 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:25.730060  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:25.770578  1716 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:25.799158  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.802695  2380 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 13:01:25.803298  2380 pending_rounds.cc:85] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 3
I20260709 13:01:25.803614  2380 pending_rounds.cc:107] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 13:01:25.805234  2380 raft_consensus.cc:2917] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 13:01:25.780894  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:25.780313  3191 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.808880  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:25.805711  2380 raft_consensus.cc:2889] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.809213  3188 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:01:25.780669  3165 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.813831  3165 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.796573  2956 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.790107  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.794031  3204 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:25.822636  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:25.781934  2793 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.807695  3000 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:25.833395  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:25.833715  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.828861  3213 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:25.834339  1721 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:25.829218  3229 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:01:25.834630  3213 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:25.835150  3213 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.823956  1719 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:25.836663  3147 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.836741  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.837299  3000 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.837492  1718 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:25.837848  3000 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:25.829566  3106 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.837956  3260 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.850291  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.850565  1719 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.852655  3147 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:25.853792  2793 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:25.859859  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:25.864591  3312 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:25.864905  1718 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:01:25.864830  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:25.867699  1649 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:25.868891  1651 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:25.803539  3189 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.863181  2372 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:25.818784  2956 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.884213  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:25.821719  2835 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:01:25.865057  3238 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.887588  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:25.824283  3219 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:01:25.816654  3311 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.868695  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:25.897531  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:25.858235  3188 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:01:25.901253  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:25.929316  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.947847  3306 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:25.936830  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.932178  3005 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.926681  3310 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.893519  3312 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:25.928862  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:25.926232  3165 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:25.890774  3260 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.894610  3301 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.977645  3260 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:25.977974  3301 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.979984  3301 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.948818  2901 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:25.935227  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:25.927166  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.980909  1649 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:25.981918  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.983244  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:25.959506  1720 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:25.984452  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:25.985288  1719 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:25.965521  1570 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:25.925578  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:25.985160  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.045678  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:25.973023  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 177521us
I20260709 13:01:25.964862  1722 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:26.042073  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.061954  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:26.063694  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:26.063864  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:26.067497  1570 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:26.067839  3008 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.068588  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.069203  1573 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:26.070102  3008 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.074059  1568 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:26.075133  3311 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.077956  3314 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:26.078367  3314 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:26.078963  1573 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:01:26.079371  1573 pending_rounds.cc:85] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 2
I20260709 13:01:26.079640  1573 pending_rounds.cc:107] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:01:26.079178  3314 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.079871  1573 raft_consensus.cc:2889] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.082866  1573 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:01:26.082969  3000 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.083709  1573 pending_rounds.cc:85] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:01:26.084260  1573 pending_rounds.cc:107] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:01:26.084690  1573 raft_consensus.cc:2889] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.085935  2956 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.087597  1573 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:26.089958  3189 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.090667  1644 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:26.098172  3260 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.967003  3306 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.099027  3306 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.099792  3222 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:26.100296  3222 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:26.101231  3222 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.084718  2374 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:25.986955  3310 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.105430  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.107108  3310 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.107846  3219 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:26.108529  3219 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.108937  3222 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:26.111039  3219 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:25.973353  3221 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:26.111567  1716 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:26.111982  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 219686us
I20260709 13:01:26.114562  3165 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.986774  3005 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.117825  3005 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:26.115056  3165 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:25.924266  3238 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.120589  3238 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.120177  3221 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.121407  3238 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:26.121930  3238 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.123015  3221 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:26.076051  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.075254  1568 pending_rounds.cc:85] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:01:26.130877  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:26.148978  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:26.133447  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:26.133780  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.151118  1720 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:26.134282  3109 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:26.136029  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.137136  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:26.139225  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.139175  1724 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:01:26.140861  3325 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:26.141732  3165 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.141938  3312 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.132225  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:26.152385  1721 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:26.152135  1724 pending_rounds.cc:85] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 2
I20260709 13:01:26.159186  1724 pending_rounds.cc:107] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:01:26.150388  1568 pending_rounds.cc:107] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:01:26.159588  1724 raft_consensus.cc:2889] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.160220  1717 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:26.148578  3314 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.158444  2901 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:26.170387  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:26.172693  1720 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:26.157125  3144 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:26.178283  2901 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:26.209146  3296 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.728s	user 0.010s	sys 0.002s
I20260709 13:01:26.165822  3109 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.213660  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:26.165539  3199 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:01:26.165832  1568 raft_consensus.cc:2889] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.220671  1717 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:26.216147  3173 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:26.175940  3144 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.281227  3144 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.177469  2374 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:26.165896  1644 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:26.163753  3325 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.287532  3180 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:26.288373  3325 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:26.300163  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.264127  3221 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.272794  3238 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.323947  3000 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.350055  3204 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.351598  3204 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.353530  3331 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:26.354077  3331 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.354487  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:26.355775  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:26.356328  3331 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.248853  3306 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.356940  3005 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.357434  3005 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.358330  3306 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:26.306674  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.359278  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:26.359478  3005 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.360021  3331 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.360365  2371 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:26.303871  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:26.317528  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.318361  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.319097  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.320655  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.321498  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.322754  3195 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:26.379762  1717 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:26.328840  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 215) took 1016 ms. Trace:
I20260709 13:01:26.176880  3213 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.240798  3109 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.388743  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.389148  3109 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.390396  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.391112  1572 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:26.321555  1574 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:26.393296  1574 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:26.366101  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.288139  3180 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.370399  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.279287  1719 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:26.371089  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.377451  1650 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 2: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:26.377492  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.379762  1723 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:26.395038  1573 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:01:26.243713  3062 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.000s
I20260709 13:01:26.383219  3195 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:26.383652  3141 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.383405  3099 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:26.384533  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.386133  3213 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.386329  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:26.386904  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 83170.7us
I20260709 13:01:26.387384  3306 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:26.387519  2835 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.387890  3238 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.380379  2380 rpcz_store.cc:276] 0709 13:01:25.312032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.315598 (+  3566us) service_pool.cc:224] Handling call
0709 13:01:25.656625 (+341027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.811960 (+155335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.328629 (+516669us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b540057e4d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:26.328777 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783405440}
I20260709 13:01:26.393198  2956 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.392076  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.397600  1576 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:01:26.408735  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.409309  1573 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:26.441408  1574 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.457993  1572 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:26.396744  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 183184us
I20260709 13:01:26.422906  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:26.458621  1574 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.331784  3221 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.430455  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:26.437906  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.451325  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:26.452800  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.395399  1570 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:26.436703  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.459033  1651 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:26.463423  1723 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:26.427459  3189 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.427657  3242 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.459725  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:26.460271  3118 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.460728  3188 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:26.458994  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.463312  1643 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:26.412882  3180 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.476459  1650 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:26.478660  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.474359  1645 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:26.490679  1572 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:26.435709  3062 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3.
I20260709 13:01:26.431056  3141 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.497375  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.497751  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.435783  3000 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:01:26.478087  3219 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.454993  2835 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.481207  3305 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:26.454989  3238 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.510197  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.523300  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:26.523744  1645 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:26.474390  2956 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.511886  1573 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:26.502324  1572 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:26.510277  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.435679  1571 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:26.515488  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.473713  3190 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.518240  3338 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:26.491719  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.484643  3189 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:26.453588  3306 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.511735  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:26.489110  3242 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.495502  3118 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.560236  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:26.564036  1645 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.527136  3062 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:26.530289  1718 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:26.525393  3238 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:26.537882  3330 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:01:26.525156  3305 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:26.525357  3173 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:26.571226  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:26.494917  3195 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.577740  1643 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.561612  3189 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:26.583539  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71881.8us
I20260709 13:01:26.528007  3219 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.567526  3062 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.569134  3005 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:26.605394  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.560312  3164 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:01:26.582996  3099 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:26.584070  3305 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:26.619773  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 174541us
I20260709 13:01:26.622992  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:26.620739  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 192732us
I20260709 13:01:26.615289  3099 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:26.672783  3099 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.673467  3242 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.674506  3221 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:26.675966  3335 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:26.677318  2956 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.679526  3118 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:26.568092  3190 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.601496  3062 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.600894  2901 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:26.689597  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 79050.3us
I20260709 13:01:26.693760  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:26.686264  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63397.7us
I20260709 13:01:26.608214  3195 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:26.686924  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60354.1us
I20260709 13:01:26.694504  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.695400  1574 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:01:26.696091  1574 pending_rounds.cc:85] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 1
W20260709 13:01:26.671018  3308 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:01:26.696564  1574 pending_rounds.cc:107] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:26.696874  1574 raft_consensus.cc:2889] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.670514  3188 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.615898  3141 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.715209  3312 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:26.722518  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 914) took 1290 ms. Trace:
I20260709 13:01:26.722929  1575 rpcz_store.cc:276] 0709 13:01:25.431936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.439526 (+  7590us) service_pool.cc:224] Handling call
0709 13:01:25.456097 (+ 16571us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:25.474890 (+ 18793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.474923 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:25.503906 (+ 28983us) log.cc:844] Serialized 52 byte log entry
0709 13:01:25.589075 (+ 85169us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:25.607353 (+ 18278us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.656909 (+ 49556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.656987 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.331508 (+674521us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b4800002dd8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:01:26.451825 (+120317us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b180022ada0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:26.722279 (+270454us) raft_consensus.cc:1685] finished
0709 13:01:26.722300 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.722432 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93830,"prepare.queue_time_us":17973,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2746,"raft.queue_time_us":93002,"raft.run_cpu_time_us":22603,"raft.run_wall_time_us":712862,"spinlock_wait_cycles":16878208,"thread_start_us":74669,"threads_started":1,"wal-append.queue_time_us":160616}
I20260709 13:01:26.693897  3144 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.726608  3213 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:26.724273  3099 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.758738  3213 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:26.759675  3213 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.760388  3099 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:26.728708  3333 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:01:26.761953  1645 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:26.737464  1650 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:26.697223  3325 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:26.739781  2793 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 13:01:26.765772  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.753648  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:26.765126  2793 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.766932  1717 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:26.755952  1643 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:26.769542  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:26.767243  2793 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:26.777350  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59605.6us
I20260709 13:01:26.777280  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.778115  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:26.778193  1645 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:26.779044  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 258634us
I20260709 13:01:26.713948  3221 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:26.729161  3334 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:01:26.745461  1644 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:26.738478  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:26.739104  2901 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.818853  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:26.749872  3005 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:26.710049  3188 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.745757  3219 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.826395  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:26.730858  3190 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.762355  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 915) took 1330 ms. Trace:
W20260709 13:01:26.839615  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45020.1us
I20260709 13:01:26.757867  3323 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:26.789857  3222 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
I20260709 13:01:26.781559  3213 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.788525  3221 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.791986  3331 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:26.728029  3109 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.846560  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:26.849372  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:26.749403  3305 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.776666  3180 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.706909  1651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:01:26.839759  3314 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.671s	user 0.005s	sys 0.007s
I20260709 13:01:26.835971  3008 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.837993  3312 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.713217  3173 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.776975  3325 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.909426  2793 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:26.910537  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.912416  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63158.6us
I20260709 13:01:26.913885  2956 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.840166  2835 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:26.915180  3342 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:26.915572  3341 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.915967  3341 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.916286  3341 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.916886  3188 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.917735  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:26.907099  3238 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.919334  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:26.919719  3238 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.845513  3190 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:26.920953  3190 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:26.921644  1575 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:26.922268  3190 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.923569  3023 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:26.915079  2956 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:26.839547  3141 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:01:26.924299  3222 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:26.924504  2956 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:26.906445  3008 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.925740  3008 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.925814  2956 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.927184  2956 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.913725  3062 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.924329  3023 raft_consensus.cc:2764] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:01:26.931154  3062 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.850826  3221 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.933544  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.936569  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:26.900280  3326 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:01:26.941833  1573 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:26.946930  1647 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:26.852509  3310 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:26.939819  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.943938  1576 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:26.726145  3144 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.836196  2901 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:26.952052  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:26.957010  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:26.958500  1570 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.955709  3222 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:01:26.961407  3023 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:26.962159  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.963627  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:26.851850  3323 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.963747  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.966151  3323 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:26.966287  3325 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.968896  1570 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:26.932418  3306 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.969528  3189 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:26.970858  1716 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:26.972609  2901 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.974542  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:26.975466  1571 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:26.840957  3005 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:26.978348  3346 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:01:26.979089  3005 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.944684  3238 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:26.980926  3238 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.981467  3238 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:26.986168  3305 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:26.994830  3180 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:26.998010  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.999066  3305 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.999945  3305 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.003612  3222 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:27.004262  3222 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:27.005313  3222 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:27.009516  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58177.7us
I20260709 13:01:27.006858  3222 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.014372  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.015599  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:27.011232  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 316143us
I20260709 13:01:27.039687  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:27.036502  1572 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:27.050401  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:27.055394  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:27.057504  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:27.035128  3339 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:01:27.033449  3195 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.034726  3188 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:27.009865  3348 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.083999  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44428us
I20260709 13:01:27.034114  3305 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:27.010743  3005 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:27.066995  3242 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:27.068714  3144 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:26.959172  3312 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.070717  3180 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.086642  3188 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.076521  3353 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.082088  3118 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.091979  3141 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:27.089221  3310 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:26.840154  1569 rpcz_store.cc:276] 0709 13:01:25.432426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.434939 (+  2513us) service_pool.cc:224] Handling call
0709 13:01:25.751194 (+316255us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:25.781674 (+ 30480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:25.824727 (+ 43053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:25.829065 (+  4338us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.948913 (+119848us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.227874 (+278961us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:26.243360 (+ 15486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.245974 (+  2614us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:26.761981 (+516007us) raft_consensus.cc:1685] finished
0709 13:01:26.762001 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:26.762296 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21725,"raft.queue_time_us":15102,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7230,"spinlock_wait_cycles":9981824,"thread_start_us":272577,"threads_started":1,"wal-append.queue_time_us":517675}
W20260709 13:01:27.124466  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.118280  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.146631  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.015s	sys 0.029s
W20260709 13:01:27.209419  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.015s	sys 0.030s
W20260709 13:01:27.209826  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 721537us
I20260709 13:01:27.101627  3144 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.209942  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:27.212565  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.212849  1570 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.147033  3141 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.086460  3195 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:27.225405  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.109417  3349 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.100811  3000 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.110s
W20260709 13:01:27.221676  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 117292us
I20260709 13:01:27.107846  3242 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:27.230574  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:27.117241  3188 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.125312  3312 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.092002  3348 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.254608  3348 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:27.220431  3213 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:27.237687  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 419056us
I20260709 13:01:27.132792  3062 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:01:27.256155  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:27.255849  3213 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:27.258337  3213 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.264216  3188 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.267650  3213 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.248384  1570 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:27.147795  3353 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.270536  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:27.271186  3353 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:27.109598  3005 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.271274  3188 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:27.274819  3005 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:27.275234  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 164297us
I20260709 13:01:27.226060  3141 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.221491  3144 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.277290  3005 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:27.278173  3005 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:27.279263  3005 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.153035  3118 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.224180  3331 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.285418  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54921.1us
W20260709 13:01:27.235039  3204 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.883s	user 0.013s	sys 0.001s
W20260709 13:01:27.290342  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 464048us
I20260709 13:01:27.302141  1649 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:27.141613  3008 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.309551  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:27.310410  1570 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.113445  3180 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:27.313125  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:27.235260  3109 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.315963  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 101) took 1240 ms. Trace:
I20260709 13:01:27.289003  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:27.316516  1649 rpcz_store.cc:276] 0709 13:01:26.075982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.076121 (+   139us) service_pool.cc:224] Handling call
0709 13:01:26.076382 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.304206 (+1227824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.315647 (+ 11441us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003fc354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:27.315883 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2818707712}
I20260709 13:01:27.238559  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 98) took 1387 ms. Trace:
I20260709 13:01:27.308812  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.218914  3310 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.323616  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:27.290036  3023 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.304203  2901 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.328390  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:27.348191  1651 raft_consensus.cc:606] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:27.375030  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.369863  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:27.369735  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46177.1us
I20260709 13:01:27.409070  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:27.377502  1651 raft_consensus.cc:614] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:27.326874  3229 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:27.303130  3331 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.321832  3347 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.410332  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:27.432864  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:27.242588  3349 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.327668  3189 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:27.421192  1570 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:27.488483  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78222.4us
I20260709 13:01:27.322319  3204 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.336580  3062 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.159s
I20260709 13:01:27.483845  2378 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:27.472707  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64408.1us
I20260709 13:01:27.442130  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:27.366262  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:27.326267  3343 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.008s	sys 0.020s
I20260709 13:01:27.517241  1649 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:27.500957  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 86915.1us
I20260709 13:01:27.490067  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:27.366360  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.533634  2793 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:27.535441  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:27.400393  3359 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.379035  3351 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:27.525602  1642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.017s	sys 0.038s
W20260709 13:01:27.511615  2956 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.001s
W20260709 13:01:27.496172  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.536744  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46751.8us
I20260709 13:01:27.616521  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:27.495811  3190 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.618846  3190 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.622802  3190 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:27.548923  3118 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.509794  3310 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.510443  3000 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:27.550462  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64523.3us
I20260709 13:01:27.528318  3188 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:27.625866  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 356444us
I20260709 13:01:27.640828  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:27.534190  3204 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:27.553928  3221 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.004s
I20260709 13:01:27.558353  3144 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.561959  3331 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.566362  3165 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.580621  3352 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.478s	user 0.005s	sys 0.001s
I20260709 13:01:27.581264  3312 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:27.581583  3213 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:27.583091  3195 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.586818  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42234 (request call id 2) took 1092 ms. Trace:
W20260709 13:01:27.528414  3343 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.010s	sys 0.020s
I20260709 13:01:27.445030  3314 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:27.329430  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.684664  3005 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:01:27.685289  1716 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 1.
I20260709 13:01:27.653044  3062 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.442957  3219 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:27.626021  2793 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:27.629329  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:27.692945  1649 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.644837  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:27.593298  1642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.018s	sys 0.038s
I20260709 13:01:27.641561  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:27.591336  3351 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:27.449172  3023 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.542692  1646 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:27.590766  1569 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:27.658897  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.659103  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:27.662631  1651 rpcz_store.cc:276] 0709 13:01:26.502589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.695525 (+192936us) service_pool.cc:224] Handling call
0709 13:01:27.586532 (+891007us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54004114d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d68853 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:27.586708 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033619840}
I20260709 13:01:27.624727  3000 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.444011  3242 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:27.713083  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71679.4us
I20260709 13:01:27.508677  3008 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:27.665256  3341 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.745s	user 0.010s	sys 0.003s
I20260709 13:01:27.656301  3118 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:27.723768  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 926) took 1293 ms. Trace:
I20260709 13:01:27.594974  3359 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:27.703675  2344 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.018s	sys 0.052s
W20260709 13:01:27.710981  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 66274.9us
I20260709 13:01:27.712177  3348 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.640250  3188 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:27.655920  3221 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.713600  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:27.716202  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 57157.2us
I20260709 13:01:27.716670  1576 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:27.654938  3141 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.720767  3366 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:27.721776  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 930) took 1266 ms. Trace:
W20260709 13:01:27.725301  2344 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.018s	sys 0.052s
I20260709 13:01:27.724293  1643 rpcz_store.cc:276] 0709 13:01:26.430027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.774431 (+344404us) service_pool.cc:224] Handling call
0709 13:01:26.838086 (+ 63655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.838552 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.838593 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.855683 (+ 17090us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.347940 (+492257us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:27.348010 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.348133 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.723482 (+375349us) raft_consensus.cc:1685] finished
0709 13:01:27.723504 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.723679 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16075,"raft.queue_time_us":106,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":17932,"thread_start_us":393476,"threads_started":1,"wal-append.queue_time_us":797924}
I20260709 13:01:27.658627  3165 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.322243  1568 rpcz_store.cc:276] 0709 13:01:25.851349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.074721 (+223372us) service_pool.cc:224] Handling call
0709 13:01:26.075046 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.265054 (+190008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.265115 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.265746 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.556905 (+291159us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:26.563893 (+  6988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.568408 (+  4515us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.217432 (+649024us) raft_consensus.cc:1685] finished
0709 13:01:27.223351 (+  5919us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.234121 (+ 10770us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":266784,"threads_started":1,"wal-append.queue_time_us":565426}
I20260709 13:01:27.664860  3213 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.747954  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:27.752213  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 648722us
I20260709 13:01:27.700798  3351 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:27.725296  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 432212us
I20260709 13:01:27.753104  1645 rpcz_store.cc:276] 0709 13:01:26.455807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.831463 (+375656us) service_pool.cc:224] Handling call
0709 13:01:26.858316 (+ 26853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.917081 (+ 58765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.917127 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.917972 (+   845us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.313690 (+395718us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:27.313742 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.313814 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.721468 (+407654us) raft_consensus.cc:1685] finished
0709 13:01:27.721500 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.721699 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58242,"raft.queue_time_us":58763,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":395273,"threads_started":1,"wal-append.queue_time_us":674615}
I20260709 13:01:27.725863  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:27.807716  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81947.6us
I20260709 13:01:27.777411  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.789763  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:27.697149  3219 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.809540  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:27.811285  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.812110  1722 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:27.796098  1649 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:27.713800  3000 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.813092  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1533) took 1683 ms. Trace:
I20260709 13:01:27.817124  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:27.813386  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:27.818192  1643 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:27.818912  2380 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:27.716979  3180 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.752192  3366 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:27.818111  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.726850  3338 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:27.827338  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:27.852226  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 138362us
I20260709 13:01:27.814706  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:27.853315  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:27.854439  1649 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
W20260709 13:01:27.833882  3373 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:27.834370  3317 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.816990  1720 rpcz_store.cc:276] 0709 13:01:26.129627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.227560 (+ 97933us) service_pool.cc:224] Handling call
0709 13:01:26.311848 (+ 84288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.495422 (+183574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.495452 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.500947 (+  5495us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.775150 (+274203us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:26.775207 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.785528 (+ 10321us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003dbdd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:26.785602 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.255254 (+469652us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:27.812863 (+557609us) raft_consensus.cc:1685] finished
0709 13:01:27.812887 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.813026 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90880,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5319,"spinlock_wait_cycles":428719232,"thread_start_us":245497,"threads_started":1,"wal-append.queue_time_us":531450}
I20260709 13:01:27.788228  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.836462  3381 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:27.836962  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:01:27.837299  3375 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:01:27.883471  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:27.837909  3369 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:01:27.811858  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.682s (SENT) [suppressed 15 similar messages]
I20260709 13:01:27.890266  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:27.835290  2901 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:01:27.841770  3180 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:27.841831  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1542) took 1354 ms. Trace:
I20260709 13:01:27.754114  3118 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:27.801602  3008 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:01:27.855742  3366 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:27.835407  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:27.860299  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:27.809057  3164 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:27.923837  1645 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:27.873422  2956 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.816754  3189 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:27.880580  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:27.936828  1649 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:27.884642  1574 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:27.938403  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.939327  1574 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:27.940779  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:27.943226  1570 raft_consensus.cc:2410] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:27.891086  1643 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.891795  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.944519  1722 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:27.726660  3341 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:27.853091  2835 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:27.897603  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:27.899489  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.846601  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 929) took 1395 ms. Trace:
I20260709 13:01:27.873689  3173 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.890640  3238 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.909s	user 0.011s	sys 0.003s
I20260709 13:01:27.858707  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.882709  3193 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:27.966054  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:27.748551  3314 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
W20260709 13:01:27.901446  3222 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.834s	user 0.014s	sys 0.000s
I20260709 13:01:27.972468  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:27.936136  3189 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.904770  3023 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:27.949360  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.019441  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.951956  1650 rpcz_store.cc:276] 0709 13:01:26.451627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.782802 (+331175us) service_pool.cc:224] Handling call
0709 13:01:26.790962 (+  8160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.825576 (+ 34614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.825608 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.839601 (+ 13993us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.321998 (+482397us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:27.322048 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.322120 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.846327 (+524207us) raft_consensus.cc:1685] finished
0709 13:01:27.846355 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.846525 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48939,"raft.queue_time_us":31848,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":10345,"thread_start_us":379210,"threads_started":1,"wal-append.queue_time_us":830698}
I20260709 13:01:27.921423  3065 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.955633  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.025034  1645 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:27.927028  3164 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.940234  2375 raft_consensus.cc:2368] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:27.951434  1569 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:01:27.901371  1721 rpcz_store.cc:276] 0709 13:01:26.487775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.489928 (+  2153us) service_pool.cc:224] Handling call
0709 13:01:26.512852 (+ 22924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.517238 (+  4386us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:27.086052 (+568814us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.541314 (+455262us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.620180 (+ 78866us) log.cc:844] Serialized 501 byte log entry
0709 13:01:27.620360 (+   180us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:27.620417 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.620506 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.841639 (+221133us) raft_consensus.cc:1685] finished
0709 13:01:27.841660 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.841779 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51339,"prepare.queue_time_us":196025,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":435,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":116,"thread_start_us":942704,"threads_started":2,"wal-append.queue_time_us":638917}
I20260709 13:01:27.907474  3348 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
W20260709 13:01:27.935804  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66186.9us
I20260709 13:01:27.995007  1647 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:27.964819  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.996160  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:27.996655  1718 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:27.940917  2956 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:27.997049  3376 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:01:27.987093  3238 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:27.954857  3144 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.052908  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:28.005415  3388 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:01:27.965210  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1538) took 1478 ms. Trace:
I20260709 13:01:28.005839  3118 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.006249  3165 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.064291  1570 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:28.009037  3372 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:01:28.009950  1572 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:28.009759  3365 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.161s
I20260709 13:01:28.011274  3325 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:27.954440  2835 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.882169  3317 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.013191  3213 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:27.952280  3204 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.016549  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:27.947489  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:27.951854  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:28.028616  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.021883  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.090778  2793 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.079259  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.048717  2380 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:28.049405  1717 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.728s	user 0.015s	sys 0.000s
I20260709 13:01:28.027740  3221 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.103360  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.104440  1570 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:27.965050  3173 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.020432  3189 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.005206  3359 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.074229  3144 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.063024  1643 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:28.064883  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.066113  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.121487  1572 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:28.121623  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 309783us
I20260709 13:01:28.030723  3164 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.071226  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.057601  2376 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:28.073524  2835 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.079159  3317 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.080452  2378 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:28.047859  3193 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:28.078977  3238 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.097024  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.104822  3213 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.099267  2793 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.101168  1717 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
W20260709 13:01:28.102128  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 85715.2us
I20260709 13:01:28.105300  3141 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:28.052132  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.901s (SENT)
I20260709 13:01:28.103499  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.151361  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:28.151624  1576 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:28.150733  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.031071  3065 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.152659  3393 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.110890  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.064306  1571 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:28.152436  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.059911  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.154232  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:01:28.153393  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 208704us
I20260709 13:01:28.156406  1722 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:28.157346  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:28.157451  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.158190  3008 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.158727  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.158902  1571 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:28.159384  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:28.159857  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.080947  2901 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.160131  3008 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.160606  2901 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.161820  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.162607  1643 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:28.162935  2901 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.163309  3008 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:28.126010  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.164427  1569 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:28.129527  1567 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.037186  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.165444  3008 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:28.166488  1574 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.103735  3222 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:28.166887  3393 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.060946  3062 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.167301  3222 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.168691  1574 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:28.168828  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.169740  3222 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.155131  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.174029  1722 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.084329  3325 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.172869  1650 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:28.118148  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.179190  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:28.176146  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.181260  1576 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:01:28.201881  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.294054  1649 raft_consensus.cc:2368] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:28.292021  1645 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:28.278187  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:28.294356  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 115243us
W20260709 13:01:28.206408  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 154316us
I20260709 13:01:28.292285  1572 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:28.298238  1723 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:28.178123  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:28.213397  2793 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:28.300129  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 122464us
I20260709 13:01:28.183475  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.251173  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.305050  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.306316  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.305766  1570 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:28.307096  1649 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:28.149677  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 115247us
I20260709 13:01:28.294384  1576 pending_rounds.cc:85] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:01:28.308825  1576 pending_rounds.cc:107] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:01:28.309163  1576 raft_consensus.cc:2889] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:28.308938  1722 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:28.301160  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:28.313827  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.314519  2793 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:28.315243  2793 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.170751  3008 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:28.316579  2793 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.317401  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.154033  3065 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.208866  1716 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:28.174882  3390 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:28.275241  3173 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.276059  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.272595  3341 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.138664  3144 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.206157  3190 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.314663  1723 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:28.263279  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 935) took 1322 ms. Trace:
I20260709 13:01:28.312202  3221 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:28.312953  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.362241  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.447852  1649 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:28.448560  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 109) took 1132 ms. Trace:
I20260709 13:01:28.371232  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:28.351980  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 66295.3us
I20260709 13:01:28.449205  1645 rpcz_store.cc:276] 0709 13:01:27.316016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.312004 (+995988us) service_pool.cc:224] Handling call
0709 13:01:28.448441 (+136437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.449864  1576 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:28.449440  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.450228  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:28.336515  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.451290  1649 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:28.451336  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:28.411417  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62497.8us
I20260709 13:01:28.217947  3213 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.357338  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.260501  3402 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:28.329783  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.348596  3229 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.271144  3164 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:28.354065  3379 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.515s	user 0.002s	sys 0.000s
W20260709 13:01:28.348327  3383 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.511s	user 0.000s	sys 0.001s
I20260709 13:01:28.272535  3403 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.305124  3310 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:28.501533  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.348678  3399 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.362577  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:28.489370  1650 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:28.363921  3023 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.275221  2835 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.345355  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.448655  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.495486  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.493213  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:28.348778  3396 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.158s
I20260709 13:01:28.518627  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.493991  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:28.497747  3242 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.444990  2793 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.498528  1570 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:28.438450  3325 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.316326  3000 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.503386  1717 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.064077  1719 rpcz_store.cc:276] 0709 13:01:26.486510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.500488 (+ 13978us) service_pool.cc:224] Handling call
0709 13:01:26.512297 (+ 11809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.786214 (+273917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.787426 (+  1212us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.900925 (+113499us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.399183 (+498258us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:27.408821 (+  9638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.479622 (+ 70801us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:27.942655 (+463033us) raft_consensus.cc:1685] finished
0709 13:01:27.948545 (+  5890us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:27.961776 (+ 13231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174505,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":6882,"thread_start_us":617129,"threads_started":2,"wal-append.queue_time_us":718916}
I20260709 13:01:28.442095  1567 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:28.348603  3395 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.147s
I20260709 13:01:28.494362  3213 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.505192  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.536993  1645 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:28.447853  3359 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:28.487386  3341 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.174868  3193 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.503548  1722 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:28.564786  1724 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.490983  3221 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:28.498323  3229 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:28.510607  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.442492  3062 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.514549  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.582007  1574 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.583078  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.329257  2901 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.521404  1723 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.504793  3310 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:28.526033  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:28.447628  3347 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:28.511366  3164 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.535030  1717 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:28.535404  2374 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:28.439909  3390 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.546702  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.547268  2793 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.551786  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.555543  3361 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:28.600669  1646 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:28.601037  3312 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:28.601570  3312 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.602190  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.602974  1646 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:28.604288  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.443074  1644 rpcz_store.cc:276] 0709 13:01:26.941235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.110409 (+169174us) service_pool.cc:224] Handling call
0709 13:01:27.283184 (+172775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.482528 (+199344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.486289 (+  3761us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.545957 (+ 59668us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.701580 (+155623us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:27.709891 (+  8311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.262700 (+552809us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b5400306954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:28.262771 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.262833 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:28.262855 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.263184 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51554,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":25119,"spinlock_wait_cycles":197020928,"thread_start_us":144069,"threads_started":1,"wal-append.queue_time_us":274110}
I20260709 13:01:28.605370  1725 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.605827  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.607076  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.508476  3403 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.609145  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.609777  1650 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:28.610477  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:28.610948  1650 raft_consensus.cc:606] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:28.611346  1650 raft_consensus.cc:614] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:28.611650  1644 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:28.566203  3222 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:28.612540  3222 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.612933  3222 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.606549  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.529572  3242 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.593091  1718 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:28.618654  3242 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.591159  2901 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:28.607811  3351 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.795s	user 0.011s	sys 0.000s
I20260709 13:01:28.620328  3351 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.621562  2901 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.619380  2956 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.565645  3359 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.622671  3351 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:28.623172  3391 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.546s	user 0.001s	sys 0.001s
I20260709 13:01:28.602581  3312 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.563479  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:28.565505  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.572446  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.610972  3403 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.621762  3221 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.626183  3213 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.597813  3062 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.604146  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.586544  3310 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:28.636911  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:28.633229  3204 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.638206  3204 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.638968  3219 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.614629  3325 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:28.640518  3204 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:28.591902  3366 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.006s
I20260709 13:01:28.641165  3365 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.641395  3366 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.641790  3365 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.642704  3109 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:28.643175  3109 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:28.628269  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.619910  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.627444  3000 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.572737  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.589134  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.544767  3238 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:28.595417  3347 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.596390  3144 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.596829  3406 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:01:28.597224  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.597772  3410 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.570842  3065 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:28.598362  1721 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:28.595501  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.543877  3411 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:28.646857  3392 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.615s	user 0.000s	sys 0.002s
I20260709 13:01:28.520184  3023 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.614926  1725 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:01:28.584949  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59012.8us
I20260709 13:01:28.581867  3314 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.507887  3402 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.516289  3399 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.629906  3193 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.645056  3365 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.616601  3141 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:28.650511  1572 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.650971  1717 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.645769  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:28.649139  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 155261us
W20260709 13:01:28.755206  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 109540us
I20260709 13:01:28.755587  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:28.755856  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:28.757288  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:28.649825  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 84442.7us
I20260709 13:01:28.758759  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:28.760529  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:28.761893  3195 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:28.763615  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:28.764394  1572 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.766396  3399 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.612810  3189 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.767753  3190 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:28.607128  2793 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:28.768319  3190 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:28.768931  3190 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:28.768729  2793 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.771522  2793 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.781234  1572 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:28.782590  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.783442  1572 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.784289  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.785790  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.786440  1649 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.787209  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.645974  1718 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.645941  3062 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.644276  3366 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.652860  3238 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.646246  3359 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.644667  3357 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:28.644798  3325 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:28.749787  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 254531us
I20260709 13:01:28.813686  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:28.629880  2835 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.714764  2373 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:28.645423  1570 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:28.808022  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.706202  3008 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:28.751139  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:28.651239  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 132671us
I20260709 13:01:28.804128  1567 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:28.775221  3189 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.787765  1649 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:28.744401  3065 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.791000  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:28.834723  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.835691  1725 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.802059  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:28.649224  1645 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:28.816262  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:28.836292  2377 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.748240  1719 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:28.788141  3000 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:28.837505  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:28.838164  2375 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:28.841528  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.842466  1645 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:28.843844  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.809861  3238 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.829734  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.848456  1649 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:28.808547  3402 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.831987  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:28.833596  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.791709  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:28.851760  1570 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:28.796097  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:28.799698  3193 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:28.792250  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 946) took 1108 ms. Trace:
I20260709 13:01:28.809150  3204 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:28.795738  3141 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.849400  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.754487  3314 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.732915  3109 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.850791  1725 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:28.898414  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:28.899128  1649 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:28.851996  1717 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:28.794561  3195 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:28.852660  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56643.8us
I20260709 13:01:28.901711  1690 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:28.747023  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:28.853144  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 62243.6us
I20260709 13:01:28.813586  3357 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:28.854200  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:28.903445  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 156533us
I20260709 13:01:28.903650  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.904832  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.906113  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.845963  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:28.887972  1572 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:28.833979  3399 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.833040  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:28.834547  3365 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.893937  3008 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.834239  3359 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:28.945645  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.745599  3410 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:28.907174  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 61307.6us
I20260709 13:01:28.930838  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.931737  2793 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.931752  1725 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:28.825695  3222 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.888646  3305 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:28.963104  3305 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.963568  3305 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:28.964473  3347 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.968907  3222 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:28.855440  1646 rpcz_store.cc:276] 0709 13:01:27.649203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.637491 (+988288us) service_pool.cc:224] Handling call
0709 13:01:28.638003 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.650213 (+ 12210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.681473 (+ 31260us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.733313 (+ 51840us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:28.736682 (+  3369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.745000 (+  8318us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.757990 (+ 12990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:28.970104  3141 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.970571  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:28.971410  1646 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:28.929028  3357 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:28.858114  3189 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:28.946511  1570 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:28.974515  1649 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:28.927573  3406 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.858321  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:28.856096  3193 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.950254  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:28.745685  3411 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.858212  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 225) took 1448 ms. Trace:
I20260709 13:01:28.859012  2835 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:28.953925  1725 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:28.956580  3420 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.957957  3423 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:28.960414  3325 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:28.955253  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49090.4us
I20260709 13:01:28.973004  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 940) took 1687 ms. Trace:
I20260709 13:01:28.942879  3062 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:28.974848  3222 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:28.978778  3314 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:28.893537  3417 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 13:01:28.980260  3416 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:28.982450  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51696us
I20260709 13:01:28.983238  3347 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:28.986754  3190 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.988026  3418 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:28.988567  3109 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.049757  3312 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.953896  3359 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:28.989841  3366 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:28.995741  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:28.996909  1721 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:29.059382  1570 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:28.997578  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:29.060508  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63056us
I20260709 13:01:29.000533  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:29.064272  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63860.9us
I20260709 13:01:29.065008  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:28.998942  2835 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.948285  2372 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.004631  3062 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:28.994997  3193 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.042415  3416 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.999409  3189 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:28.941329  3399 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:28.989455  3065 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.098573  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 943) took 1810 ms. Trace:
I20260709 13:01:29.001354  1650 rpcz_store.cc:276] 0709 13:01:27.285762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.612078 (+1326316us) service_pool.cc:224] Handling call
0709 13:01:28.614905 (+  2827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.616133 (+  1228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.616252 (+   119us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:28.624448 (+  8196us) log.cc:844] Serialized 26 byte log entry
0709 13:01:28.900732 (+276284us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:28.900763 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.900819 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.972793 (+ 71974us) raft_consensus.cc:1685] finished
0709 13:01:28.972814 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.972947 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":207,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":275917,"threads_started":1,"wal-append.queue_time_us":340638}
I20260709 13:01:28.962090  3229 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.038825  3423 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.085441  3390 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:28.994918  2371 rpcz_store.cc:276] 0709 13:01:27.407632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.409637 (+  2005us) service_pool.cc:224] Handling call
0709 13:01:27.539626 (+129989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.649136 (+109510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.651641 (+  2505us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.715130 (+ 63489us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.902611 (+187481us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:27.905964 (+  3353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.931854 (+ 25890us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.646181 (+714327us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:28.825038 (+178857us) raft_consensus.cc:1685] finished
0709 13:01:28.829143 (+  4105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.855352 (+ 26209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50584,"raft.queue_time_us":100340,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":13639,"spinlock_wait_cycles":98628480,"thread_start_us":126765,"threads_started":1,"wal-append.queue_time_us":239373}
I20260709 13:01:29.070019  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.072134  2956 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:28.993516  3353 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:29.090103  3331 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:29.092554  2901 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:29.102897  2377 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.046341  3204 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:29.070710  3213 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:29.070523  2835 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.043241  3222 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.105728  3023 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.067013  3312 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:29.086828  3062 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.104490  1644 rpcz_store.cc:276] 0709 13:01:27.288233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.612222 (+1323989us) service_pool.cc:224] Handling call
0709 13:01:28.612518 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.612736 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:28.928635 (+315899us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.098239 (+169604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.098293 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:29.098330 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.098396 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.098518 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":371639,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":4084,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3703,"thread_start_us":484051,"threads_started":2}
W20260709 13:01:29.109876  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 255804us
I20260709 13:01:29.108460  3423 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.112574  3065 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:29.126201  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:29.110436  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.096645  3416 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.128845  1539 leader_election.cc:400] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:29.129264  1539 leader_election.cc:403] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:29.130143  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.118952  2956 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:29.133541  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.134594  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.100544  3195 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.112458  3238 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.073043  3310 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.071113  3365 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:29.100231  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:29.113735  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 72161.5us
I20260709 13:01:29.116906  2901 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.127588  3065 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.133376  3213 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.136308  3418 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:29.152045  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:29.133086  3331 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:29.148763  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:29.158226  1570 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:29.137658  3312 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:29.135669  3417 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:29.165100  3411 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:29.165184  2956 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.166004  3411 raft_consensus.cc:2764] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:29.155781  3065 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.169782  1649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:29.170087  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:29.170360  1649 raft_consensus.cc:606] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:29.170679  1649 raft_consensus.cc:614] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:29.168037  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:29.174571  1725 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:29.126415  3189 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.160677  3312 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.176719  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.179356  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.180156  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.163163  3417 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.184860  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.191085  1649 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:29.186700  3430 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:01:29.186764  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.195449  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:29.200419  2378 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:29.178151  3390 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:29.235581  3390 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:29.236599  3390 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.198992  3229 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:29.241140  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 937) took 2299 ms (client timeout 2232 ms). Trace:
W20260709 13:01:29.241565  1647 rpcz_store.cc:269] 0709 13:01:26.941886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.963367 (+ 21481us) service_pool.cc:224] Handling call
0709 13:01:29.240957 (+2277590us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400133b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:29.241076 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369842944}
I20260709 13:01:29.243952  3188 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.245260  3188 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.247329  3238 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:29.193231  3417 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.248399  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1683) took 1293 ms. Trace:
I20260709 13:01:29.249059  1723 rpcz_store.cc:276] 0709 13:01:27.955126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.448396 (+493270us) service_pool.cc:224] Handling call
0709 13:01:29.248183 (+799787us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b5400294ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:29.248315 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12911360}
W20260709 13:01:29.250466  3434 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:01:29.250483  3222 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:29.251660  3385 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.416s	user 0.003s	sys 0.009s
I20260709 13:01:29.251784  1645 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:29.200109  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.253289  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:29.201278  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.262260  2380 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.203516  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.187496  3190 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.209930  3305 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.162848  3204 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:29.211833  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 921) took 2787 ms (client timeout 1682 ms). Trace:
W20260709 13:01:29.216672  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.216703  3411 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:29.218108  3399 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.219794  3310 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.180186  3359 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:29.227891  3141 raft_consensus.cc:1064] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:01:29.183682  3312 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.264909  1649 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.320508  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.260844  3338 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.291080  2373 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:29.317845  1718 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.003s
I20260709 13:01:29.324474  2377 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:29.325611  3338 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.329516  3366 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:29.314992  3062 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.316258  3411 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.357465  1718 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:29.357811  3062 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:29.364807  3193 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.369642  3305 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:29.331945  2373 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 314) took 1401 ms (client timeout 1501 ms). Trace:
W20260709 13:01:29.370657  2373 rpcz_store.cc:269] 0709 13:01:27.929171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.997366 (+ 68195us) service_pool.cc:224] Handling call
0709 13:01:28.563152 (+565786us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:29.330360 (+767208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40164096}
I20260709 13:01:29.332194  3213 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:29.375453  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 948) took 1532 ms. Trace:
I20260709 13:01:29.382107  1650 rpcz_store.cc:276] 0709 13:01:27.843084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.118474 (+1275390us) service_pool.cc:224] Handling call
0709 13:01:29.125404 (+  6930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.125760 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.125790 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:29.126249 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.315886 (+189637us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:29.315926 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.315998 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.375244 (+ 59246us) raft_consensus.cc:1685] finished
0709 13:01:29.375264 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.375393 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1123,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":7806,"thread_start_us":169297,"threads_started":1,"wal-append.queue_time_us":204778}
I20260709 13:01:29.384302  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.385749  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.387117  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.377700  1717 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:29.334630  3418 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:29.313264  1648 rpcz_store.cc:269] 0709 13:01:26.424200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.427552 (+  3352us) service_pool.cc:224] Handling call
0709 13:01:29.211610 (+2784058us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540038a9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:29.211759 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604367616}
I20260709 13:01:29.389431  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.389971  1648 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.390331  3141 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.394102  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.395125  1650 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:29.398245  3065 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.401181  3411 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.376972  3358 raft_consensus.cc:2764] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:29.404707  3065 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.410663  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.413235  1650 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:29.416033  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.418756  1650 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:29.367515  3193 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:29.422395  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 287834us
I20260709 13:01:29.426012  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.438608  2835 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.440065  1648 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.440563  3442 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:29.441042  3442 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.441370  3442 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.441367  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.442510  3338 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.442975  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.443578  1648 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.450424  3338 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.451025  1649 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:29.451327  1648 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.433950  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.459553  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.460099  1650 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.455125  2835 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:29.467054  1539 leader_election.cc:400] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:29.458088  3065 raft_consensus.cc:2764] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:29.396432  3357 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.457984  3361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49746 (request call id 74) took 4162 ms. Trace:
I20260709 13:01:29.405913  3213 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.484355  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:29.488535  2373 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:29.484676  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.483482  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1555) took 2540 ms. Trace:
I20260709 13:01:29.491058  1575 rpcz_store.cc:276] 0709 13:01:26.943208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.947128 (+  3920us) service_pool.cc:224] Handling call
0709 13:01:26.947464 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:27.124388 (+176924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.124416 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:27.125017 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 13:01:27.598081 (+473064us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:27.656413 (+ 58332us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.676872 (+ 20459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.352625 (+675753us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:29.483129 (+1130504us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54002152d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:29.483184 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.483227 (+    43us) raft_consensus.cc:1685] finished
0709 13:01:29.483243 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.483398 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20034,"prepare.queue_time_us":53737,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":6812,"raft.queue_time_us":24997,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":16911,"spinlock_wait_cycles":436699776,"thread_start_us":471485,"threads_started":1,"wal-append.queue_time_us":662334}
I20260709 13:01:29.494940  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:29.497176  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:29.499629  3393 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.500237  3393 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.504779  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 944) took 1070 ms. Trace:
I20260709 13:01:29.505357  3366 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.505755  3331 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.506397  3450 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.505234  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:29.480935  3361 rpcz_store.cc:276] 0709 13:01:25.292486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.296536 (+  4050us) service_pool.cc:224] Handling call
0709 13:01:25.334356 (+ 37820us) log.cc:844] Serialized 501 byte log entry
0709 13:01:25.353790 (+ 19434us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b54001831a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce7142 00007fc833cc7eaa 00007fc833cced53 00007fc833ccecd9 00007fc833ccec69 00007fc833ccec31 00007fc833ccddfd 000000000038f354 000000000038f199 00007fc830c81475
0709 13:01:25.379321 (+ 25531us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800216d80. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833daad01 00007fc833ce7691 00007fc833cc7eaa 00007fc833cced53 00007fc833ccecd9 00007fc833ccec69 00007fc833ccec31 00007fc833ccddfd 000000000038f354 000000000038f199
0709 13:01:29.453716 (+4074395us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7985,"raft.queue_time_us":6040,"raft.run_cpu_time_us":4261,"raft.run_wall_time_us":79935,"spinlock_wait_cycles":29259392}
I20260709 13:01:29.506313  3331 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.511157  3365 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.511602  3365 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.511854  3331 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:29.506315  3440 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.513337  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:29.514420  3331 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:29.514849  3331 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:29.515272  3331 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:29.516611  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 109898us
I20260709 13:01:29.520862  1650 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:29.506562  3310 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.522552  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:29.530814  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.531744  1650 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:29.506022  3366 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.530884  3195 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:29.548537  3393 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:29.548658  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1677) took 1651 ms (client timeout 1668 ms). Trace:
W20260709 13:01:29.549151  1721 rpcz_store.cc:269] 0709 13:01:27.897406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.031453 (+134047us) service_pool.cc:224] Handling call
0709 13:01:29.548465 (+1517012us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54003ab954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:29.548574 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097745152}
I20260709 13:01:29.551513  1573 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:29.551887  1573 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:29.553943  1573 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.554908  1573 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.556517  3417 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.505220  1568 rpcz_store.cc:276] 0709 13:01:28.435277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.401051 (+965774us) service_pool.cc:224] Handling call
0709 13:01:29.401387 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.401600 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:29.453538 (+ 51938us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.502784 (+ 49246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.503079 (+   295us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:29.504485 (+  1406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.504575 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.504714 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12150,"prepare.queue_time_us":101032,"prepare.run_wall_time_us":46,"raft.queue_time_us":23075,"raft.run_wall_time_us":5242,"spinlock_wait_cycles":14071552,"thread_start_us":41188,"threads_started":2,"wal-append.queue_time_us":62145}
I20260709 13:01:29.553453  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.525501  3310 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.516691  3440 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.560148  3366 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.503497  3418 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.560407  3349 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:29.517158  3450 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.531001  3312 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.522485  3416 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.566393  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:29.567128  1568 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:29.567992  3382 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.537s	user 0.003s	sys 0.001s
I20260709 13:01:29.568109  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:29.533428  3442 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.569437  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.552955  3195 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:29.574371  1570 raft_consensus.cc:2368] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5.
I20260709 13:01:29.560236  3440 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.575809  1572 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:29.570182  1568 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.584826  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.778956  1650 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:01:29.602561  3452 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:01:29.782275  2901 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.783547  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1674) took 1918 ms. Trace:
I20260709 13:01:29.784514  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.784935  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.785660  2379 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:29.783928  1720 rpcz_store.cc:276] 0709 13:01:27.868778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.882485 (+ 13707us) service_pool.cc:224] Handling call
0709 13:01:27.905823 (+ 23338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.359702 (+453879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:28.359737 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:28.359784 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:28.359814 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.783334 (+1423520us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b5400213c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:29.783379 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.783492 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043078400}
I20260709 13:01:29.785784  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:29.671039  3454 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:01:29.677987  3450 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.684823  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:29.538105  3423 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.573200  3366 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:29.801306  3366 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.801738  3366 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:29.769555  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.592816  3195 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:29.771975  3005 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:01:29.773566  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.780522  1575 raft_consensus.cc:2368] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3.
I20260709 13:01:29.775965  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 7)
W20260709 13:01:29.786044  2379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 313) took 1895 ms (client timeout 1760 ms). Trace:
I20260709 13:01:29.790184  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:29.793816  3374 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.703s	user 0.006s	sys 0.005s
W20260709 13:01:29.793689  3455 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.189s	user 0.003s	sys 0.004s
W20260709 13:01:29.813292  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 135850us
W20260709 13:01:29.813854  2379 rpcz_store.cc:269] 0709 13:01:27.891026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.950406 (+ 59380us) service_pool.cc:224] Handling call
0709 13:01:29.784704 (+1834298us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:29.785981 (+  1277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42384640}
I20260709 13:01:29.817024  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.820842  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.825925  1647 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.815955  3310 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.821461  1644 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:29.819078  3173 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:29.812037  3357 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:29.820292  1458 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44482.1us
W20260709 13:01:29.845067  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.835856  3173 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.859365  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.860009  1650 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:29.861581  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.863320  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.863898  1650 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.855381  3173 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:29.870155  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.873549  1648 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:29.857563  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 924) took 3427 ms. Trace:
I20260709 13:01:29.868891  3417 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.889819  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:29.892040  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.893512  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.895018  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:29.897671  1650 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:29.899379  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:29.875633  3416 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:29.874109  1642 rpcz_store.cc:276] 0709 13:01:26.429280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.431440 (+  2160us) service_pool.cc:224] Handling call
0709 13:01:26.740682 (+309242us) spinlock_profiling.cc:243] Waited 9.7 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:26.740862 (+   180us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.005811 (+264949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.005858 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.006606 (+   748us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.704914 (+698308us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:27.714784 (+  9870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.255356 (+1540572us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:29.255412 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.587470 (+332058us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:29.846499 (+259029us) raft_consensus.cc:1685] finished
0709 13:01:29.847708 (+  1209us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.855239 (+  7531us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31439,"raft.queue_time_us":32626,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":602962816,"thread_start_us":687941,"threads_started":1,"wal-append.queue_time_us":1066672}
I20260709 13:01:29.908378  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.911573  1648 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:29.890729  3442 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:29.914000  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 141123us
I20260709 13:01:29.897024  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 947) took 2245 ms. Trace:
I20260709 13:01:29.901734  3357 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:29.916559  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.917158  1650 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.920483  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.924252  2377 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:29.925408  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:29.933476  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:29.933547  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.934942  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:29.916100  1568 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:29.910713  3331 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.919446  3204 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.923627  3442 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.913548  3365 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.941484  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.941574  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.942211  1648 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:29.932447  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:29.932901  3416 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:29.946652  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.938061  3418 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:29.945499  3310 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:29.948786  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:29.949292  3450 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.950508  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:29.938990  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:29.954582  1650 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:29.955030  1648 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:29.954197  1642 raft_consensus.cc:606] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:29.958539  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:29.944669  3204 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:29.961944  3357 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:29.953912  3416 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:29.958762  1642 raft_consensus.cc:614] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:29.944262  3442 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:29.968582  1648 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:29.955600  3418 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.971158  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:29.971592  1650 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:29.961501  2371 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.959014  3450 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:29.974164  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1648) took 1966 ms. Trace:
I20260709 13:01:29.979417  1650 rpcz_store.cc:276] 0709 13:01:28.008294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.964783 (+1956489us) service_pool.cc:224] Handling call
0709 13:01:29.971416 (+  6633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.973889 (+  2473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.974101 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:29.916121  1646 rpcz_store.cc:276] 0709 13:01:27.649550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.038748 (+1389198us) service_pool.cc:224] Handling call
0709 13:01:29.056172 (+ 17424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:29.316397 (+260225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.317811 (+  1414us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:29.421557 (+103746us) log.cc:844] Serialized 52 byte log entry
0709 13:01:29.540954 (+119397us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:29.572243 (+ 31289us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.590833 (+ 18590us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b180042f160. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:29.622301 (+ 31468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.699714 (+ 77413us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.871104 (+171390us) raft_consensus.cc:1685] finished
0709 13:01:29.877703 (+  6599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.895394 (+ 17691us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6383,"prepare.queue_time_us":19198,"prepare.run_cpu_time_us":5859,"prepare.run_wall_time_us":180194,"raft.queue_time_us":15598,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":2660,"spinlock_wait_cycles":10073856,"thread_start_us":266873,"threads_started":2,"wal-append.queue_time_us":110336}
I20260709 13:01:29.981356  1568 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:29.973811  3418 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.987052  3008 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:29.989406  1648 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:29.990118  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1652) took 1980 ms. Trace:
I20260709 13:01:29.988438  3310 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:29.990700  3310 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:29.967880  3416 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:29.993075  1648 rpcz_store.cc:276] 0709 13:01:28.010265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.971683 (+1961418us) service_pool.cc:224] Handling call
0709 13:01:29.989234 (+ 17551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.989888 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.990061 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:29.999271  3310 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.000969  3310 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:30.002238  3204 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:30.005142  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.005041  3450 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.970208  3357 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:30.011243  3418 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:29.965845  3457 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:01:29.989143  3065 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:30.096583  3442 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:30.098343  3365 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.099278  3065 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:30.100301  3065 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.100628  3118 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.104926  3357 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.016067  3416 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.104977  3118 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.111750  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:30.113277  3118 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:30.109216  3366 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.116358  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:30.118686  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110219us
I20260709 13:01:30.120013  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 949) took 1806 ms. Trace:
I20260709 13:01:30.141567  3411 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:30.151563  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.152606  1650 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.156930  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:30.116190  3331 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.159605  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:30.139816  3347 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.163597  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.115314  3173 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.178375  1650 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:30.182719  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.184679  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.186466  1650 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:30.160169  3348 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:30.196835  3421 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:30.207413  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48037.1us
W20260709 13:01:30.208297  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53654.3us
I20260709 13:01:30.206146  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.208200  3418 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.167949  3347 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.220093  1576 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:30.220875  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 942) took 1814 ms (client timeout 1814 ms). Trace:
I20260709 13:01:30.199226  3421 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:30.224756  3421 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:30.221447  1576 rpcz_store.cc:269] 0709 13:01:28.406655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.487325 (+ 80670us) service_pool.cc:224] Handling call
0709 13:01:30.163414 (+1676089us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:30.220785 (+ 57371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18183552}
I20260709 13:01:30.121415  1648 rpcz_store.cc:276] 0709 13:01:28.313416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.004387 (+1690971us) service_pool.cc:224] Handling call
0709 13:01:30.004775 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.004992 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:30.006821 (+  1829us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.119629 (+112808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.119694 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:30.119750 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.119818 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.119950 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":209,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":133,"thread_start_us":112158,"threads_started":1}
I20260709 13:01:30.264900  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:30.266073  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:30.270095  1648 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:30.271255  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1655) took 1949 ms. Trace:
I20260709 13:01:30.271792  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.272723  1576 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:30.274431  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.275157  1576 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:30.276153  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.271872  1648 rpcz_store.cc:276] 0709 13:01:28.321519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.250621 (+1929102us) service_pool.cc:224] Handling call
0709 13:01:30.269885 (+ 19264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.270960 (+  1075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.271193 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.286298  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.287482  1648 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:30.287540  1576 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:30.288440  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1656) took 1966 ms. Trace:
I20260709 13:01:30.288866  1648 rpcz_store.cc:276] 0709 13:01:28.321890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.277475 (+1955585us) service_pool.cc:224] Handling call
0709 13:01:30.287276 (+  9801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.288069 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.288329 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.271766  3348 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:30.293560  1648 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:30.294872  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1657) took 1971 ms. Trace:
I20260709 13:01:30.295233  1648 rpcz_store.cc:276] 0709 13:01:28.323090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.289595 (+1966505us) service_pool.cc:224] Handling call
0709 13:01:30.293179 (+  3584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.294549 (+  1370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.294795 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.296962  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.298072  3173 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:30.284845  3463 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.308s	user 0.003s	sys 0.006s
I20260709 13:01:30.300254  1576 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:30.304903  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 966) took 1129 ms. Trace:
I20260709 13:01:30.305729  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1688) took 1873 ms. Trace:
I20260709 13:01:30.306188  1717 rpcz_store.cc:276] 0709 13:01:28.432425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.900288 (+467863us) service_pool.cc:224] Handling call
0709 13:01:28.949641 (+ 49353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.388723 (+439082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.305441 (+916718us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b5400233054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:30.305658 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983989504}
I20260709 13:01:30.305524  1649 rpcz_store.cc:276] 0709 13:01:29.174964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.320169 (+145205us) service_pool.cc:224] Handling call
0709 13:01:30.304674 (+984505us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54000f55d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.304811 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298458880}
I20260709 13:01:30.306474  1650 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:30.307801  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:30.308589  1649 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:30.308688  1648 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:30.309434  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 955) took 1862 ms. Trace:
I20260709 13:01:30.309434  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 956) took 1857 ms. Trace:
I20260709 13:01:30.311419  1648 rpcz_store.cc:276] 0709 13:01:28.447304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.296264 (+1848960us) service_pool.cc:224] Handling call
0709 13:01:30.308477 (+ 12213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.309186 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.309366 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3968}
I20260709 13:01:30.312398  1649 rpcz_store.cc:276] 0709 13:01:28.452523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.307706 (+1855183us) service_pool.cc:224] Handling call
0709 13:01:30.308376 (+   670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.309148 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.309356 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.314049  1648 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:30.313908  3416 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.315035  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 119) took 1507 ms. Trace:
I20260709 13:01:30.315814  1649 rpcz_store.cc:276] 0709 13:01:28.807453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.313547 (+1506094us) service_pool.cc:224] Handling call
0709 13:01:30.314110 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.314449 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.314478 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.314601 (+   123us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:30.314839 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.314899 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.314985 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.317138  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.318576  1576 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.319518  3416 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.315035  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 118) took 1508 ms. Trace:
I20260709 13:01:30.323084  1648 rpcz_store.cc:276] 0709 13:01:28.807086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.312956 (+1505870us) service_pool.cc:224] Handling call
0709 13:01:30.313470 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.314722 (+  1252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.314958 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.325039  1649 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:30.299582  3348 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.307495  3347 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:30.351243  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.352100  1571 raft_consensus.cc:2376] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4. Re-sending same reply.
W20260709 13:01:30.352910  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 941) took 1946 ms (client timeout 1700 ms). Trace:
W20260709 13:01:30.353449  1571 rpcz_store.cc:269] 0709 13:01:28.406117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.426847 (+ 20730us) service_pool.cc:224] Handling call
0709 13:01:30.351046 (+1924199us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:30.352778 (+  1732us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81165696}
I20260709 13:01:30.355015  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.321846  3366 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:30.355963  1575 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:30.362620  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 963) took 1189 ms. Trace:
I20260709 13:01:30.363045  1649 rpcz_store.cc:276] 0709 13:01:29.172887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.324296 (+1151409us) service_pool.cc:224] Handling call
0709 13:01:30.324782 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.326895 (+  2113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.362539 (+ 35644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.363425  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.364101  1575 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.364523  1649 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:30.356710  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:30.359304  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1645) took 2385 ms (client timeout 1821 ms). Trace:
I20260709 13:01:30.365813  1647 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:30.366068  1649 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:30.365854  1643 rpcz_store.cc:269] 0709 13:01:27.974374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.151174 (+176800us) service_pool.cc:224] Handling call
0709 13:01:30.358989 (+2207815us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b5400389fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.359185 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15466880}
I20260709 13:01:30.310567  3473 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:30.335893  3348 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.381193  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1654) took 2369 ms. Trace:
I20260709 13:01:30.382946  3402 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:30.387054  3414 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.852s	user 0.005s	sys 0.005s
I20260709 13:01:30.388181  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1568) took 2454 ms. Trace:
I20260709 13:01:30.415063  1573 rpcz_store.cc:276] 0709 13:01:27.933339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.933492 (+   153us) service_pool.cc:224] Handling call
0709 13:01:27.934804 (+  1312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.551417 (+1616613us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5400252c20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d701c0 00007fc833d71231 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:01:29.771513 (+220096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.771579 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:29.771654 (+    75us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:29.771721 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.387807 (+616086us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b5400001c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:30.387881 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.388087 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":45009280}
I20260709 13:01:30.417805  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.388067  1647 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:30.397403  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:30.397015  1575 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:30.399446  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 91763.4us
W20260709 13:01:30.409504  1651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 111) took 3079 ms (client timeout 2080 ms). Trace:
I20260709 13:01:30.420889  1690 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:30.421077  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.420889  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:30.422696  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.423594  2344 leader_election.cc:400] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3.
I20260709 13:01:30.424430  2344 leader_election.cc:403] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:30.423609  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.424542  3357 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:30.425283  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.425645  3365 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:30.420646  1651 rpcz_store.cc:269] 0709 13:01:27.329952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.713270 (+383318us) service_pool.cc:224] Handling call
0709 13:01:30.409077 (+2695807us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54000915d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.409384 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4847772032}
I20260709 13:01:30.426209  3365 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:30.425156  1647 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.426684  3365 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:30.427415  3474 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:01:30.425586  3357 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:30.429250  3357 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.432744  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:30.433570  3471 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
I20260709 13:01:30.434243  3475 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:30.435001  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:30.435446  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:30.437601  3470 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:01:30.437122  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.438482  1575 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:30.440565  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.442921  3347 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:30.445163  3365 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.448701  3347 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.449370  3347 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:30.450819  1575 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.453090  3358 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:30.466727  1647 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:30.469146  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:30.470891  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:30.456280  3365 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:30.413630  3402 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:30.475874  3358 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:30.476354  3402 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:30.477136  3357 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.398622  3473 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:30.485569  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1668) took 2697 ms (client timeout 2014 ms). Trace:
W20260709 13:01:30.486044  1724 rpcz_store.cc:269] 0709 13:01:27.776611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.777088 (+   477us) service_pool.cc:224] Handling call
0709 13:01:30.474044 (+2696956us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540010d6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.474191 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1738568576}
W20260709 13:01:30.465561  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 254643us
W20260709 13:01:30.472834  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 117) took 1683 ms (client timeout 1973 ms). Trace:
I20260709 13:01:30.411458  1646 rpcz_store.cc:276] 0709 13:01:28.011643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.982273 (+1970630us) service_pool.cc:224] Handling call
0709 13:01:29.982727 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:29.983295 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.983330 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:29.984076 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 13:01:30.187520 (+203444us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:30.212488 (+ 24968us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.215255 (+  2767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.215403 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.380875 (+165472us) raft_consensus.cc:1685] finished
0709 13:01:30.380908 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.381112 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32358,"prepare.queue_time_us":16067,"prepare.run_cpu_time_us":915,"prepare.run_wall_time_us":27240,"raft.queue_time_us":1132,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":307,"thread_start_us":179259,"threads_started":1,"wal-append.queue_time_us":228230}
I20260709 13:01:30.474939  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 965) took 1714 ms. Trace:
I20260709 13:01:30.490108  1572 rpcz_store.cc:276] 0709 13:01:28.760213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.253033 (+492820us) service_pool.cc:224] Handling call
0709 13:01:30.474739 (+1221706us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54001a9ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.474871 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740204288}
I20260709 13:01:30.476440  3358 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.463932  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.480485  1643 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:30.483564  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:30.487221  1645 rpcz_store.cc:269] 0709 13:01:28.789507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.843513 (+ 54006us) service_pool.cc:224] Handling call
0709 13:01:30.472640 (+1629127us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54001bc1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.472760 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932559744}
I20260709 13:01:30.498759  1647 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:30.503170  3357 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.488709  3473 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:30.505144  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.506608  1568 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:30.507189  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 120) took 1036 ms. Trace:
I20260709 13:01:30.507550  1568 rpcz_store.cc:276] 0709 13:01:29.470378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.425012 (+954634us) service_pool.cc:224] Handling call
0709 13:01:30.507115 (+ 82103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.508711  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:30.509799  2377 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:30.514256  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.514824  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.515707  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.478865  3410 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:30.512785  3118 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.518376  3118 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.509027  3395 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:30.489044  3403 raft_consensus.cc:2764] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:30.526146  1575 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.511161  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:30.532294  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.494539  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.545679  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.547164  1572 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:30.548457  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.551031  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:30.553009  1651 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:30.556388  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.557369  1649 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:01:30.558153  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 63758.5us
I20260709 13:01:30.558723  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 958) took 2066 ms. Trace:
I20260709 13:01:30.563999  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:30.555277  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:30.559157  1650 rpcz_store.cc:276] 0709 13:01:28.492821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.309368 (+1816547us) service_pool.cc:224] Handling call
0709 13:01:30.309689 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.310403 (+   714us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:30.360729 (+ 50326us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.558348 (+197619us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.558411 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:30.558467 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.558536 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.558660 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17791,"prepare.queue_time_us":112558,"prepare.run_cpu_time_us":175,"prepare.run_wall_time_us":175,"thread_start_us":179213,"threads_started":1}
I20260709 13:01:30.531450  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.496438  3421 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.541823  3406 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:30.570021  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.579435  1573 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.570729  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1667) took 2795 ms. Trace:
W20260709 13:01:30.582273  2371 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.568s	user 0.005s	sys 0.009s
I20260709 13:01:30.573864  3406 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:30.585515  1567 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:30.557518  1571 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:30.600458  1575 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:30.605842  1576 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
W20260709 13:01:30.605124  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 169749us
I20260709 13:01:30.632872  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:30.634728  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.620787  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.593361  3118 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.593737  3475 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:30.622799  3406 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:30.606720  3410 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:30.682789  1649 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:30.633040  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 129) took 1159 ms. Trace:
I20260709 13:01:30.684600  1576 rpcz_store.cc:276] 0709 13:01:29.474087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.302370 (+828283us) service_pool.cc:224] Handling call
0709 13:01:30.632969 (+330599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.606472  2371 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:30.690712  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140228us
I20260709 13:01:30.693253  1575 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:30.693310  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60541.3us
I20260709 13:01:30.697923  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:30.690941  3458 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:30.702565  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 972) took 1505 ms. Trace:
I20260709 13:01:30.702836  1651 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:30.607996  3473 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:30.702169  3458 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:30.816046  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:30.769708  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1644) took 2813 ms. Trace:
I20260709 13:01:30.839042  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:30.739190  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:30.865774  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:30.966509  1717 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:30.769824  1647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:30.838107  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 947) took 2387 ms. Trace:
W20260709 13:01:30.922154  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 290643us
I20260709 13:01:30.725766  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 946) took 2282 ms. Trace:
I20260709 13:01:30.838632  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:30.741307  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:30.839485  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.843315  1574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 956) took 2284 ms (client timeout 1783 ms). Trace:
I20260709 13:01:30.836059  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:30.902942  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:30.730116  1573 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:30.831100  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:30.709311  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.983564  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.983724  2379 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:01:30.931231  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.965116  1721 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:30.985013  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151244us
W20260709 13:01:30.769040  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.781188  1474 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 3 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:30.983404  1720 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:30.985292  1644 rpcz_store.cc:276] 0709 13:01:27.956610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.129093 (+1172483us) service_pool.cc:224] Handling call
0709 13:01:29.146579 (+ 17486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.835953 (+689374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.769439 (+933486us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54000ba754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:30.769638 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1551823616}
I20260709 13:01:30.983130  1572 rpcz_store.cc:276] 0709 13:01:28.443795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.570988 (+2127193us) service_pool.cc:224] Handling call
0709 13:01:30.579163 (+  8175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.579385 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:30.626685 (+ 47300us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.725007 (+ 98322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.725073 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:30.725145 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.725266 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.725650 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27261,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":665,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":3204,"thread_start_us":97170,"threads_started":1,"wal-append.queue_time_us":208024}
I20260709 13:01:30.983057  1575 rpcz_store.cc:276] 0709 13:01:28.450735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.629343 (+2178608us) service_pool.cc:224] Handling call
0709 13:01:30.649348 (+ 20005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.694100 (+ 44752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.837729 (+143629us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b54000c2954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:30.838017 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100248832}
W20260709 13:01:30.833853  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 951) took 2307 ms (client timeout 3004 ms). Trace:
I20260709 13:01:30.989008  1647 raft_consensus.cc:606] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:30.693749  3195 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3.
W20260709 13:01:30.988731  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 247540us
W20260709 13:01:30.861716  3478 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 13:01:30.983049  1574 rpcz_store.cc:269] 0709 13:01:28.558376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.582781 (+ 24405us) service_pool.cc:224] Handling call
0709 13:01:30.842997 (+2260216us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54001aa9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.843166 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3197066496}
I20260709 13:01:31.018042  1573 raft_consensus.cc:2376] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3. Re-sending same reply.
I20260709 13:01:30.735505  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:30.873822  3421 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:30.703078  1648 rpcz_store.cc:276] 0709 13:01:29.197592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.325711 (+1128119us) service_pool.cc:224] Handling call
0709 13:01:30.326079 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:30.326405 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:30.624171 (+297766us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.702159 (+ 77988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.702219 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:30.702275 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.702345 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.702504 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34755,"prepare.queue_time_us":130012,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":132,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"thread_start_us":279564,"threads_started":2,"wal-append.queue_time_us":217670}
I20260709 13:01:30.771406  3423 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:30.835965  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 130) took 1260 ms. Trace:
I20260709 13:01:30.593662  1716 rpcz_store.cc:276] 0709 13:01:27.775402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.785712 (+ 10310us) service_pool.cc:224] Handling call
0709 13:01:27.855346 (+ 69634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.437474 (+582128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.542315 (+2104841us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:30.568689 (+ 26374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716289280}
I20260709 13:01:30.692615  3411 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.833401  3442 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:30.833339  2373 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:30.709857  3357 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:30.839543  3484 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:31.032131  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:31.035377  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:30.772946  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1552) took 2646 ms. Trace:
W20260709 13:01:31.091631  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 354046us
I20260709 13:01:30.842624  3450 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.095451  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.096103  1576 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:31.016650  1570 rpcz_store.cc:269] 0709 13:01:28.526017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.158901 (+632884us) service_pool.cc:224] Handling call
0709 13:01:30.833623 (+1674722us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54000c2e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.833782 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928132992}
I20260709 13:01:31.037707  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.018079  1647 raft_consensus.cc:614] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:30.842128  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.027706  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.012332  3395 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:31.061517  3310 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:31.015025  3396 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.015858  3204 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:30.740401  1568 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:31.051105  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.052023  1573 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.108825  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 950) took 2599 ms. Trace:
I20260709 13:01:31.109416  1573 rpcz_store.cc:276] 0709 13:01:28.509578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.042332 (+2532754us) service_pool.cc:224] Handling call
0709 13:01:31.051848 (+  9516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.108461 (+ 56613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.108748 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.728629  3118 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.724151  3482 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:30.828725  3475 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.112571  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.031426  3487 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.751013  3008 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.297s
I20260709 13:01:30.700923  3410 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.058162  1721 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:31.100215  3357 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:31.120549  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.879874  3348 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.106666  3165 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:31.060130  3358 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
W20260709 13:01:31.092742  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60710.6us
I20260709 13:01:31.128777  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:31.130656  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:30.710759  3476 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.053045  1571 raft_consensus.cc:2376] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4. Re-sending same reply.
I20260709 13:01:31.066414  3411 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.094045  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.095014  3493 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.091741  3190 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:01:30.817379  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:31.098721  1574 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:01:31.100975  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73385.8us
I20260709 13:01:31.102447  1718 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:30.843251  3312 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.096006  3450 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.114223  1570 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.124636  3165 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:30.745745  3416 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.312s
I20260709 13:01:31.180730  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.181640  1723 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:31.183789  2378 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:31.065290  3442 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:31.113186   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:31.196676  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.197757  1717 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:31.203265  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.204200  1571 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:31.093276  1719 rpcz_store.cc:276] 0709 13:01:28.126515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.889838 (+763323us) service_pool.cc:224] Handling call
0709 13:01:28.896291 (+  6453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:28.896867 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:29.113086 (+216219us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.385109 (+1272023us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b18003c8080. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:01:30.734970 (+349861us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:30.768762 (+ 33792us) log.cc:844] Serialized 26 byte log entry
0709 13:01:30.769024 (+   262us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:30.769096 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.769215 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.772183 (+  2968us) raft_consensus.cc:1685] finished
0709 13:01:30.772395 (+   212us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.772749 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29132,"prepare.queue_time_us":239010,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2501,"raft.queue_time_us":268505,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":11289,"spinlock_wait_cycles":4856064,"thread_start_us":509712,"threads_started":3,"wal-append.queue_time_us":1480300}
I20260709 13:01:31.168929  3493 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.210012  1722 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:31.210939  3493 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.119763  3487 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.211648  1720 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:31.213876  3487 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.178808  3165 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:31.217862  3366 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.218613  3366 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.220984  3366 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.221203  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.221621  3450 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.222380  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.118134  3482 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.223698  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.224440  3450 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:31.173595  3410 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.181766  3312 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:01:31.224298  1575 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:31.103783  3423 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:31.227972  3347 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.778s	user 0.014s	sys 0.002s
W20260709 13:01:31.121201  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 131941us
I20260709 13:01:31.228477  3347 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.230734  3347 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.108289  3484 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.231559  3484 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:31.121347  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 382298us
W20260709 13:01:31.118467  3464 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.849s	user 0.004s	sys 0.001s
I20260709 13:01:30.827973  3402 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:31.140087  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.139859  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:31.140925  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:31.144230  2379 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:31.156956  3476 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.145520  3357 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.149053  3348 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.238476  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:31.245055  1651 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:31.246146  3484 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.153381  3481 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:31.180608  3496 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.247363  1724 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:31.247524  3496 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.247386  3481 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:31.248936  3481 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:31.250058  3496 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.118656  3204 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:31.251382  3204 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.252421  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:31.223505  3411 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.166312  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.240406  3482 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:31.255288  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 89142.1us
I20260709 13:01:31.167973  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 954) took 2610 ms. Trace:
I20260709 13:01:31.256826  1574 rpcz_store.cc:276] 0709 13:01:28.557694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.098115 (+2540421us) service_pool.cc:224] Handling call
0709 13:01:31.098494 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.167590 (+ 69096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.167883 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.258749  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:31.259483  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 962) took 2500 ms. Trace:
W20260709 13:01:31.256242  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 116524us
I20260709 13:01:31.110675  3310 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.174928  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 953) took 2617 ms. Trace:
I20260709 13:01:31.117455  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.288965  1721 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:31.115720  3118 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.126652  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 948) took 2631 ms. Trace:
I20260709 13:01:31.138535  3395 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:31.137048  3173 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:31.114956  3396 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:31.186244  1473 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 205) took 14809 ms (client timeout 15000 ms). Trace:
I20260709 13:01:31.188323  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.189236  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.194422  1723 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:31.224687  3458 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.153916  1474 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:31.054435  1567 rpcz_store.cc:276] 0709 13:01:29.575678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.164282 (+588604us) service_pool.cc:224] Handling call
0709 13:01:30.835445 (+671163us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b540027d654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:30.835859 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705762688}
I20260709 13:01:31.264628  1651 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:31.262235  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.260270  1574 rpcz_store.cc:276] 0709 13:01:28.759196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.258186 (+2498990us) service_pool.cc:224] Handling call
0709 13:01:31.258831 (+   645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.259070 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.259104 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.259165 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:31.259216 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.259286 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.259412 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.336223  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74080.4us
I20260709 13:01:31.274158  1572 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:31.337934  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.338172  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 959) took 2778 ms. Trace:
I20260709 13:01:31.338841  1572 rpcz_store.cc:276] 0709 13:01:28.560093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.223613 (+2663520us) service_pool.cc:224] Handling call
0709 13:01:31.272746 (+ 49133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.337853 (+ 65107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.338108 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.273926  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:31.284011  1575 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:31.290032  1647 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:31.288059  1570 rpcz_store.cc:276] 0709 13:01:28.557237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.097436 (+2540199us) service_pool.cc:224] Handling call
0709 13:01:31.113961 (+ 16525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.174658 (+ 60697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.174868 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.196774   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:31.298027  3487 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.298966  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46682.7us
I20260709 13:01:31.346645  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:31.299521  3482 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:31.293572  3423 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.232257  3366 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:31.396452  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49881.1us
I20260709 13:01:31.347191  1474 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.399744  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:31.400956  1474 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 2 to 3, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 3 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.140525  3358 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:31.306517  1473 rpcz_store.cc:269] 0709 13:01:16.373717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.374012 (+   295us) service_pool.cc:224] Handling call
0709 13:01:17.154442 (+780430us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:01:31.182772 (+14028330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:25.823755 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:25.839402 (+ 15647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:25.839452 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:01:25.839473 (+    21us) tablet.cc:662] Decoding operations
0709 13:01:25.850041 (+ 10568us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:25.850140 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:25.850173 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:01:25.850719 (+   546us) tablet.cc:701] Row locks acquired
0709 13:01:25.850772 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:01:25.850949 (+   177us) write_op.cc:270] Start()
0709 13:01:25.851105 (+   156us) write_op.cc:276] Timestamp: P: 1783602085850885 usec, L: 0
0709 13:01:25.851148 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:01:25.851813 (+   665us) log.cc:844] Serialized 1696 byte log entry
0709 13:01:26.701474 (+849661us) op_driver.cc:464] REPLICATION: finished
0709 13:01:27.947728 (+1246254us) write_op.cc:301] APPLY: starting
0709 13:01:27.962112 (+ 14384us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:28.031659 (+ 69547us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:28.074722 (+ 43063us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:28.275186 (+200464us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:28.276142 (+   956us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:01:28.276233 (+    91us) write_op.cc:312] APPLY: finished
0709 13:01:28.354428 (+ 78195us) log.cc:844] Serialized 41 byte log entry
0709 13:01:28.490568 (+136140us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:28.497924 (+  7356us) write_op.cc:454] Released schema lock
0709 13:01:28.506328 (+  8404us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:28.533327 (+ 26999us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:28.533360 (+    33us) write_op.cc:454] Released schema lock
0709 13:01:28.774023 (+240663us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6b32a 00007fc833cf0f14 00007fc833cf0ec3 00007fc833cf0e38 00007fc833cf0df3 00007fc833cf00dc 00007fc833d19e16 00007fc833d01cbb 00007fc833cec39d 00007fc833cefd75
Metrics: {"mutex_wait_us":274900,"reported_tablets":10,"spinlock_wait_cycles":157384448,"child_traces":[["op",{"apply.queue_time_us":1164982,"apply.run_cpu_time_us":19909,"apply.run_wall_time_us":608908,"num_ops":4,"prepare.queue_time_us":116897,"prepare.run_cpu_time_us":21604,"prepare.run_wall_time_us":94708,"raft.queue_time_us":325473,"raft.run_cpu_time_us":37349,"raft.run_wall_time_us":2081318,"replication_time_us":850167,"spinlock_wait_cycles":12387200,"thread_start_us":1286786,"threads_started":5,"wal-append.queue_time_us":478134,"wal-append.run_cpu_time_us":39100,"wal-append.run_wall_time_us":2403919}]]}
I20260709 13:01:31.361562  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:31.287040  3411 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.225274  3450 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.292785  3473 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.242610  3190 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.156s
I20260709 13:01:31.429461  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.308732  3173 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.331408  3365 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:31.320180  3416 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.424805  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63392.8us
I20260709 13:01:31.408207  1474 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:31.412573  3482 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:31.430452  1574 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 1.
I20260709 13:01:31.293810  3500 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:31.348973  3395 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:31.308208  3396 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.237454  3497 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.182s
I20260709 13:01:31.432132  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.397742  3423 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.432924  1646 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:31.459555  1719 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:31.435921  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.502188  1575 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:31.503104  3118 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.503544  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.377689  1716 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:31.436933  1717 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.295918  3305 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:01:31.506214  3476 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.455693  3173 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.472410  3365 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:31.453576  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.476147  3484 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.511132  1572 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:31.511351  3484 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.512517  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.513263  1572 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:31.515255  3484 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.348786  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 943) took 2941 ms. Trace:
I20260709 13:01:31.516654  1569 rpcz_store.cc:276] 0709 13:01:28.406951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.390520 (+983569us) service_pool.cc:224] Handling call
0709 13:01:29.391304 (+   784us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.391662 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.391692 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:29.392152 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:01:29.955069 (+562917us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b180022ace0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:01:30.141203 (+186134us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:30.141258 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.141387 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.933633 (+792246us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:31.256612 (+322979us) raft_consensus.cc:1685] finished
0709 13:01:31.287535 (+ 30923us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.345304 (+ 57769us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4077,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":33006080,"thread_start_us":184210,"threads_started":1,"wal-append.queue_time_us":813596}
I20260709 13:01:31.518970  3423 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.519497  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.520577  3494 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:31.521953  1569 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 2: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:31.487260  3500 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:31.525925  3500 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.523456  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.527488  1569 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:31.478106  3395 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.528798  3395 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:31.529567  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.532444  1716 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:31.533136  3402 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.533862  3358 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.534646  3502 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:31.535208  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.458972  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.535831  3496 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.536324  1567 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:31.536437  3496 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.538153  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 123) took 2066 ms. Trace:
I20260709 13:01:31.538262  1572 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:31.538595  1567 rpcz_store.cc:276] 0709 13:01:29.471350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.537164 (+2065814us) service_pool.cc:224] Handling call
0709 13:01:31.537540 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.537743 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.537776 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.537832 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:31.537880 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.537945 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.538082 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.539562  3496 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.540197  1572 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.540846  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 124) took 2069 ms. Trace:
I20260709 13:01:31.541131  1572 rpcz_store.cc:276] 0709 13:01:29.471594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.539664 (+2068070us) service_pool.cc:224] Handling call
0709 13:01:31.540045 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.540632 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.540797 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.473824  3501 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:31.429898  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.549028  1572 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:31.471257  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 957) took 2912 ms. Trace:
I20260709 13:01:31.549937  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 975) took 1963 ms. Trace:
I20260709 13:01:31.406060  3495 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:31.543761  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.465173  3498 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.474762  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:31.571790  1574 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:31.434763  3483 raft_consensus.cc:2764] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:31.294662  1568 rpcz_store.cc:276] 0709 13:01:28.495231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.635967 (+2140736us) service_pool.cc:224] Handling call
0709 13:01:30.708962 (+ 72995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.112106 (+403144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.126582 (+ 14476us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.450798  3403 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:31.482995  1570 raft_consensus.cc:2376] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4. Re-sending same reply.
I20260709 13:01:31.352043  3487 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.314805  3481 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:31.476328  1691 connection.cc:256] RPC call timeout handler was delayed by -1.056836345000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:01:31.492515  3406 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.004s
I20260709 13:01:31.504401  1575 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:31.509627  3365 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.400527  3366 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:31.550103  1576 rpcz_store.cc:276] 0709 13:01:28.558719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.117380 (+2558661us) service_pool.cc:224] Handling call
0709 13:01:31.126946 (+  9566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.127195 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:31.144634 (+ 17439us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.470818 (+326184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.470888 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:31.470950 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.471032 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.471192 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17166,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":131,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3963,"thread_start_us":325762,"threads_started":1}
I20260709 13:01:31.520623  3310 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:31.419370  3411 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.574751  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:31.470413  3416 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:31.589578  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 101533us
I20260709 13:01:31.550446  1572 rpcz_store.cc:276] 0709 13:01:29.586231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.541834 (+1955603us) service_pool.cc:224] Handling call
0709 13:01:31.542194 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.549602 (+  7408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.549867 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.543136  3501 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.631422  3501 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:31.623646  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.632675  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.624272  1725 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:31.624038  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49566.4us
I20260709 13:01:31.594321  3305 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:01:31.465453  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 365308us
I20260709 13:01:31.634567  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:31.524420  3442 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.697603  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1580) took 1923 ms. Trace:
I20260709 13:01:31.572453  3476 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:31.669502  1571 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:31.734375  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 976) took 1890 ms. Trace:
W20260709 13:01:31.744655  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.010s	sys 0.019s
I20260709 13:01:31.633436  1716 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:31.745113  1575 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:31.789750  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1587) took 1952 ms. Trace:
I20260709 13:01:31.790194  1575 rpcz_store.cc:276] 0709 13:01:29.838839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.722561 (+1883722us) service_pool.cc:224] Handling call
0709 13:01:31.744951 (+ 22390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.789407 (+ 44456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.789672 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.760149  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.760495  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.586494  3423 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:31.650938  3396 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:31.771552  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 137098us
I20260709 13:01:31.582026  3495 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:31.633533  1574 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:31.586828  1567 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:31.772500  1651 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:31.805124  1571 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.745767  1576 rpcz_store.cc:276] 0709 13:01:29.774585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.668068 (+1893483us) service_pool.cc:224] Handling call
0709 13:01:31.696911 (+ 28843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.697122 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.697162 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.697219 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:31.697271 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.697344 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.697520 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.760818  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.749095  2377 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:31.672559  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.808621  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.672439  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:31.814117  1724 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:31.791447  1720 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 1.
I20260709 13:01:31.623438  3165 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:31.770913  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 497909us
I20260709 13:01:31.777253  3507 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:31.796928  1570 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:31.635385  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.825938  1723 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:31.799705  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.826978  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:31.622305  3406 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.801636  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:31.805953  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1588) took 1951 ms. Trace:
W20260709 13:01:31.807334  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 384675us
I20260709 13:01:31.807377  1721 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:31.831898  1571 rpcz_store.cc:276] 0709 13:01:29.854713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.804578 (+1949865us) service_pool.cc:224] Handling call
0709 13:01:31.804950 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.805718 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.805894 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:31.755225  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96390.3us
W20260709 13:01:31.844873  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.011s	sys 0.019s
W20260709 13:01:31.845652  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 711192us
I20260709 13:01:31.591434  3403 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.626428  3118 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:31.811620  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.682057  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:31.728587  3481 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.727322  3173 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.817349  1651 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:01:31.870886  1651 pending_rounds.cc:85] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 1
I20260709 13:01:31.791652  1572 rpcz_store.cc:276] 0709 13:01:29.843606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.733030 (+1889424us) service_pool.cc:224] Handling call
0709 13:01:31.733520 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.733752 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.733789 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.733843 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:31.733892 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.733960 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.734256 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.810730  2375 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:31.871268  1651 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:31.872611  1651 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:31.873600  1572 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:31.774662  3487 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:31.784575  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:31.750233  3195 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:01:31.859984  3396 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:31.851825  1690 leader_election.cc:400] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5.
I20260709 13:01:31.793638  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.775506  3476 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:31.669188  3483 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.145s
I20260709 13:01:31.801697  3347 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.860507  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:31.860700  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.854898  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:31.880853  1575 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:31.882320  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1597) took 1988 ms. Trace:
I20260709 13:01:31.823910  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:31.584874  3498 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:31.853425  3504 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.859771  3473 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 13:01:31.798982  3423 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.865900  3403 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:31.916808  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 132384us
I20260709 13:01:31.852476  3450 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:31.681695  3482 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.083343  1724 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:32.105901  1572 rpcz_store.cc:276] 0709 13:01:29.893933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.872215 (+1978282us) service_pool.cc:224] Handling call
0709 13:01:31.873353 (+  1138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.874062 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.882251 (+  8189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.135583  1572 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:32.136598  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 132) took 2211 ms. Trace:
I20260709 13:01:32.136997  1572 rpcz_store.cc:276] 0709 13:01:29.925019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.134999 (+2209980us) service_pool.cc:224] Handling call
0709 13:01:32.135281 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.135933 (+   652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.136543 (+   610us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.137693  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 282868us
I20260709 13:01:31.914423  3173 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:32.107555  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1591) took 2215 ms. Trace:
I20260709 13:01:32.143322  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.078128  3357 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:31.867983  3484 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:01:31.860988  3475 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 13:01:32.109561  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 955) took 3551 ms. Trace:
I20260709 13:01:32.158581  1573 rpcz_store.cc:276] 0709 13:01:28.558064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.110401 (+2552337us) service_pool.cc:224] Handling call
0709 13:01:31.113702 (+  3301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.181663 (+ 67961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.181699 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.182297 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.743733 (+561436us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:31.743786 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.875930 (+132144us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b5400310bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:31.876012 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.074563 (+198551us) raft_consensus.cc:1685] finished
0709 13:01:32.074597 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.109483 (+ 34886us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5981,"raft.queue_time_us":14779,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":141606272,"thread_start_us":500022,"threads_started":1,"wal-append.queue_time_us":638138}
I20260709 13:01:32.119503  3423 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:32.104067  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.885134  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:32.127997  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 280826us
I20260709 13:01:32.138149  1572 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:32.106541  1644 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:32.106704  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278734us
I20260709 13:01:31.963992  1642 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:31.913933  3310 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.277050  1642 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:32.277329  3310 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.278267  1642 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.279521  1642 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:31.871191  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 127) took 2394 ms. Trace:
I20260709 13:01:32.280951  1567 rpcz_store.cc:276] 0709 13:01:29.473606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.539847 (+2066241us) service_pool.cc:224] Handling call
0709 13:01:31.540245 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.811509 (+271264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.868171 (+ 56662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.283089  1567 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:32.283941  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1603) took 1186 ms. Trace:
I20260709 13:01:32.143452  1575 rpcz_store.cc:276] 0709 13:01:29.891809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.814481 (+1922672us) service_pool.cc:224] Handling call
0709 13:01:31.867615 (+ 53134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.107203 (+239588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.107478 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.108369  1690 leader_election.cc:403] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:01:32.069842  3165 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.285974  3165 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:32.286676  3165 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.149258  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:31.964672  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:32.069437  3411 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.290338  3411 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:32.120250  3347 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.151113  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.109525  3348 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.294548  3310 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:32.153798  2379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.005s	sys 0.017s
I20260709 13:01:32.077292  3510 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:32.295171  2379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.017s
I20260709 13:01:32.296077  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 324) took 1160 ms. Trace:
W20260709 13:01:32.296271  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 471102us
I20260709 13:01:32.297260  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:32.297544  3511 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.429s	user 0.000s	sys 0.000s
I20260709 13:01:32.298070  3421 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:31.917271  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:32.225859  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.300154  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.304500  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.305351  1723 raft_consensus.cc:2376] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3. Re-sending same reply.
I20260709 13:01:32.291406  3411 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:32.264756  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:32.076985  3118 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:01:32.164053  3473 consensus_queue.cc:1243] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:32.270661  1572 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:32.290174  3523 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 13:01:32.275310  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 108933us
I20260709 13:01:32.293901  3348 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.116325  3512 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.146937  3402 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:32.188786  3357 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:32.143208  3173 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:32.072540  3495 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:31.862226  3507 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.295456  3510 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:32.380980  1725 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:32.259321  3450 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.333719  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.341522  1724 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:32.336954  3494 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:01:32.356715  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:32.356506  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 239474us
I20260709 13:01:32.351409  1572 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.333587  1568 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:32.347103  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 82459.3us
I20260709 13:01:32.118716  3504 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.396571  3510 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.352290  3524 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:32.398003  3504 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.342968  3487 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:32.296568  2379 rpcz_store.cc:276] 0709 13:01:31.135618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.135746 (+   128us) service_pool.cc:224] Handling call
0709 13:01:31.141782 (+  6036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.144101 (+  2319us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:31.568733 (+424632us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.295679 (+726946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:32.295733 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:32.295809 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.295880 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.296008 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3393,"thread_start_us":726515,"threads_started":1}
I20260709 13:01:32.366097  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.400825  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.402226  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.293579  1567 rpcz_store.cc:276] 0709 13:01:31.097889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.282340 (+1184451us) service_pool.cc:224] Handling call
0709 13:01:32.282827 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.283608 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.283871 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:32.351598  3508 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.528s	user 0.002s	sys 0.000s
I20260709 13:01:32.360706  3473 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:01:32.260322  3195 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.379173  1720 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:32.405090  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 131) took 2480 ms. Trace:
I20260709 13:01:32.405561  1568 rpcz_store.cc:276] 0709 13:01:29.924510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.130757 (+2206247us) service_pool.cc:224] Handling call
0709 13:01:32.263991 (+133234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.404815 (+140824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.405029 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.405598  3195 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.406844  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:32.407086  3507 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:32.407987  3195 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.382432  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:32.388528  1724 pending_rounds.cc:85] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 1
I20260709 13:01:32.413525  1724 pending_rounds.cc:107] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:32.413913  1724 raft_consensus.cc:2889] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:32.364219  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.293977  3475 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:32.367326  3520 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:01:32.394620  3529 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:32.395597  1572 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:32.403177  3521 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:01:32.404036  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:32.414376  3406 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.470640  3310 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.473477  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1583) took 2698 ms. Trace:
I20260709 13:01:32.473639  3310 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.473958  1570 rpcz_store.cc:276] 0709 13:01:29.775492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.700905 (+1925413us) service_pool.cc:224] Handling call
0709 13:01:31.795255 (+ 94350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.824787 (+ 29532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.473226 (+648439us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b540022d8d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:32.473405 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215966976}
I20260709 13:01:32.473654  3529 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:32.475726  3529 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:32.476001  3506 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.476545  3506 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.476682  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:32.453980  3165 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.476765  3529 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.478756  3529 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:32.384686  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:32.479303  3357 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:32.409451  1720 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:32.409721  1568 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:32.480283  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.410351  3498 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:32.368424  3173 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:32.481148  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.482198  1720 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:32.362195  3512 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.484413  3411 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.466728  3416 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.486238  3512 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:32.471701  3395 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.942s	user 0.010s	sys 0.000s
I20260709 13:01:32.487466  3395 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.487890  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 325) took 1348 ms. Trace:
I20260709 13:01:32.488397  2375 rpcz_store.cc:276] 0709 13:01:31.139399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.141122 (+  1723us) service_pool.cc:224] Handling call
0709 13:01:31.141670 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.884905 (+743235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.487588 (+602683us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54005e0e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:32.487796 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461292416}
I20260709 13:01:32.489691  3395 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:32.490867  3347 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:32.351046  3481 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:01:32.443409  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.490754  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.493718  3442 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.483004  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.495482  1720 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:32.481245  1725 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:32.499401  3310 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:32.500722  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:32.501446  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:32.502169  1571 raft_consensus.cc:2376] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 3. Re-sending same reply.
I20260709 13:01:32.502737  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:32.503432  1571 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:32.505462  3173 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.503770  1570 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:32.506827  3506 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:32.508701  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.503998  3504 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:01:32.509460  1721 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:32.509981  3416 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:32.510977  3416 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:32.511389  3506 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:32.504884  1567 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:32.511812  3506 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:32.512205  3310 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:32.513151  3310 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:32.513477  3506 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.514310  3476 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.515618  3476 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:32.515983  3506 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:32.494619  1572 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:32.516983  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:32.535279  1572 pending_rounds.cc:85] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 1
I20260709 13:01:32.516326  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:32.535619  1572 pending_rounds.cc:107] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:32.536242  1572 raft_consensus.cc:2889] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:32.519371  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:32.449167  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66887.6us
I20260709 13:01:32.537341  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1599) took 1520 ms. Trace:
I20260709 13:01:32.537883  1572 rpcz_store.cc:276] 0709 13:01:31.016730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.137768 (+1121038us) service_pool.cc:224] Handling call
0709 13:01:32.138053 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.537049 (+398996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.537278 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:32.535702  1719 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:32.519462  1614 connection.cc:256] RPC call timeout handler was delayed by -1.6383977499999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 13:01:32.542994  1642 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:32.538926  1651 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:32.514812  3310 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.508453  3173 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:32.729423  3521 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:01:32.550566  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.733830  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.735286  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:32.653677  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 297381us
I20260709 13:01:32.583907  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:32.555341  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 121) took 3084 ms. Trace:
I20260709 13:01:32.739880  1569 rpcz_store.cc:276] 0709 13:01:29.470629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.536119 (+2065490us) service_pool.cc:224] Handling call
0709 13:01:31.536702 (+   583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.584876 (+ 48174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.584910 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.590279 (+  5369us) log.cc:844] Serialized 26 byte log entry
0709 13:01:32.086103 (+495824us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:32.554628 (+468525us) log.cc:844] Serialized 14 byte log entry
0709 13:01:32.554908 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.554998 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:32.555060 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:32.555079 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:32.555257 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47678,"raft.queue_time_us":50516,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":42972,"thread_start_us":728944,"threads_started":2,"wal-append.queue_time_us":604428}
W20260709 13:01:32.658043  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76893.6us
W20260709 13:01:32.469002  3514 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:01:32.758351  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.753176  1723 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:32.754612  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:32.728757  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 357888us
I20260709 13:01:32.762897  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:32.765039  1541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.044s
I20260709 13:01:32.518445  3348 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:32.582867  3005 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 13:01:32.766263  1541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.002s	sys 0.044s
W20260709 13:01:32.643723  3501 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.012s	user 0.008s	sys 0.004s
W20260709 13:01:32.793653  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 236198us
I20260709 13:01:32.795440  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:32.797111  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:32.659364  1719 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:32.728955  3526 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:32.772856  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:32.774941  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.777139  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 148396us
W20260709 13:01:32.801539  2342 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.526s (SENT)
I20260709 13:01:32.414608  3450 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:32.802461  2342 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.544s (SENT)
I20260709 13:01:32.428129  3475 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:32.805111  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 265218us
I20260709 13:01:32.447180  3507 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:32.765331  1646 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:32.848146  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:32.862218  1576 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:32.847878  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.864112  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:32.850050  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.848619  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:32.865456  1572 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:32.846105  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:32.846190  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 73463.3us
I20260709 13:01:32.868810  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:32.869454  1569 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:01:32.865068  1614 connection.cc:256] RPC call timeout handler was delayed by -1.3467977499999506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 13:01:32.851576  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49133.4us
W20260709 13:01:32.870754  2342 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.959s (SENT) [suppressed 33 similar messages]
W20260709 13:01:32.766938  3396 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.014s	sys 0.009s
I20260709 13:01:32.447793  3528 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:32.517215  3190 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:32.717603  3421 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5.
I20260709 13:01:32.860313  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:32.868899  1717 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:32.852841  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48905.3us
W20260709 13:01:32.809265  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.776893  3484 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.007s	sys 0.007s
W20260709 13:01:32.883075  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.894865  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.802598  3502 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:32.582165  3411 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:32.894400  1572 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:32.931893  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:32.930673  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:32.929698  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.792032  3310 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:32.876672  3396 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.014s	sys 0.010s
I20260709 13:01:32.802529  3526 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:32.809842  3348 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:32.946532  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139855us
I20260709 13:01:32.662554  3242 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.214s
I20260709 13:01:32.792537  3416 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 13:01:32.516861  3366 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.807811  3365 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:32.955324  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 58132.8us
I20260709 13:01:32.855233  3513 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:32.958992  1722 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:32.896588  3528 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.957149  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:32.808827  3395 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.140s
I20260709 13:01:32.808502  3501 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:32.931623  3524 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:32.809054  3195 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.145s
I20260709 13:01:32.896909  3483 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:32.997246  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:33.024507  1719 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:32.938587  3118 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.954819  3531 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.876586  3525 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:32.935748  3190 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:32.808288  3500 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.140s
I20260709 13:01:32.997939  3482 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:32.869916  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:32.986485  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 55932.9us
I20260709 13:01:32.956302  3396 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:32.868518  3494 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.119s
I20260709 13:01:32.899407  1718 raft_consensus.cc:1813] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Handling vote request from an unknown peer 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:32.808943  3165 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.150s
W20260709 13:01:33.029178  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97384.3us
I20260709 13:01:33.049203  3473 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:33.036612  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1600) took 2002 ms. Trace:
I20260709 13:01:33.037225  1644 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:01:33.043808   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:33.050972  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93977.6us
I20260709 13:01:32.868280  3173 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.119s
I20260709 13:01:33.058013  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:33.060499  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:33.061728  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:33.113945  1644 pending_rounds.cc:85] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 4
I20260709 13:01:33.186918  1644 pending_rounds.cc:107] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:01:33.186815  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:33.187455  1644 raft_consensus.cc:2889] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:33.064422  3525 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.062374  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:33.001329  1567 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:32.987766  3544 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.063684  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:33.044667  3403 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.905s	user 0.008s	sys 0.003s
I20260709 13:01:33.009963  3348 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:33.010983  3551 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:33.084640  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:33.044564  3366 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:33.031836  3524 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:32.979537  3507 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:01:33.088102  3411 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:33.092105  3546 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:33.098348  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:33.100328  1646 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:33.100962  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:33.101866  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:33.036732  3357 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:01:33.051121  1568 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:33.054446  3242 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:33.242264  3242 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:33.242628  3547 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:01:33.243273  3242 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.244408  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:33.102965  3442 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.000s
I20260709 13:01:33.103500  3005 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:33.095311  3481 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:33.108872  3190 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.055240  3526 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.042322  3118 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.110822  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 139) took 1253 ms. Trace:
I20260709 13:01:33.112246  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1604) took 2003 ms. Trace:
I20260709 13:01:33.112308  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:32.861516  3523 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.141s
I20260709 13:01:33.256528  3546 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:32.997080  3402 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:01:33.189114  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:33.120015  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:33.120621  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1689) took 1531 ms. Trace:
I20260709 13:01:33.016609  3365 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:33.150116  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.193744  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.684s (SENT)
I20260709 13:01:33.113867  1573 rpcz_store.cc:276] 0709 13:01:31.034446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.161156 (+1126710us) service_pool.cc:224] Handling call
0709 13:01:32.261186 (+100030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.261488 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:32.584388 (+322900us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.036201 (+451813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.036254 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:33.036312 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.036413 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.036551 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":803339,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":7455,"thread_start_us":773782,"threads_started":3}
I20260709 13:01:33.120137  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.121809  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.064281  3502 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:33.138870  2342 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.907s (SENT)
W20260709 13:01:33.203966  1651 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.004s
I20260709 13:01:33.069936  3482 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:33.212798  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1608) took 1866 ms. Trace:
I20260709 13:01:33.064102  3421 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 13:01:33.095175  3527 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:33.036413  3483 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.210719  1716 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:33.030545  3475 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.237349  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:33.220826  3403 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.226011  3524 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:33.227450  1718 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:33.255182  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.255681  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:33.103892  3534 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:01:33.235272  2371 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.052688  3531 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.113771  3473 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:33.308982  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.256052  3395 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:33.312983  1643 rpcz_store.cc:276] 0709 13:01:31.857596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.867398 (+  9802us) service_pool.cc:224] Handling call
0709 13:01:31.917817 (+ 50419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.965646 (+ 47829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.965686 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.966360 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:01:32.626032 (+659672us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:32.626130 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.626405 (+   275us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:33.110527 (+484122us) raft_consensus.cc:1685] finished
0709 13:01:33.110557 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.110746 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47332,"raft.queue_time_us":367562,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"thread_start_us":486092,"threads_started":1,"wal-append.queue_time_us":997231}
I20260709 13:01:33.313122  1575 rpcz_store.cc:276] 0709 13:01:31.109361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.284764 (+1175403us) service_pool.cc:224] Handling call
0709 13:01:32.285191 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.030590 (+745399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:33.030647 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:33.045635 (+ 14988us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:33.045716 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.112025 (+ 66309us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b540030a054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:33.112074 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.112190 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5174,"raft.queue_time_us":316551,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":20613,"spinlock_wait_cycles":1423270784,"thread_start_us":101656,"threads_started":1}
I20260709 13:01:33.245635  3562 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:01:33.038535  3528 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:33.316018  1648 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:33.303627  3450 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:33.260671  3481 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:33.292228  3504 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.220633  3544 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.260890  3396 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.231674  3551 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:33.427385  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.442183  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.444439  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.416735  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:33.410491  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.445705  1721 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:33.450376  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.404251  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.443723  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.451695  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:33.452915  1651 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.009s	sys 0.005s
I20260709 13:01:33.452800  1721 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:33.453817   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:33.454057  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.454016  1574 rpcz_store.cc:276] 0709 13:01:31.346457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.294638 (+948181us) service_pool.cc:224] Handling call
0709 13:01:32.295088 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.295379 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.295408 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:32.295930 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:01:32.622999 (+327069us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:32.623061 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.623180 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:33.212500 (+589320us) raft_consensus.cc:1685] finished
0709 13:01:33.212531 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.212715 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":326678,"threads_started":1,"wal-append.queue_time_us":740284}
I20260709 13:01:33.444237  1717 rpcz_store.cc:276] 0709 13:01:31.589066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.589402 (+   336us) service_pool.cc:224] Handling call
0709 13:01:31.632770 (+ 43368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:32.931056 (+1298286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.116115 (+185059us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400372b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:33.120539 (+  4424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2842616192}
W20260709 13:01:33.452813  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 314017us
W20260709 13:01:33.447350  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 253642us
W20260709 13:01:33.458388  2342 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.973s (SENT)
W20260709 13:01:33.375526  3541 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.264s	user 0.002s	sys 0.000s
I20260709 13:01:33.451717  1642 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
W20260709 13:01:33.486636  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.460017  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.295414  3357 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.469477  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:33.425040  3549 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:01:33.473831  1721 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:33.375262  3423 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.125s	user 0.008s	sys 0.004s
I20260709 13:01:33.486500  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.749105  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:33.459599  1569 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 13:01:33.346666  2372 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:33.719096  3558 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.384567  3190 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:33.719782  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.480172  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1709) took 1124 ms. Trace:
I20260709 13:01:33.390632  3495 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.721180  3365 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:33.721246  3544 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:33.417188  3173 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.721813  3450 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.477926  3504 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.374230  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.722137  3483 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:33.722500  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.349223  3501 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:33.722892  3403 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:33.368271  3526 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:33.723003  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.854813  1725 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:33.474161  3005 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:33.460963  3525 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:33.446097  3395 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:33.724056  3561 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:33.841439  3473 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.417096  3204 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.841923  3365 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:01:33.819422  3165 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:33.725103  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.725461  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 137) took 1868 ms. Trace:
I20260709 13:01:33.726102  3484 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:33.468895  3411 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.726769  3572 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:33.832801  3495 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.752974  3357 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.465695  3482 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:33.376667  3348 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:33.728572  3506 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.212s	user 0.015s	sys 0.005s
I20260709 13:01:33.910151  1573 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:33.390666  3513 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.659498  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 422201us
I20260709 13:01:33.548815  3481 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:33.416545  3548 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:01:33.731887  3118 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:33.732084  3531 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:33.426057  3546 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.445842  3527 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:33.728518  3310 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.010s
I20260709 13:01:33.870935  3484 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:33.734129  3551 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:33.464390  3416 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.453856  3502 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.734776  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.734932  1645 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:01:33.737030  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.315053  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 359702us
I20260709 13:01:33.866086  3528 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.737347  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:33.744457  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:33.745473  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:33.747056  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:33.750087  1569 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:33.957428  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1620) took 1110 ms. Trace:
I20260709 13:01:33.957868  1569 rpcz_store.cc:276] 0709 13:01:32.847171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.848274 (+  1103us) service_pool.cc:224] Handling call
0709 13:01:33.957364 (+1109090us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:33.719817  3476 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.204s	user 0.008s	sys 0.010s
I20260709 13:01:33.754135  1644 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:33.754329  3423 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.756183  3558 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.792035  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:33.793337  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:33.833145  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 114655us
W20260709 13:01:33.961495  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.945266  3481 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:33.794184  3510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.037s	sys 0.053s
I20260709 13:01:33.860304  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.003321  1724 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:33.730791  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.909763  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:33.910131  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.005716  1725 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:33.840552  3570 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.850934  3574 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:01:34.009914  1649 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:33.868798  1647 rpcz_store.cc:276] 0709 13:01:31.856887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.858035 (+  1148us) service_pool.cc:224] Handling call
0709 13:01:31.858452 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.131266 (+272814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.131299 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:32.144798 (+ 13499us) log.cc:844] Serialized 26 byte log entry
0709 13:01:32.538812 (+394014us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:32.538885 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.538976 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:33.725267 (+1186291us) raft_consensus.cc:1685] finished
0709 13:01:33.725287 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.725398 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":426535,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":553035,"threads_started":2,"wal-append.queue_time_us":683140}
I20260709 13:01:33.925242  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:33.864711  3561 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:33.938812  2378 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:33.939831  1574 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:34.011183  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.757081  1716 rpcz_store.cc:276] 0709 13:01:32.349585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.351730 (+  2145us) service_pool.cc:224] Handling call
0709 13:01:32.845774 (+494044us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:33.473903 (+628129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6378240}
I20260709 13:01:34.012612  1642 raft_consensus.cc:2368] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4.
I20260709 13:01:33.912557  3475 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:34.004455  3510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.037s	sys 0.054s
I20260709 13:01:33.538347  3515 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:33.956799  3568 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.844740  3173 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:33.744529  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.867123  3165 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.064173  3165 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:33.956864  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.959218  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:34.065109  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 106026us
I20260709 13:01:33.957373  1576 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:33.987689  1570 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:33.838902  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:33.989555  3507 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:33.946796  3563 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.994809  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.866341  3204 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.864624  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:33.864940  3395 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.006752  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:33.937471  3501 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:01:34.008610  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.958614  3502 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:34.011063  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 85909.7us
I20260709 13:01:33.848229  3450 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.012090  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:34.014211  1650 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:33.867534  1646 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:33.867754  3005 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.997398  3396 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.996735  3506 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:01:33.958472  3416 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.923797  3513 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:33.867336  3524 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:34.007315  3310 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:01:33.914029  2371 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:34.018330  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.020892  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.054181  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.155414  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.156071  1569 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:34.156729  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.158586  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.057935  1643 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:01:34.015890  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.062197  3173 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.073523  1647 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:34.075150  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59126.5us
W20260709 13:01:34.078228  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 239419us
I20260709 13:01:34.167428  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:34.168354  1571 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:34.171972  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:34.172648  1569 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.133004  3501 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:34.081033  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 144) took 1522 ms. Trace:
I20260709 13:01:34.178784  1651 rpcz_store.cc:276] 0709 13:01:32.558634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.558850 (+   216us) service_pool.cc:224] Handling call
0709 13:01:32.559329 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.625413 (+ 66084us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:33.453438 (+828025us) log.cc:844] Serialized 14 byte log entry
0709 13:01:33.683439 (+230001us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.727988 (+ 44549us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.728132 (+   144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:33.728164 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.728223 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.080824 (+352601us) raft_consensus.cc:1685] finished
0709 13:01:34.080847 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.080978 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65450,"prepare.queue_time_us":1095505,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":101173,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":24397,"thread_start_us":1021168,"threads_started":2,"wal-append.queue_time_us":270040}
I20260709 13:01:34.180749  1576 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:34.181746  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.182199  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.120587  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 983) took 1022 ms. Trace:
I20260709 13:01:34.183353  1650 rpcz_store.cc:276] 0709 13:01:33.097970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.006539 (+908569us) service_pool.cc:224] Handling call
0709 13:01:34.014046 (+  7507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.120350 (+106304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.120531 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:34.118332  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 113825us
I20260709 13:01:34.056777  3515 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.014720  3570 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.015620  3365 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:01:34.132009  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.182518  1651 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:34.017238  3579 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:01:34.064785  3580 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:34.141485  2380 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:34.065353  3568 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:34.148171  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80556.6us
I20260709 13:01:34.215395  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.215761  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.216759  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:34.217619  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.129436  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:34.081710  3476 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:34.118700  3584 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:34.008421  3532 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.131s
I20260709 13:01:34.174890  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 984) took 1076 ms. Trace:
I20260709 13:01:34.236701  1643 rpcz_store.cc:276] 0709 13:01:33.098241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.006555 (+908314us) service_pool.cc:224] Handling call
0709 13:01:34.057571 (+ 51016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.160255 (+102684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.174766 (+ 14511us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.175532  1648 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:34.184108  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.186833  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:34.198724  1717 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:34.238504  1576 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:34.018288  3483 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:01:34.148763  3562 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:34.145933  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.239302  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:34.239650  3562 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:34.240471  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.240674  3562 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.241598  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.242245  1717 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:34.210116  1574 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:34.081146  3531 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:01:34.213212  1575 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:34.243729  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.244362  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.245092  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.140544  3403 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:34.178526  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.246074  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.075560  3575 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 13:01:34.247495  3403 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.248046  1576 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:34.015447  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.248832  3575 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:34.220860  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.250079  1573 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:34.199744  3366 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.968s	user 0.019s	sys 0.000s
I20260709 13:01:34.222918  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:34.252828  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:34.225095  1569 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:34.057612  3484 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.232648  1571 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:34.140075  3585 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:01:34.234588  1651 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:34.178418  3501 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.257727  1575 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:34.242342  3562 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:34.178527  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 242) took 1115 ms. Trace:
I20260709 13:01:34.266426  2371 rpcz_store.cc:276] 0709 13:01:33.057379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.057582 (+   203us) service_pool.cc:224] Handling call
0709 13:01:34.173176 (+1115594us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.221284  3242 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:34.254616  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.255479  1570 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:34.267562  3242 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.269203  3594 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:34.255496  1571 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:34.261878  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.270625  3570 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.271134  1725 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:34.262727  1574 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 3: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:34.259408  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72677.6us
W20260709 13:01:34.223984  3577 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:01:34.018560  3527 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.250236  3575 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.267982  1651 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:34.263042  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:34.284526  1574 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:34.284915  1574 pending_rounds.cc:85] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 1
I20260709 13:01:34.284986  1569 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.285298  1574 pending_rounds.cc:107] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:34.285574  1574 raft_consensus.cc:2889] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.286608  1576 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:34.286845  1725 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:34.287622  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1576) took 1302 ms. Trace:
I20260709 13:01:34.274982  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:34.140265  3396 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.291425  1569 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:34.291718  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.224025  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.278870  1570 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:34.262079  3442 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.301465  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 135118us
I20260709 13:01:34.271662  3118 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:34.288014  1725 rpcz_store.cc:276] 0709 13:01:32.984965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.280373 (+1295408us) service_pool.cc:224] Handling call
0709 13:01:34.286639 (+  6266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.287348 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.287566 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.288626  1571 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:34.275965  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 111786us
I20260709 13:01:34.293088  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:34.294004  3484 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.294564  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.258800  3580 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.081486  3563 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.304235  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:34.327538  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:34.274484  3528 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.337435  1725 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:34.338580  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1584) took 1114 ms. Trace:
I20260709 13:01:34.339177  1725 rpcz_store.cc:276] 0709 13:01:33.223982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.307972 (+1083990us) service_pool.cc:224] Handling call
0709 13:01:34.314729 (+  6757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.338231 (+ 23502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.338504 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.329492  1571 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:34.333587  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:34.275053  3495 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:34.283675  3584 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:34.344271  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:34.347958  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 330251us
I20260709 13:01:34.304773  3527 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.307665  3118 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.313350  3570 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.284705  3365 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:01:34.307114  3423 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.327718  3242 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.266952  3501 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:34.349020  3190 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.000s
I20260709 13:01:34.313464  3442 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:34.382508  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55100.1us
I20260709 13:01:34.384785  3348 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:34.386649  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 384612us
W20260709 13:01:34.393373  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1659282440000425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
I20260709 13:01:34.350711  3497 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:34.396585  1539 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.668s (SENT)
I20260709 13:01:34.350044  3357 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.374749  3118 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.295562  3575 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.409283  3396 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.382745  3476 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:34.383040  3591 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:34.434404  3582 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:01:34.402283  3501 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:34.351961  3589 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:34.378120  3495 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:34.355077  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.399577  1716 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:34.402436  3204 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.464450  3416 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.394073  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:34.486603  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94457.7us
I20260709 13:01:34.398185  3348 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.387349  3483 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
W20260709 13:01:34.459165  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 108605us
I20260709 13:01:34.490330  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:34.460886  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.462805  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0770737490000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
I20260709 13:01:34.396649  3497 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:34.402243  3310 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.399016  3551 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.290081  3515 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:34.480652  3572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.000s	sys 0.012s
I20260709 13:01:34.414193  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.523792  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.524641  1646 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:34.525399  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.525957  1646 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:34.526758  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.527446  1646 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:34.434897  3476 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.528561  3476 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:34.529047  3575 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.435711  3591 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:34.532747  3591 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.533869  3591 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:34.438651  3450 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:34.475958  2342 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.527s (SENT)
W20260709 13:01:34.537314  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 61402.8us
W20260709 13:01:34.537981  2342 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 2.193s (SENT)
W20260709 13:01:34.538723  2342 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 2.344s (SENT)
I20260709 13:01:34.539086  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:34.384601  3570 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:34.540967  1651 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:01:34.476187  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 172616us
I20260709 13:01:34.400832  3190 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:01:34.403016  3584 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:34.483420  1691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.018s	sys 0.025s
I20260709 13:01:34.459617  1568 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:34.487802  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:34.545601  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58010.1us
I20260709 13:01:34.437127  3580 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.491891  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.473901  3589 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.402225  3524 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.479126  3513 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:34.507215  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.507906  3593 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:01:34.509234  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 995) took 1304 ms. Trace:
I20260709 13:01:34.512698  1725 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:34.569082  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1585) took 1338 ms. Trace:
I20260709 13:01:34.569427  1725 rpcz_store.cc:276] 0709 13:01:33.230962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.340230 (+1109268us) service_pool.cc:224] Handling call
0709 13:01:34.512440 (+172210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.568534 (+ 56094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.569025 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.504436  3551 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.494357  3348 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.495082  3495 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:34.515836  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.576355  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.565564  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.494863  3481 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:34.542526  1691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.018s	sys 0.025s
W20260709 13:01:34.582094  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.502024  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1581) took 1402 ms. Trace:
I20260709 13:01:34.445314  3592 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 13:01:34.582757  1716 rpcz_store.cc:276] 0709 13:01:33.094512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.306892 (+1212380us) service_pool.cc:224] Handling call
0709 13:01:34.355274 (+ 48382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.480358 (+125084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.496738 (+ 16380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.546371  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:34.584403  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:34.585808  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.586059  3603 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:34.586480  3603 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.586790  3603 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:34.492802  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0714587350000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:01:34.587821  1539 connection.cc:256] RPC call timeout handler was delayed by -1.8676942960000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:01:34.588146  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2508737490000499s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2577s for a response
W20260709 13:01:34.588429  1539 connection.cc:256] RPC call timeout handler was delayed by -1.080558735000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32970000000000005s for a response
I20260709 13:01:34.590111  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.590935  3005 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:34.555469  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.561348  3396 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:34.555984  3563 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:34.567051  2342 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.889s (SENT)
I20260709 13:01:34.596788  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:34.487933  3561 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.501377  3310 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.528182  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:34.565002  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73208.5us
I20260709 13:01:34.567682  1647 rpcz_store.cc:276] 0709 13:01:33.205119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.014230 (+809111us) service_pool.cc:224] Handling call
0709 13:01:34.019577 (+  5347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.163207 (+143630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.508877 (+345670us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b54001bc1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:34.509167 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114621696}
I20260709 13:01:34.542626  3572 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:34.505584  3581 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
W20260709 13:01:34.563907  3525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.025s
W20260709 13:01:34.582484  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.510877  3558 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.593703  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.572119  3513 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:34.601100  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.580406  3165 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:34.619701  3513 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:34.622700  1647 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:34.627648  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 996) took 1404 ms. Trace:
I20260709 13:01:34.612968  3495 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.631908  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64902us
W20260709 13:01:34.616707  3525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.016s	sys 0.026s
I20260709 13:01:34.638025  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:34.621218  1646 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:34.622313  3561 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:34.625566  3513 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:34.619604  3481 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:01:34.608424  3421 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:01:34.603986  3396 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:34.651409  3596 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.168s	user 0.001s	sys 0.001s
W20260709 13:01:34.295184  3544 consensus_queue.cc:1175] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f737473248e4c059fa74af3b8e1f77e" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:34.626293  3507 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:01:34.652415  3608 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:34.650547  2342 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 2.047s (SENT)
I20260709 13:01:34.625838  3005 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:01:34.627321  3165 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.623898  3411 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:01:34.643190  3190 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:34.635582  1647 rpcz_store.cc:276] 0709 13:01:33.222911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.622134 (+1399223us) service_pool.cc:224] Handling call
0709 13:01:34.622528 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.627168 (+  4640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.627568 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.787288  1725 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:34.721765  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.788206  1647 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:34.789149  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1006) took 1059 ms. Trace:
I20260709 13:01:34.789534  1647 rpcz_store.cc:276] 0709 13:01:33.729647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.787196 (+1057549us) service_pool.cc:224] Handling call
0709 13:01:34.787696 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.788764 (+  1068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.789062 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.719228  1573 raft_consensus.cc:1217] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 13:01:34.790957  1573 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:34.731588  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 138) took 2874 ms. Trace:
I20260709 13:01:34.732323  1644 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:01:34.737217  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.644505  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.835443  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.743626  1646 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:34.743430  3411 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.674504  3475 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.619s	user 0.005s	sys 0.004s
I20260709 13:01:34.659138  3204 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:34.665418  3421 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:34.745846  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1720) took 2334 ms (client timeout 2181 ms). Trace:
I20260709 13:01:34.652941  3507 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:34.684793  3595 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:01:34.666620  3597 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:34.667768  3600 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:34.682651  3603 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.676889  3609 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:34.660642  3572 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:34.665349  3601 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.160s	user 0.000s	sys 0.002s
I20260709 13:01:34.614931  3589 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.669816  3310 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.641520  3495 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.756369  3616 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:34.757457  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.758646  3195 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.747995  1723 raft_consensus.cc:2955] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:01:34.774912  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.775162  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 153435us
I20260709 13:01:34.775918  3561 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:34.780447  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.775826  1750 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:32927 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:32927 timed out after 15.000s (SENT)
I20260709 13:01:34.779992  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1719) took 2368 ms. Trace:
I20260709 13:01:34.791533  1647 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:34.791991  1645 rpcz_store.cc:276] 0709 13:01:31.857274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.858943 (+  1669us) service_pool.cc:224] Handling call
0709 13:01:31.882771 (+ 23828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:33.951266 (+2068495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.730787 (+779521us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540005cdd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:34.731297 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3821867392}
I20260709 13:01:34.858141  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1008) took 1015 ms. Trace:
I20260709 13:01:34.858206  3421 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.858637  1647 rpcz_store.cc:276] 0709 13:01:33.842821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.790525 (+947704us) service_pool.cc:224] Handling call
0709 13:01:34.791011 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.857841 (+ 66830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.858093 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.860596  3204 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.860847  3165 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.861398  3204 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:34.810807  2342 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 2.191s (SENT)
I20260709 13:01:34.862697  3195 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:34.863969  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53493.9us
I20260709 13:01:34.812230  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1001) took 1517 ms. Trace:
I20260709 13:01:34.818991  1717 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:34.865890  3195 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:34.866119  3411 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.866732  2379 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
W20260709 13:01:34.842332  2378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 246) took 1773 ms (client timeout 1983 ms). Trace:
W20260709 13:01:34.867784  2378 rpcz_store.cc:269] 0709 13:01:33.068444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.068575 (+   131us) service_pool.cc:224] Handling call
0709 13:01:34.841720 (+1773145us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54006d1ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:34.842073 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137308032}
I20260709 13:01:34.868494  3507 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.840497  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:34.869977  3592 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.870191  3507 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.870663  3592 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.871958  3507 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:34.872845  3396 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:34.873363  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.873651  3396 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.874260  3396 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.822619  3442 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.831790  3523 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:34.876971  3523 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:34.857399  1724 rpcz_store.cc:276] 0709 13:01:32.411323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.492068 (+ 80745us) service_pool.cc:224] Handling call
0709 13:01:32.492440 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.003732 (+1511292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.779309 (+775577us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003dd6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:34.779707 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3455209216}
W20260709 13:01:34.835824  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 192978us
I20260709 13:01:34.872459  1644 rpcz_store.cc:276] 0709 13:01:33.294780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.727385 (+1432605us) service_pool.cc:224] Handling call
0709 13:01:34.728714 (+  1329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.794640 (+ 65926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.812125 (+ 17485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.880529  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 999) took 1601 ms. Trace:
I20260709 13:01:34.879185  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:34.881868  2377 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:34.882426  1724 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:34.883431  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.884737  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.886662  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:34.866037  3533 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:01:34.889678  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:01:34.889909  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 989) took 1790 ms (client timeout 2194 ms). Trace:
W20260709 13:01:34.890362  1648 rpcz_store.cc:269] 0709 13:01:33.099459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.443450 (+343991us) service_pool.cc:224] Handling call
0709 13:01:34.889547 (+1446097us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54003fcd54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:34.889736 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663782656}
I20260709 13:01:34.891166  1720 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:34.892774  3495 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:34.883924  3603 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:34.888128  3533 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:34.881150  1646 rpcz_store.cc:276] 0709 13:01:33.254350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.640087 (+1385737us) service_pool.cc:224] Handling call
0709 13:01:34.742970 (+102883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.851803 (+108833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.855619 (+  3816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:34.843405  3504 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:34.927156  3616 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:34.888201  3592 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:34.891990  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:34.928020  3406 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:34.936365  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:34.938179  1719 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:34.891883  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:34.918737  3533 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:34.878623  3523 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.859915  3421 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.951363  3421 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:34.952427  3609 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:34.952688  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60901.1us
I20260709 13:01:34.953864  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:34.956822  3531 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:34.957628  3531 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:34.959093  3531 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:34.960007  3532 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:34.956833  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70389us
I20260709 13:01:34.935443  3616 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.964035  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:34.962232  3584 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:34.852007  1718 rpcz_store.cc:269] 0709 13:01:32.411611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.472449 (+ 60838us) service_pool.cc:224] Handling call
0709 13:01:34.745224 (+2272775us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400232b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:34.745541 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892593408}
I20260709 13:01:34.955828  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:34.974113  1648 raft_consensus.cc:606] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:34.962589  1724 raft_consensus.cc:2376] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 2. Re-sending same reply.
I20260709 13:01:34.976953  3406 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:34.982275  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 97879.7us
I20260709 13:01:34.982856  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:01:34.984584  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.985399  3495 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:34.986263  3500 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:34.987383  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.960770  3396 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.988601  3532 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:34.994674  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:34.995385  3501 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:34.995819  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.974262  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:34.789455  3497 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.983314  3406 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:35.002518  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.000988  3495 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:34.949126  3411 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:34.981671  3584 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:35.009368  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53760.2us
I20260709 13:01:35.016130  3481 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:35.022938  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.025750  1643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:35.026525  1643 raft_consensus.cc:606] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:35.026999  1643 raft_consensus.cc:614] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:35.029211  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:34.974440  3616 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:34.998649  3533 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:35.042224  3118 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.637s	user 0.007s	sys 0.010s
W20260709 13:01:35.035286  2342 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.959s (SENT)
I20260709 13:01:35.044287  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.031550  3523 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:35.051438  3310 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.042750  3118 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:35.046743  3588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.011s	sys 0.011s
I20260709 13:01:35.040702  3531 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:35.063935  3557 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:35.065724  3588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.011s	sys 0.011s
I20260709 13:01:35.012980  3495 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:35.068325  3497 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.098609  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:35.099175  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 102659us
I20260709 13:01:35.069802  3557 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:35.104429  3562 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.009s
W20260709 13:01:35.117100  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.121003  3118 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:35.124083  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1580) took 2029 ms (client timeout 2064 ms). Trace:
W20260709 13:01:35.124605  1720 rpcz_store.cc:269] 0709 13:01:33.094141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.746745 (+652604us) service_pool.cc:224] Handling call
0709 13:01:35.123520 (+1376775us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400372154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.123820 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156342016}
I20260709 13:01:35.126652  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:35.128818  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.129381  3620 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:01:35.131670  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 104315us
I20260709 13:01:35.132459  1724 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:35.112290  3580 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:01:35.136209  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.103533  3557 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.133066  3533 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.147396  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:35.152631  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.046324  3584 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.165649  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:35.166977  3411 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:35.148516  1648 raft_consensus.cc:606] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:35.170258  1648 raft_consensus.cc:614] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:35.173601  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1007) took 1443 ms (client timeout 1792 ms). Trace:
W20260709 13:01:35.174634  3624 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:01:35.174194  1642 rpcz_store.cc:269] 0709 13:01:33.730147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.748811 (+ 18664us) service_pool.cc:224] Handling call
0709 13:01:35.173007 (+1424196us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000f55d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.173393 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604184320}
I20260709 13:01:35.176399  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 148) took 1102 ms. Trace:
I20260709 13:01:35.176880  1650 rpcz_store.cc:276] 0709 13:01:34.073352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.817649 (+744297us) service_pool.cc:224] Handling call
0709 13:01:34.818347 (+   698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.818990 (+   643us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.819022 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:34.819739 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.920597 (+100858us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:34.920660 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.920753 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.175907 (+255154us) raft_consensus.cc:1685] finished
0709 13:01:35.175946 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.176127 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":826,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":203,"thread_start_us":91025,"threads_started":1,"wal-append.queue_time_us":210850}
W20260709 13:01:35.179498  3482 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.250s	user 0.010s	sys 0.010s
I20260709 13:01:35.108073  3531 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:35.188135  1720 raft_consensus.cc:2376] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 5. Re-sending same reply.
I20260709 13:01:35.155710  3572 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:01:35.157238  3527 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.675s	user 0.018s	sys 0.000s
I20260709 13:01:35.104513  3348 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:35.168067  3513 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.513s	user 0.000s	sys 0.015s
I20260709 13:01:35.099820  3609 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:01:35.221335  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:35.227312  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81242.9us
I20260709 13:01:35.228241  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:35.229128  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.229056  1724 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:35.233525  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 100691us
I20260709 13:01:35.235325  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:35.230692  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:35.241616  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.241009  1647 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.241192  3204 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:35.250239  3204 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:35.250603  1642 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:35.228969  3118 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.296581  1648 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:35.301505  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:35.308723  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54299.8us
I20260709 13:01:35.185274  3482 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.303045  3562 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:01:35.239792  3513 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.323073  1650 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:35.326113  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 152) took 1195 ms. Trace:
W20260709 13:01:35.327243  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 139433us
I20260709 13:01:35.337453  1720 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:35.341308  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:35.348915  3502 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.096s	user 0.007s	sys 0.012s
I20260709 13:01:35.349658  3502 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.352593  1722 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:35.357024  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 193632us
W20260709 13:01:35.357990  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.328380  1645 rpcz_store.cc:276] 0709 13:01:34.131099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.858174 (+727075us) service_pool.cc:224] Handling call
0709 13:01:34.858697 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.859159 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:34.938225 (+ 79066us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.321672 (+383447us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:35.321734 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:35.325495 (+  3761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.325644 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.325975 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21879,"prepare.queue_time_us":110121,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":2999,"raft.queue_time_us":13130,"thread_start_us":392865,"threads_started":2}
I20260709 13:01:35.356385  3496 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:35.329605  3118 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.379911  1647 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:35.379822  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.387657  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 250) took 1660 ms. Trace:
I20260709 13:01:35.388267  2380 rpcz_store.cc:276] 0709 13:01:33.717813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.717950 (+   137us) service_pool.cc:224] Handling call
0709 13:01:33.718232 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.212715 (+494483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.368025 (+1155310us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54005d25d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:35.378523 (+ 10498us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972674816}
I20260709 13:01:35.389942  3395 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:35.390471  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47181.8us
I20260709 13:01:35.347084  3421 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:01:35.399727  3636 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:01:35.400951  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.402485  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.403640  1724 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:35.404856  3348 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:35.406024  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1601) took 1473 ms. Trace:
I20260709 13:01:35.406491  1716 rpcz_store.cc:276] 0709 13:01:33.912574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.583603 (+671029us) service_pool.cc:224] Handling call
0709 13:01:34.583974 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:34.584437 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.584471 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:34.585097 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 13:01:35.239249 (+654152us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18003ebea0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1
0709 13:01:35.339342 (+100093us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:35.346506 (+  7164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.356066 (+  9560us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.356142 (+    76us) raft_consensus.cc:1685] finished
0709 13:01:35.356154 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.386194 (+ 30040us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11144,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":8761,"spinlock_wait_cycles":6086016,"thread_start_us":11755,"threads_started":1,"wal-append.queue_time_us":43339}
I20260709 13:01:35.408666  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.409617  1720 raft_consensus.cc:2376] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 5. Re-sending same reply.
I20260709 13:01:35.410735  3570 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:35.411327  1720 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:35.411571  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.412460  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.413511  3640 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:01:35.417497  3513 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:35.372223  3572 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:01:35.421177  3476 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.892s	user 0.011s	sys 0.005s
I20260709 13:01:35.421761  3476 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.422371  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.423275  3421 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:35.423487  1573 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:35.423839  3421 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:35.425586  1575 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:35.425190  3421 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.426703  3421 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:35.426873  3633 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:01:35.429203  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:35.431457  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1700) took 1175 ms. Trace:
I20260709 13:01:35.431692  3526 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:35.432734  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.435191  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:35.436791  1650 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:35.390023  3551 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:35.438753  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.431989  1646 rpcz_store.cc:276] 0709 13:01:34.256506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.919341 (+662835us) service_pool.cc:224] Handling call
0709 13:01:34.997102 (+ 77761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.025743 (+ 28641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.025790 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.026620 (+   830us) log.cc:844] Serialized 26 byte log entry
0709 13:01:35.086998 (+ 60378us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:35.087050 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.087132 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.425603 (+338471us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:35.428188 (+  2585us) raft_consensus.cc:1685] finished
0709 13:01:35.428211 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.431370 (+  3159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1863,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":21935,"spinlock_wait_cycles":8077696,"thread_start_us":46653,"threads_started":1,"wal-append.queue_time_us":210190}
I20260709 13:01:35.440598  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.440894  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.441751  1567 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:35.435719  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 329) took 1477 ms. Trace:
I20260709 13:01:35.443269  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.444377  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.435779  3403 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:35.445259  1569 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:35.443596  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.447002  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.447994  1575 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:35.449180  3482 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:35.447062  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 338) took 1452 ms. Trace:
I20260709 13:01:35.450039  2377 rpcz_store.cc:276] 0709 13:01:33.994632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.995194 (+   562us) service_pool.cc:224] Handling call
0709 13:01:34.839996 (+844802us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:35.446872 (+606876us) spinlock_profiling.cc:243] Waited 9.69 ms on lock 0x7b5400730354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.446994 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25757312}
I20260709 13:01:35.450893  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:35.451190  1716 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:35.449070  3515 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:35.416091  3590 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.032s	user 0.000s	sys 0.003s
I20260709 13:01:35.412096  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1603) took 1498 ms. Trace:
W20260709 13:01:35.453631  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.477249  1688 leader_election.cc:400] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4.
I20260709 13:01:35.477689  1688 leader_election.cc:403] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:35.477877  3118 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:35.478605  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.479501  3526 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:35.471835  3572 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 13:01:35.481357  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1578) took 2452 ms (client timeout 1640 ms). Trace:
W20260709 13:01:35.481956  1722 rpcz_store.cc:269] 0709 13:01:33.028592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.121490 (+ 92898us) service_pool.cc:224] Handling call
0709 13:01:35.481011 (+2359521us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b5400294fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.481271 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4387790208}
I20260709 13:01:35.483055  1570 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:35.483613  1722 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:01:35.422534  3635 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:01:35.484690  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.484869  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.485978  2371 raft_consensus.cc:2410] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:35.480384  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.488178  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:35.463153  1721 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:35.452198  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:35.469898  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 91773.3us
I20260709 13:01:35.471756  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1579) took 2397 ms. Trace:
I20260709 13:01:35.474459  1720 rpcz_store.cc:276] 0709 13:01:33.913228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.410516 (+1497288us) service_pool.cc:224] Handling call
0709 13:01:35.411005 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.411769 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.411983 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:35.485881  1567 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:35.496470  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.497095  1722 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:35.498493  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.499925  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.491072  1717 rpcz_store.cc:276] 0709 13:01:33.076579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.262577 (+1185998us) service_pool.cc:224] Handling call
0709 13:01:35.452066 (+1189489us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b540010e0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.471678 (+ 19612us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210182016}
I20260709 13:01:35.500836  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1607) took 1586 ms. Trace:
I20260709 13:01:35.501363  1716 rpcz_store.cc:276] 0709 13:01:33.914411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.450225 (+1535814us) service_pool.cc:224] Handling call
0709 13:01:35.450714 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.469404 (+ 18690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.500722 (+ 31318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:35.502863  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.503494  1716 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:35.506285  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.508250  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.510854  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.512980  1573 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.501401  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.519268  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.520722  1575 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:35.521967  3476 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:35.522250  3572 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.520491  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.523959  1716 raft_consensus.cc:2368] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:01:35.529654  1567 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:35.531337  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1599) took 1687 ms. Trace:
I20260709 13:01:35.444288  2372 rpcz_store.cc:276] 0709 13:01:33.958126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.960823 (+  2697us) service_pool.cc:224] Handling call
0709 13:01:35.412055 (+1451232us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:35.434589 (+ 22534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.435077 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.435110 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:35.435214 (+   104us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:35.435267 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.435348 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.435508 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7852416}
I20260709 13:01:35.534737  3562 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:35.529625  3531 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:35.531801  1725 rpcz_store.cc:276] 0709 13:01:33.844260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.570310 (+726050us) service_pool.cc:224] Handling call
0709 13:01:34.570595 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.788207 (+217612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.531018 (+742811us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400087354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:35.531254 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437854080}
I20260709 13:01:35.546783  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:35.547997  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60008.6us
I20260709 13:01:35.523697  3476 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:35.548766  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:35.557565  1720 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:35.537657  3609 raft_consensus.cc:2764] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:35.565593  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.555212  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:35.570777  1570 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:35.580577  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.582187  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.583010  1725 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:35.584159  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.555917  3482 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:35.556180  3476 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:35.584542  3495 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:35.556656  3310 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:35.604799  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.605366  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.558144  3242 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:01:35.612640  1643 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.617111  1725 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:35.617775  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:35.619434  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.622032  3568 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:35.623364  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1626) took 1879 ms. Trace:
W20260709 13:01:35.623699  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.624574  3502 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:35.624956  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1574) took 2673 ms. Trace:
I20260709 13:01:35.625082  3562 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.620103  3523 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.639776  3495 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.623775  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 993) took 2505 ms. Trace:
I20260709 13:01:35.650132  1573 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:35.650693  1723 rpcz_store.cc:276] 0709 13:01:32.951872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.952018 (+   146us) service_pool.cc:224] Handling call
0709 13:01:34.746629 (+1794611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.747419 (+   790us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:34.971230 (+223811us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.068658 (+ 97428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:35.069111 (+   453us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:35.069160 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.073750 (+  4590us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.624739 (+550989us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400232b50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.624879 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4243,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":16661,"spinlock_wait_cycles":4126474112,"thread_start_us":55255,"threads_started":1,"wal-append.queue_time_us":135675}
I20260709 13:01:35.658507  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.658378  3523 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.659451  1569 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.661067  3523 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:35.661983  3495 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.663460  3562 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.654767  1649 rpcz_store.cc:276] 0709 13:01:33.117843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.010748 (+892905us) service_pool.cc:224] Handling call
0709 13:01:34.011001 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:34.065551 (+ 54550us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:34.065824 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.450778 (+1384954us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b18004091e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:01:35.620660 (+169882us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:35.620772 (+   112us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:35.620902 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.623287 (+  2385us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b54003fd754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:35.623346 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.623536 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34602,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2271,"raft.queue_time_us":36773,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"spinlock_wait_cycles":142244352,"thread_start_us":114784,"threads_started":1}
I20260709 13:01:35.593369  3482 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.556279  3557 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.682730  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.685199  1720 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:35.685631  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.695355  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.695787  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.666872  3609 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:35.698990  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.666531  3526 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.647789  1572 rpcz_store.cc:276] 0709 13:01:33.744097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.744244 (+   147us) service_pool.cc:224] Handling call
0709 13:01:33.744631 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.148438 (+403807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.148468 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:34.148988 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.271718 (+122730us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:34.271755 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.811227 (+539472us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54003638d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:34.811309 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.414968 (+603659us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:35.622886 (+207918us) raft_consensus.cc:1685] finished
0709 13:01:35.622911 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.623135 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3309,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":22906,"spinlock_wait_cycles":933375104,"thread_start_us":117337,"threads_started":1,"wal-append.queue_time_us":176573}
I20260709 13:01:35.687023  3482 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.709347  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:35.711731  1720 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:35.711434  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.696990  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:35.720562  1573 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:01:35.719893  3476 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:35.722445  3476 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.700785  3609 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.686292  3562 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:35.738879  1573 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:35.783842  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.783784  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:35.739048  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.781698  1569 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:35.791869  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.793026  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:35.801941  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:35.743036  3495 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.804813  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:35.809084  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:35.817215  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.847074  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.736456  3609 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:35.880417  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.881227  1573 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:35.882709  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.889071  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 268726us
W20260709 13:01:35.890007  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.735426  3611 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.084s	user 0.002s	sys 0.002s
I20260709 13:01:35.901224  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:35.902241  1720 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:35.911803  1573 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [2.5-3.6]
I20260709 13:01:35.913281  1572 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:35.878572  3495 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:35.917593  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:35.919363  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:01:35.922907  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.932649  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.953662  1643 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
W20260709 13:01:35.963719  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.970095  1574 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:35.736955  3526 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:35.981269  1569 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:35.984863  1575 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:01:35.994861  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1577) took 2993 ms (client timeout 1788 ms). Trace:
W20260709 13:01:35.996167  1719 rpcz_store.cc:269] 0709 13:01:33.001729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.036163 (+ 34434us) service_pool.cc:224] Handling call
0709 13:01:35.968535 (+2932372us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400213754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:35.994585 (+ 26050us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3862743936}
I20260709 13:01:36.003592  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:36.017686  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81080.9us
W20260709 13:01:36.022083  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 102829us
I20260709 13:01:36.000833  3562 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:36.025019  1720 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:36.030182  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.036331  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.022251  3568 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:36.047578  3557 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:36.029264  3501 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.061574  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.064469  3575 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:36.064603  3585 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.064874  3310 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:36.045439  3616 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.088006  1569 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:36.070159  3575 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.100898  1724 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:36.108798  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1608) took 2194 ms. Trace:
I20260709 13:01:36.109293  1724 rpcz_store.cc:276] 0709 13:01:33.914709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.493516 (+1578807us) service_pool.cc:224] Handling call
0709 13:01:36.047893 (+554377us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:36.061380 (+ 13487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.108356 (+ 46976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.108722 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12031232}
I20260709 13:01:36.055544  3507 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:01:36.114936  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.116432  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.110836  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:36.122581  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.079181  3585 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:36.129868  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.071906  3501 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:36.114496  3575 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:36.144069  1643 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:01:36.148015  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 251186us
I20260709 13:01:36.087952  3616 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:36.085786  3310 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:36.161398  1724 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:36.189883  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:36.190371  1720 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:36.150004  2373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.020s	sys 0.010s
I20260709 13:01:36.209462  1569 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:36.214619  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:36.213795  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.010s	sys 0.017s
W20260709 13:01:36.219298  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.011s	sys 0.018s
W20260709 13:01:36.222465  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 427527us
I20260709 13:01:36.226780  1643 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:36.245854  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:36.246486  1647 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:36.246493  1646 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:36.211542  2373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.560s	user 0.021s	sys 0.011s
W20260709 13:01:36.255895  3643 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:01:36.255759  1642 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:36.256654  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:36.257467  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:36.257697  1642 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:36.257995  1643 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:36.258750  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:36.261075  1614 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.776s (SENT)
I20260709 13:01:36.261606  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:36.261775  3481 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:36.262293  3481 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:36.263331  3481 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:36.263839  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:36.265609  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:36.296707  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.301790  3403 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:36.308395  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:01:36.282236  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.299391  3572 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.002s
I20260709 13:01:36.323846  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.334722  1720 raft_consensus.cc:2376] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3. Re-sending same reply.
I20260709 13:01:36.315514  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:36.318270  3473 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:36.322263  3585 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:36.329245  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68289.3us
I20260709 13:01:36.307761  3403 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:36.331439  3653 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:01:36.332952  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.332895  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1604) took 2419 ms (client timeout 1553 ms). Trace:
I20260709 13:01:36.305317  3406 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:36.368584  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:36.367252  1721 rpcz_store.cc:269] 0709 13:01:33.913505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.971720 (+1058215us) service_pool.cc:224] Handling call
0709 13:01:36.332683 (+1360963us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400214154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:36.332820 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774183680}
I20260709 13:01:36.379494  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.379933  1643 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:36.314946  3481 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:36.382763  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.383597  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.350942  3572 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4.
I20260709 13:01:36.351250  3495 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:36.367713  3403 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:36.393321  1614 leader_election.cc:336] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.788s (SENT)
I20260709 13:01:36.394680  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:36.387763  1724 raft_consensus.cc:2410] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:36.396960  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.379142  3557 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:36.404218  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.405582  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:36.406651  3649 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:01:36.407196  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.407411  3528 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:36.407809  1721 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:36.407919  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:36.405615  3644 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.277s	user 0.000s	sys 0.002s
W20260709 13:01:36.409055  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100765us
I20260709 13:01:36.409926  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:36.410145  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.411144  1721 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:36.411404  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:36.412071  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.412243  3584 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:36.412998  3559 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:36.413398  3559 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:36.414433  3559 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:36.412541  3500 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:36.413401  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.419302  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.404892  1724 raft_consensus.cc:2376] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4. Re-sending same reply.
I20260709 13:01:36.384940  3616 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:36.420930  1569 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.427143  1572 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.435894  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.436611  1724 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:36.438319  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:36.404619  3557 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:36.438154  3500 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:01:36.442875  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:36.450430  1570 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:36.427196  3495 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:36.444780  1569 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:36.464563  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.446069  3584 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.446550  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.447436  1572 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:01:36.431371  3648 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.311s	user 0.001s	sys 0.001s
I20260709 13:01:36.449249  1719 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.451536  3557 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:36.465830  1724 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4.
I20260709 13:01:36.434559  3501 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:36.470347  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.445248  3575 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:36.475518  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.481669  1569 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:36.483132  1719 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:36.484467  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 99073.3us
I20260709 13:01:36.490557  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.467026  3395 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.497995  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 387347us
I20260709 13:01:36.558109  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:36.562567  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.564121  1724 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:36.565708  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.567595  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.470978  3616 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.575789  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:36.581651  1614 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.882s (SENT)
I20260709 13:01:36.484766  3562 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.559656  3482 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.806s	user 0.011s	sys 0.004s
I20260709 13:01:36.599275  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.568482  1719 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.601742  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:36.602411  1647 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
W20260709 13:01:36.698567  2344 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.028s	sys 0.054s
W20260709 13:01:36.584800  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.040s
W20260709 13:01:36.701552  2344 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.028s	sys 0.054s
I20260709 13:01:36.702262  1643 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:36.480423  3584 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:36.704393  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.027s	sys 0.023s
W20260709 13:01:36.711851  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.027s	sys 0.023s
I20260709 13:01:36.701151  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:36.603854  3476 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.881s	user 0.011s	sys 0.006s
I20260709 13:01:36.728204  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:36.709224  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.036s	sys 0.022s
W20260709 13:01:36.731319  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.037s	sys 0.023s
I20260709 13:01:36.731374  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:36.732728  1647 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:36.731859  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 575525us
W20260709 13:01:36.712683  2345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.024s	sys 0.028s
W20260709 13:01:36.737089  2345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.024s	sys 0.028s
W20260709 13:01:36.737658  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.708689  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 788157us
I20260709 13:01:36.737269  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:36.747834  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 178239us
I20260709 13:01:36.569922  3395 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:36.707253  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.001s	sys 0.040s
W20260709 13:01:36.714368  3649 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:01:36.754269  1723 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:36.705561  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.586622  3616 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:36.763499  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.764853  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:36.709996  3310 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:36.603443  3482 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:36.771807  1647 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:36.773432  1614 leader_election.cc:336] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.894s (SENT)
I20260709 13:01:36.739344  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 157) took 1959 ms. Trace:
I20260709 13:01:36.776958  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:36.761842  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.788501  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:36.709507  3562 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:36.800117  3656 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260709 13:01:36.801170  2378 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:36.806303  3526 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:36.791306  3655 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 13:01:36.850625  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:36.851677  2374 raft_consensus.cc:2410] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:36.851620  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1013) took 1430 ms. Trace:
I20260709 13:01:36.852965  2379 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:36.854336  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:36.807875  3584 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:36.808642  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1020) took 1199 ms. Trace:
I20260709 13:01:36.808887  3579 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:36.810446  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:36.813583  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1629) took 1185 ms (client timeout 1573 ms). Trace:
I20260709 13:01:36.852252  1568 rpcz_store.cc:276] 0709 13:01:35.421378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.426978 (+  5600us) service_pool.cc:224] Handling call
0709 13:01:35.427445 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.500655 (+ 73210us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:35.696466 (+195811us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.202736 (+506270us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.323227 (+120491us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.323437 (+   210us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:36.323479 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.323566 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.851264 (+527698us) raft_consensus.cc:1685] finished
0709 13:01:36.851291 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.851453 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45689,"prepare.queue_time_us":528610,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":7631,"raft.queue_time_us":77515,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":11273,"thread_start_us":584368,"threads_started":2,"wal-append.queue_time_us":577994}
I20260709 13:01:36.824321  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:36.866732  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2.
I20260709 13:01:36.829308  3572 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.835574  3562 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:36.837291  3616 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:36.837332  3585 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.780174  3476 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:36.839780  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 147) took 1256 ms. Trace:
W20260709 13:01:36.840757  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74775.2us
I20260709 13:01:36.842041  3482 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:36.844077  3395 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:36.846944  1646 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.000s
I20260709 13:01:36.893553  1646 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:36.777029  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:36.863679  1723 rpcz_store.cc:269] 0709 13:01:35.628739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.691042 (+ 62303us) service_pool.cc:224] Handling call
0709 13:01:36.813286 (+1122244us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:36.813494 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432370048}
I20260709 13:01:36.817574  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1649) took 1334 ms. Trace:
W20260709 13:01:36.865605  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.020s	sys 0.087s
W20260709 13:01:36.867792  1642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.026s	sys 0.051s
W20260709 13:01:36.896525  1642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.027s	sys 0.051s
I20260709 13:01:36.827895  1725 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.713212  3495 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:36.876076  1719 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
W20260709 13:01:36.877792  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.858639  1575 rpcz_store.cc:276] 0709 13:01:35.609329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.986472 (+377143us) service_pool.cc:224] Handling call
0709 13:01:36.009549 (+ 23077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.031433 (+ 21884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.031487 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.049329 (+ 17842us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.448740 (+399411us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:36.448796 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.448888 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.808424 (+359536us) raft_consensus.cc:1685] finished
0709 13:01:36.808444 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.808582 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14174,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":11559,"thread_start_us":363310,"threads_started":1,"wal-append.queue_time_us":733273}
W20260709 13:01:36.859794  2372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.029s	sys 0.074s
I20260709 13:01:36.876956  3579 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:36.887876  1573 rpcz_store.cc:276] 0709 13:01:35.583392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.913742 (+330350us) service_pool.cc:224] Handling call
0709 13:01:35.914629 (+   887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.009250 (+ 94621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.009287 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.041386 (+ 32099us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.432747 (+391361us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:36.432786 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.432858 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.839467 (+406609us) raft_consensus.cc:1685] finished
0709 13:01:36.839495 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.839703 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1895,"raft.queue_time_us":19714,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":16080,"thread_start_us":353820,"threads_started":1,"wal-append.queue_time_us":708767}
I20260709 13:01:36.897482  1570 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:36.897585  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.020s	sys 0.087s
I20260709 13:01:36.895183  1571 rpcz_store.cc:276] 0709 13:01:35.483590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.486762 (+  3172us) service_pool.cc:224] Handling call
0709 13:01:35.530725 (+ 43963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.531104 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.531143 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.532314 (+  1171us) log.cc:844] Serialized 26 byte log entry
0709 13:01:35.708625 (+176311us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:35.708668 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.708750 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.817335 (+1108585us) raft_consensus.cc:1685] finished
0709 13:01:36.817356 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.817513 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":175752,"threads_started":1,"wal-append.queue_time_us":509393}
I20260709 13:01:36.867307  3569 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:36.882666  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.891685  3575 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:36.917850  1614 leader_election.cc:336] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.952s (SENT)
I20260709 13:01:36.916885  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:36.902238  2372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.029s	sys 0.075s
I20260709 13:01:36.899355  3616 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.925108  3672 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.925665  3672 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:36.925709  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.927191  1723 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:36.926808  3557 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:36.927798  3672 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:36.928087  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.928745  1723 raft_consensus.cc:2376] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4. Re-sending same reply.
I20260709 13:01:36.929796  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:36.925665  3481 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.003s
I20260709 13:01:36.930680  1723 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:36.890161  3559 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:36.932593  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.932773  3642 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.582s	user 0.008s	sys 0.012s
I20260709 13:01:36.933692  3559 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:36.934655  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:36.935093  3557 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:36.935631  1723 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:36.936868  3589 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:36.937328  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.938918  1723 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 2: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:36.940618  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 381812us
W20260709 13:01:36.942847  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.943751  3592 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:36.953806  1615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.003s	sys 0.011s
W20260709 13:01:36.970697  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.971009  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.895823  3585 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:36.947649  1575 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:36.948372  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:36.949108  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:36.949201  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:36.903512  3572 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:36.949971  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:36.950366  1568 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:36.979532  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.899595  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 251) took 2539 ms. Trace:
I20260709 13:01:36.952981  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.984773  1719 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
W20260709 13:01:36.945411  3641 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.493s	user 0.009s	sys 0.012s
I20260709 13:01:36.972973  1724 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:36.977440  1572 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:36.973798  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:36.950035  1720 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:36.979316  1573 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:36.989065  1722 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:36.951174  2371 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:36.983856  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:36.975874  3609 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.031s	user 0.006s	sys 0.008s
W20260709 13:01:36.997390  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1733) took 1497 ms (client timeout 1900 ms). Trace:
W20260709 13:01:36.997941  1722 rpcz_store.cc:269] 0709 13:01:35.500071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.501092 (+  1021us) service_pool.cc:224] Handling call
0709 13:01:36.973632 (+1472540us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:36.997310 (+ 23678us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91943680}
I20260709 13:01:36.953924  3594 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.991760  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84657.7us
I20260709 13:01:36.982089  1570 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:36.986873  3671 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.001168  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.000571  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:36.953720  3616 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.008406  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:37.009899  3594 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.011355  3616 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:36.977789  3592 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:36.996915  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 342) took 1227 ms. Trace:
I20260709 13:01:37.012131  3594 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:37.069281  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 318427us
W20260709 13:01:37.024818  3507 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.004s	sys 0.029s
I20260709 13:01:37.076721  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:37.025241  3473 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.001s
W20260709 13:01:37.080291  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 380344us
W20260709 13:01:37.063946  3641 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:37.083240  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:37.001441  3609 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.086531  1722 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:37.089526  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:37.026738  3482 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:37.019207  3681 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:37.092552  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:37.101917  1645 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.102190  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:37.082551  3507 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.004s	sys 0.029s
I20260709 13:01:37.098026  1572 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
W20260709 13:01:37.097925  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.099244  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:37.072155  3616 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:37.019021  3481 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:01:37.103500  1649 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:37.128760  3406 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:37.129972  3609 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:36.984450  2373 rpcz_store.cc:276] 0709 13:01:34.356428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.384762 (+ 28334us) service_pool.cc:224] Handling call
0709 13:01:34.449209 (+ 64447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:34.485319 (+ 36110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.489087 (+  3768us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:34.641825 (+152738us) log.cc:844] Serialized 564 byte log entry
0709 13:01:35.631014 (+989189us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b
0709 13:01:36.366355 (+735341us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:36.382097 (+ 15742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.427838 (+ 45741us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.867232 (+439394us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b540057eca0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce3dbe 00007fc833cee930 00007fc833cee8d8 00007fc833cee858 00007fc833cee813 00007fc833ceda9c 00007fc839bd9626 00007fc839bd3dd8 00007fc833d2a7cb 00007fc833d2d2c0
0709 13:01:36.880169 (+ 12937us) raft_consensus.cc:1685] finished
0709 13:01:36.883155 (+  2986us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.893130 (+  9975us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47440512,"thread_start_us":1622144,"threads_started":1,"wal-append.queue_time_us":1492218}
W20260709 13:01:37.130715  3673 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:01:37.131378  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:37.131842  3664 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.132576  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:37.132428  3664 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.133633  3310 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.134174  3310 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.134925  3664 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:37.135526  3664 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.135872  3664 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:37.137286  3677 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:01:37.136652  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 150) took 1552 ms. Trace:
I20260709 13:01:37.023521  3579 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.137804  1574 rpcz_store.cc:276] 0709 13:01:35.584305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.983256 (+398951us) service_pool.cc:224] Handling call
0709 13:01:36.017469 (+ 34213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.043094 (+ 25625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.043140 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.079420 (+ 36280us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.596320 (+516900us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:36.596398 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.596496 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.136408 (+539912us) raft_consensus.cc:1685] finished
0709 13:01:37.136440 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.136578 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":821,"raft.queue_time_us":18305,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":13911,"thread_start_us":431332,"threads_started":1,"wal-append.queue_time_us":786210}
I20260709 13:01:37.016753  3671 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.136307  3664 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.139652  3669 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:01:37.092103  3603 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.088582  3476 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:37.136965  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:37.081156  3502 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:01:37.103844  1648 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:37.152570  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.126241  2342 connection.cc:256] RPC call timeout handler was delayed by -1.3137331439999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:01:37.170308  2342 connection.cc:256] RPC call timeout handler was delayed by -1.2903331439999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 13:01:37.103451  3575 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:37.126385  3562 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.140780  3669 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:37.137005  3572 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:37.136125  3310 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:37.166083  3681 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:37.185343  3608 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:37.162662  3406 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:37.184583  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52044us
W20260709 13:01:37.197058  3679 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:01:37.200004  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1614) took 1593 ms (client timeout 1894 ms). Trace:
W20260709 13:01:37.200544  1720 rpcz_store.cc:269] 0709 13:01:35.606466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.335549 (+729083us) service_pool.cc:224] Handling call
0709 13:01:37.199745 (+864196us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b540010ead4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:37.199877 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1272579328}
I20260709 13:01:37.202030  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.203404  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:37.204872  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68063.5us
I20260709 13:01:37.207808  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:36.786747  1716 rpcz_store.cc:276] 0709 13:01:34.751009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.524838 (+773829us) service_pool.cc:224] Handling call
0709 13:01:35.525287 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.525784 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.525820 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:35.526524 (+   704us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.017729 (+491205us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:36.550751 (+533022us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.583985 (+ 33234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.597680 (+ 13695us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.602398 (+  4718us) raft_consensus.cc:1685] finished
0709 13:01:36.610494 (+  8096us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.710165 (+ 99671us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":870484,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":198263,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":9389,"thread_start_us":880955,"threads_started":2,"wal-append.queue_time_us":691682}
I20260709 13:01:37.218982  3585 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:37.219595  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1702) took 1008 ms. Trace:
I20260709 13:01:37.221746  3568 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:37.181943  3473 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:01:37.258591  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:37.164996  3603 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.225934  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1019) took 1001 ms. Trace:
I20260709 13:01:37.204149  1720 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:37.250527  1650 rpcz_store.cc:276] 0709 13:01:36.211162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.211344 (+   182us) service_pool.cc:224] Handling call
0709 13:01:36.225222 (+ 13878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.305645 (+ 80423us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:01:36.305674 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.333586 (+ 27912us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.886974 (+553388us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:36.887038 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.887140 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.219301 (+332161us) raft_consensus.cc:1685] finished
0709 13:01:37.219326 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.219525 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":497312,"threads_started":1,"wal-append.queue_time_us":636147}
W20260709 13:01:37.255399  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59930.3us
W20260709 13:01:37.256093  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.260301  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52577.5us
I20260709 13:01:37.182431  3482 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:37.269721  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:37.183679  3669 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:37.265692  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 183983us
I20260709 13:01:37.272498  3568 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.276450  3658 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.168s
I20260709 13:01:37.249145  3310 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:37.278290  3403 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.858s	user 0.009s	sys 0.005s
I20260709 13:01:37.247385  3686 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
W20260709 13:01:37.292963  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63611.3us
I20260709 13:01:37.293236  3482 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.263764  3671 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:37.293844  3585 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:37.188966  3562 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.294205  3568 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:37.288520  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:37.263947  3575 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.296906  1645 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:37.298836  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:37.268743  3528 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:37.276450  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.227700  3681 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.304854  3403 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.308705  3528 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:37.321177  3528 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.266649  1651 rpcz_store.cc:276] 0709 13:01:36.188864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.189094 (+   230us) service_pool.cc:224] Handling call
0709 13:01:36.217568 (+ 28474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:36.345620 (+128052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.345672 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:36.377407 (+ 31735us) log.cc:844] Serialized 52 byte log entry
0709 13:01:36.891132 (+513725us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:37.094107 (+202975us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.094278 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.094325 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.189809 (+ 95484us) raft_consensus.cc:1685] finished
0709 13:01:37.189833 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.190004 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":290435,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":40819,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":61553,"thread_start_us":659800,"threads_started":2,"wal-append.queue_time_us":715263}
I20260709 13:01:37.323904  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.294983  3608 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:01:37.325476  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.326262  1716 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:37.308701  1720 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:37.327452  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.328066  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.301517  3572 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:01:37.328671  3669 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.329351  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.309666  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:37.330083  3669 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.311158  3674 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:01:37.337942  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:37.338707  1722 raft_consensus.cc:2376] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 2. Re-sending same reply.
I20260709 13:01:37.306147  1718 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:37.340101  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 140) took 2729 ms (client timeout 1907 ms). Trace:
I20260709 13:01:37.340374  1717 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:37.340816  1718 rpcz_store.cc:269] 0709 13:01:34.610317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.994095 (+383778us) service_pool.cc:224] Handling call
0709 13:01:37.339888 (+2345793us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54003dc2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:37.340037 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5297177472}
I20260709 13:01:37.342391  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.343845  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.311199  3687 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.314436  3562 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:37.392725  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 104276us
I20260709 13:01:37.393648  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:37.315232  3575 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:37.315379  3685 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:37.311340  3690 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:37.331233  1720 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:37.333386  3680 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.335786  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:37.337330  3592 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:37.313323  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:37.401479  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 131144us
I20260709 13:01:37.339643  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.403198  1722 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:37.331286  1716 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:37.348989  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:37.349779  3661 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.274s	user 0.000s	sys 0.003s
W20260709 13:01:37.404772  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 314983us
I20260709 13:01:37.357197  3403 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:37.357537  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.411064  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1642) took 1676 ms. Trace:
I20260709 13:01:37.411463  1723 rpcz_store.cc:276] 0709 13:01:35.734907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.357189 (+1622282us) service_pool.cc:224] Handling call
0709 13:01:37.410899 (+ 53710us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:37.359082  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1627) took 1731 ms (client timeout 1550 ms). Trace:
W20260709 13:01:37.412488  1725 rpcz_store.cc:269] 0709 13:01:35.627796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.654351 (+ 26555us) service_pool.cc:224] Handling call
0709 13:01:37.358873 (+1704522us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54003dccd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:37.358992 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1677965696}
W20260709 13:01:37.414824  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 101737us
I20260709 13:01:37.359539  3603 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:37.368954  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60991.3us
I20260709 13:01:37.374975  1721 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:37.400274  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64967.8us
I20260709 13:01:37.421005  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:37.399757  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:37.345788  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.405320  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1632) took 1775 ms. Trace:
I20260709 13:01:37.422873  1722 rpcz_store.cc:276] 0709 13:01:35.629850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.339368 (+1709518us) service_pool.cc:224] Handling call
0709 13:01:37.405247 (+ 65879us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.409914  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:37.400928  3685 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:37.424870  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:37.429994  1725 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:37.399768  3680 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:37.430047  1720 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:37.403904  3557 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.435827  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1626) took 1808 ms. Trace:
I20260709 13:01:37.438022  1719 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:37.440220  1720 rpcz_store.cc:276] 0709 13:01:35.627474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.397568 (+1770094us) service_pool.cc:224] Handling call
0709 13:01:37.435748 (+ 38180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.408288  3562 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:37.441817  1723 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:37.443277  1719 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:37.441143  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:37.452879  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 161) took 2452 ms. Trace:
I20260709 13:01:37.481066  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1628) took 1815 ms. Trace:
I20260709 13:01:37.427198  3572 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:37.482440  1719 rpcz_store.cc:276] 0709 13:01:35.628111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.440697 (+1812586us) service_pool.cc:224] Handling call
0709 13:01:37.441032 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.443658 (+  2626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.443865 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.487190  1719 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:37.487679  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1645) took 1751 ms. Trace:
I20260709 13:01:37.487876  1719 rpcz_store.cc:276] 0709 13:01:35.735922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.486522 (+1750600us) service_pool.cc:224] Handling call
0709 13:01:37.486894 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.487457 (+   563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.487631 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.422933  3608 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:37.426985  3658 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.446463  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:01:37.482321  1723 rpcz_store.cc:276] 0709 13:01:34.989766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.418610 (+2428844us) service_pool.cc:224] Handling call
0709 13:01:37.441683 (+ 23073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.442131 (+   448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.442272 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:37.496609  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96963.9us
I20260709 13:01:37.063722  2372 rpcz_store.cc:276] 0709 13:01:35.770057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.780748 (+ 10691us) service_pool.cc:224] Handling call
0709 13:01:35.921529 (+140781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.050744 (+129215us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:36.245410 (+194666us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.969785 (+724375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:36.975727 (+  5942us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:36.981356 (+  5629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.986248 (+  4892us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.996223 (+  9975us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9874,"prepare.queue_time_us":519342,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":57213,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":16290,"spinlock_wait_cycles":15273856,"thread_start_us":820693,"threads_started":2,"wal-append.queue_time_us":740479}
I20260709 13:01:37.498409  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1631) took 1868 ms. Trace:
I20260709 13:01:37.499241  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:37.484493  3559 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:37.442880  3403 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:37.440218  3557 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.485539  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78192.2us
I20260709 13:01:37.482834  3411 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:37.521909  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:37.522253  3411 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.439831  3680 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.522716  1574 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:37.512912  1718 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:37.503259  1720 rpcz_store.cc:276] 0709 13:01:35.629511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.441023 (+1811512us) service_pool.cc:224] Handling call
0709 13:01:37.489850 (+ 48827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.498062 (+  8212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.498091 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.498134 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:37.498166 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.498231 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.498331 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:37.503729  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 196149us
I20260709 13:01:37.427364  3609 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:01:37.491693  3572 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.511540  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:37.502202  3661 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:01:37.516983  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:37.518673  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:37.519636  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:37.536530  1576 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:37.538477  1572 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:37.520977  1568 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:37.521409  3557 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.524637  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:37.497293  3658 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:37.526355  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78632.5us
I20260709 13:01:37.520637  3559 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:37.541568  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:37.503504  3603 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:37.396311  3687 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.547514  1723 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:37.562623  1723 pending_rounds.cc:85] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 1
I20260709 13:01:37.549043  1571 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:37.549587  1720 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:37.583241  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1649) took 1605 ms. Trace:
I20260709 13:01:37.586879  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:37.589471  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:37.590240  3497 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:37.591105  3589 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:37.591755  3589 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.594254  3589 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:37.522722  3411 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:37.595969  3559 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.586967  3403 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.600834  3403 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:37.601423  1722 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:37.551335  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1728) took 2049 ms. Trace:
I20260709 13:01:37.603820  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:37.604698  1572 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:37.591248  3497 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:37.543301  3473 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:37.537869  3572 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:37.583829  1720 rpcz_store.cc:276] 0709 13:01:35.977658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.525876 (+1548218us) service_pool.cc:224] Handling call
0709 13:01:37.549259 (+ 23383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.582799 (+ 33540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.583147 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:37.623102  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.602610  1718 rpcz_store.cc:276] 0709 13:01:35.498237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.421064 (+1922827us) service_pool.cc:224] Handling call
0709 13:01:37.480992 (+ 59928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.532709 (+ 51717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.547546 (+ 14837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.590472  3698 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:37.542320  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.562988  1723 pending_rounds.cc:107] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:37.640185  1723 raft_consensus.cc:2889] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:37.499966  3608 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.534034  3562 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.601447  3403 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.644695  1569 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:01:37.637270  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 120405us
I20260709 13:01:37.496668  3568 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:01:37.548703  3658 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.626070  3497 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.651433  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72825.8us
I20260709 13:01:37.511195  3692 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.131s
I20260709 13:01:37.660846  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:37.663118  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1648) took 1726 ms. Trace:
I20260709 13:01:37.663708  1723 rpcz_store.cc:276] 0709 13:01:35.920717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.502899 (+1582182us) service_pool.cc:224] Handling call
0709 13:01:37.547113 (+ 44214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.647063 (+ 99950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.647326 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.664773  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:37.512790  1716 raft_consensus.cc:606] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:37.648002  3608 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:37.670593  3603 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:37.707281  3603 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.708660  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 162) took 2692 ms. Trace:
I20260709 13:01:37.709156  1725 rpcz_store.cc:276] 0709 13:01:34.990227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.417833 (+2427606us) service_pool.cc:224] Handling call
0709 13:01:37.418361 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.430741 (+ 12380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.681968 (+251227us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5400087854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:37.682210 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25693056}
I20260709 13:01:37.717593  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:37.718745  3697 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.288s	user 0.003s	sys 0.000s
I20260709 13:01:37.720953  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1623) took 2094 ms. Trace:
I20260709 13:01:37.721163  1723 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:37.721797  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1738) took 1150 ms. Trace:
I20260709 13:01:37.722158  1723 rpcz_store.cc:276] 0709 13:01:36.571085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.677715 (+1106630us) service_pool.cc:224] Handling call
0709 13:01:37.720809 (+ 43094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.721559 (+   750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.721742 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.723244  3674 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:37.724889  3700 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:01:37.721419  1722 rpcz_store.cc:276] 0709 13:01:35.626488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.423996 (+1797508us) service_pool.cc:224] Handling call
0709 13:01:37.433316 (+  9320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.602012 (+168696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.720703 (+118691us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54001a86d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:37.720886 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386104192}
I20260709 13:01:37.726437  1722 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:37.647564  3562 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.727813  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.624225  3588 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:01:37.659457  3497 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:37.659973  3671 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.744798  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1018) took 2189 ms. Trace:
I20260709 13:01:37.677928  3589 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:37.748732  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 249570us
I20260709 13:01:37.679911  3616 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:37.752694  1722 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:37.688737  1716 raft_consensus.cc:614] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:37.758386  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.764256  2380 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:37.659322  3575 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:01:37.659584  3585 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:01:37.743420  3562 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.685941  3406 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:01:37.762926  3616 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.635110  3685 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:37.799254  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.770344  3476 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.861992  2375 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:37.862460  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:37.863152  2377 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:37.875706  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.879739  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:37.882413  2375 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:37.883927  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:37.762362  3589 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.799460  3616 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:37.791209  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:37.824842  3704 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.934715  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:37.801460  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46374 (request call id 1) took 1296 ms. Trace:
I20260709 13:01:37.891373  1472 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 2 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.933626  3706 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:37.800048  3411 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:37.892740  3703 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.269s
I20260709 13:01:37.929543  3476 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:37.963500  3681 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.948412  1716 rpcz_store.cc:276] 0709 13:01:36.485713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.408332 (+922619us) service_pool.cc:224] Handling call
0709 13:01:37.768844 (+360512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.967641  3706 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:38.035140  1474 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 113) took 12272 ms (client timeout 15000 ms). Trace:
I20260709 13:01:38.003248  3496 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.017788  3685 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:38.020255  1472 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 2 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:38.060560  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1755) took 1274 ms. Trace:
I20260709 13:01:38.061084  1725 rpcz_store.cc:276] 0709 13:01:36.786514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.710232 (+923718us) service_pool.cc:224] Handling call
0709 13:01:37.710721 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.766830 (+ 56109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.766864 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.770223 (+  3359us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.952623 (+182400us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:37.952678 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.952769 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.060232 (+107463us) raft_consensus.cc:1685] finished
0709 13:01:38.060266 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.060480 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101200,"raft.queue_time_us":78528,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":101253,"thread_start_us":173777,"threads_started":1,"wal-append.queue_time_us":277238}
I20260709 13:01:38.063035  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:38.065029  3699 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.430s	user 0.003s	sys 0.000s
I20260709 13:01:38.067574  3528 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:38.068755  3707 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
W20260709 13:01:38.070899  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.071354  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1647) took 2256 ms. Trace:
I20260709 13:01:38.072032  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 153) took 3427 ms. Trace:
I20260709 13:01:38.072893  1717 rpcz_store.cc:276] 0709 13:01:34.644285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.518258 (+873973us) service_pool.cc:224] Handling call
0709 13:01:36.937911 (+1419653us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:37.340032 (+402121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.341035 (+  1003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.071813 (+730778us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b54003dc2d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:38.071966 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929481344}
I20260709 13:01:38.075068  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:38.075811  2377 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:38.078172  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.959282  3712 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 13:01:38.035017  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:38.085326  3719 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:01:38.071300  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1639) took 2337 ms (client timeout 2073 ms). Trace:
W20260709 13:01:38.088445  1721 rpcz_store.cc:269] 0709 13:01:35.733890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.419049 (+685159us) service_pool.cc:224] Handling call
0709 13:01:38.071049 (+1652000us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400224554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:38.071187 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100903680}
I20260709 13:01:38.072005  1719 rpcz_store.cc:276] 0709 13:01:35.815860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.488501 (+1672641us) service_pool.cc:224] Handling call
0709 13:01:37.497677 (+  9176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.506548 (+  8871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.506576 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.545657 (+ 39081us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.770118 (+224461us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:37.801690 (+ 31572us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.801970 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.802053 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.071099 (+269046us) raft_consensus.cc:1685] finished
0709 13:01:38.071124 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.071288 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27757,"prepare.run_wall_time_us":44,"raft.queue_time_us":72424,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8399,"spinlock_wait_cycles":96139904,"thread_start_us":117551,"threads_started":1,"wal-append.queue_time_us":65930}
I20260709 13:01:38.098896  1717 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:38.115880  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1652) took 1307 ms. Trace:
I20260709 13:01:37.952594  3704 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.116698  1717 rpcz_store.cc:276] 0709 13:01:36.814180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.093608 (+1279428us) service_pool.cc:224] Handling call
0709 13:01:38.094168 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.099581 (+  5413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.115781 (+ 16200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.031985  3710 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:37.749670  1567 rpcz_store.cc:276] 0709 13:01:35.555727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.557788 (+  2061us) service_pool.cc:224] Handling call
0709 13:01:35.567664 (+  9876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:35.580855 (+ 13191us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:35.993509 (+412654us) log.cc:844] Serialized 14 byte log entry
0709 13:01:37.090245 (+1096736us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.140703 (+ 50458us) log.cc:844] Serialized 26 byte log entry
0709 13:01:37.142964 (+  2261us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:37.164559 (+ 21595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.170213 (+  5654us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.349024 (+178811us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:37.699694 (+350670us) raft_consensus.cc:1685] finished
0709 13:01:37.744377 (+ 44683us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.744661 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28881,"prepare.queue_time_us":545390,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":6670,"raft.queue_time_us":22634,"raft.run_cpu_time_us":617,"raft.run_wall_time_us":67160,"spinlock_wait_cycles":6420352,"thread_start_us":1455280,"threads_started":3,"wal-append.queue_time_us":991766}
I20260709 13:01:38.055624  3687 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:38.122938  3669 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.790s	user 0.005s	sys 0.009s
I20260709 13:01:38.097487  3568 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.051664  3692 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:38.135586  1472 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 2 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:38.033450  3557 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.117846  3609 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:38.165580  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.123245  3709 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.047205  3681 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:38.186910  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 385062us
W20260709 13:01:38.188551  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 436185us
W20260709 13:01:38.189293  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 513828us
I20260709 13:01:38.056704  3496 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.215730  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:38.255297  1651 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:38.219189  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.255875  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.276317  1725 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:38.194522  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.993s (SENT) [suppressed 41 similar messages]
W20260709 13:01:38.280495  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64863.7us
I20260709 13:01:38.280730  3602 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:38.281756  3602 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:38.282199  3726 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.216614  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:38.280579  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 86127.4us
W20260709 13:01:38.292215  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 644305us
W20260709 13:01:38.294984  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.183718  3709 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.173130  3710 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.302515  3557 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:38.303555  3709 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:38.303683  3710 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:38.305109  1567 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:38.306639  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:38.308650  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:38.309401  1575 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:38.245222  3664 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.076s	user 0.017s	sys 0.000s
I20260709 13:01:38.315897  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:38.321959  1650 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:38.322885  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:38.322791  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:38.324164  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 107660us
I20260709 13:01:38.327301  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.328544  1649 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:38.170625  3609 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.328096  1644 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:38.274544  3589 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:38.331296  3709 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:38.336987  3709 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.299737  3708 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.254128  3585 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:01:38.353621  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.340734  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.308669  1472 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 2 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:38.175482  3568 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.308205  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:38.373986  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:38.374598  3720 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:01:38.376909  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:38.405874  1643 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:38.254395  3692 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.329442  3664 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.340457  3710 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:38.387148  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:38.282786  3602 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.340274  3709 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:38.390376  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55393.4us
W20260709 13:01:38.042797  1474 rpcz_store.cc:269] 0709 13:01:25.680973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.700549 (+ 19576us) service_pool.cc:224] Handling call
0709 13:01:37.896913 (+12196364us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:32.179585 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:32.399382 (+219797us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:32.399420 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:01:32.399437 (+    17us) tablet.cc:662] Decoding operations
0709 13:01:32.400130 (+   693us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:32.400194 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:32.400221 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:01:32.400799 (+   578us) tablet.cc:701] Row locks acquired
0709 13:01:32.400814 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:01:32.400974 (+   160us) write_op.cc:270] Start()
0709 13:01:32.401097 (+   123us) write_op.cc:276] Timestamp: P: 1783602092400925 usec, L: 0
0709 13:01:32.401115 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:01:32.401930 (+   815us) log.cc:844] Serialized 2069 byte log entry
0709 13:01:34.198837 (+1796907us) op_driver.cc:464] REPLICATION: finished
0709 13:01:34.481334 (+282497us) write_op.cc:301] APPLY: starting
0709 13:01:34.486307 (+  4973us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:34.504941 (+ 18634us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:34.543245 (+ 38304us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:34.668651 (+125406us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:34.684500 (+ 15849us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:01:34.688355 (+  3855us) write_op.cc:312] APPLY: finished
0709 13:01:34.688864 (+   509us) log.cc:844] Serialized 47 byte log entry
0709 13:01:34.689019 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:34.689122 (+   103us) write_op.cc:454] Released schema lock
0709 13:01:34.689167 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:34.689534 (+   367us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:34.689560 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1082504,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":233154,"apply.run_cpu_time_us":4393,"apply.run_wall_time_us":213724,"num_ops":5,"prepare.queue_time_us":621411,"prepare.run_cpu_time_us":37062,"prepare.run_wall_time_us":670198,"raft.queue_time_us":212138,"raft.run_cpu_time_us":22247,"raft.run_wall_time_us":277610,"replication_time_us":1746352,"thread_start_us":922453,"threads_started":4,"wal-append.queue_time_us":686857,"wal-append.run_cpu_time_us":65688,"wal-append.run_wall_time_us":2688231}]]}
I20260709 13:01:38.391777  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1651) took 2353 ms. Trace:
I20260709 13:01:38.376904  3557 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:38.273018  3718 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
W20260709 13:01:38.398667  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102403us
I20260709 13:01:38.403884  1647 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:38.315140  3726 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.253711  3706 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.408231  3704 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:38.419153  1569 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:01:38.419574  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.424204  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.422482  1472 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:38.425868  1576 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.875s	user 0.016s	sys 0.000s
I20260709 13:01:38.398878  3703 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:38.427179  1573 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:38.435757  1720 rpcz_store.cc:276] 0709 13:01:36.038268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.655021 (+1616753us) service_pool.cc:224] Handling call
0709 13:01:37.680071 (+ 25050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.789306 (+109235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.789342 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.894798 (+105456us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.098159 (+203361us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:38.098228 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.098335 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.391433 (+293098us) raft_consensus.cc:1685] finished
0709 13:01:38.391462 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.391693 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58916,"raft.queue_time_us":81864,"spinlock_wait_cycles":68673664,"thread_start_us":127328,"threads_started":1,"wal-append.queue_time_us":326182}
I20260709 13:01:38.399006  3528 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:38.451646  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:38.487668  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.419852  1567 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:38.509040  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:38.566844  1576 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:38.527299  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 215145us
I20260709 13:01:38.583484  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:38.417618  3669 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:38.539948  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.591192  1575 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:38.428756  3608 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.004s
I20260709 13:01:38.416559  3476 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:38.421663  3710 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.684649  3710 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:38.662693  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1037) took 1081 ms. Trace:
I20260709 13:01:38.687017  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.687458  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1042) took 1160 ms. Trace:
I20260709 13:01:38.664928  1642 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:01:38.667253  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 158391us
W20260709 13:01:38.692885  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 109577us
I20260709 13:01:38.693302  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.694996  1643 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:38.686668  1576 rpcz_store.cc:276] 0709 13:01:37.500700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.501940 (+  1240us) service_pool.cc:224] Handling call
0709 13:01:38.582402 (+1080462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:38.399605  3708 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.501194  3502 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.687896  1574 rpcz_store.cc:276] 0709 13:01:37.526946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.527131 (+   185us) service_pool.cc:224] Handling call
0709 13:01:37.548411 (+ 21280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.770354 (+221943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.770391 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.893969 (+123578us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.279371 (+385402us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:38.279409 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.279601 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.687178 (+407577us) raft_consensus.cc:1685] finished
0709 13:01:38.687205 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.687371 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52177,"raft.queue_time_us":286492,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":9149,"thread_start_us":411877,"threads_started":2,"wal-append.queue_time_us":567808}
W20260709 13:01:38.714484  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 134885us
I20260709 13:01:38.500753  3496 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:38.553824  1568 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
W20260709 13:01:38.742002  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51542.7us
I20260709 13:01:38.748607  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:38.694772  3706 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:38.763553  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47790.5us
I20260709 13:01:38.703428  3476 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.750245  3496 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:01:38.769238  3496 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:38.699393  3608 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.772702  3608 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:38.773862  3671 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.774394  3671 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:38.779072  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.663852  3533 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:01:38.781621  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1756) took 1977 ms. Trace:
I20260709 13:01:38.713805  3704 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:38.620563  3557 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.787632  1723 rpcz_store.cc:276] 0709 13:01:36.804671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.722919 (+918248us) service_pool.cc:224] Handling call
0709 13:01:37.723304 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:37.789349 (+ 66045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.789377 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:37.894202 (+104825us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.044216 (+150014us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:38.044260 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.044378 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.781350 (+736972us) raft_consensus.cc:1685] finished
0709 13:01:38.781376 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.781544 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65570,"raft.queue_time_us":88514,"spinlock_wait_cycles":66773376,"thread_start_us":89233,"threads_started":1,"wal-append.queue_time_us":242472}
I20260709 13:01:38.780059  1647 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:38.803217  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:38.792130  1539 leader_election.cc:400] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:01:38.645542  3528 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.731688  3411 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:38.746050  3502 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.832494  1647 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:38.832623  3735 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:38.804780  3703 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:38.837663  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1762) took 2031 ms. Trace:
I20260709 13:01:38.839759  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:38.847986  1724 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:38.814870  3664 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:38.765862  3495 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:01:38.817251  3533 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:38.868263  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:38.742822  3669 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:38.856594  1539 leader_election.cc:403] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:38.873492  1722 rpcz_store.cc:276] 0709 13:01:36.806759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.755891 (+949132us) service_pool.cc:224] Handling call
0709 13:01:37.778799 (+ 22908us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:38.062275 (+283476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.062310 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.063037 (+   727us) log.cc:844] Serialized 52 byte log entry
0709 13:01:38.160528 (+ 97491us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:38.160621 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.837120 (+676499us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54003ac354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:38.837182 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.837239 (+    57us) raft_consensus.cc:1685] finished
0709 13:01:38.837254 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.837574 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6389,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":159,"spinlock_wait_cycles":630372736,"thread_start_us":96905,"threads_started":1,"wal-append.queue_time_us":220020}
I20260709 13:01:38.827286  3704 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:38.899515  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:38.826606  3609 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:38.904212  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 514140us
I20260709 13:01:38.788710  3476 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:38.923738  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:38.903873  3572 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.260s	user 0.010s	sys 0.005s
I20260709 13:01:38.864830  3735 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:38.902348  3403 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.233s	user 0.016s	sys 0.000s
W20260709 13:01:38.906004  3616 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.965s	user 0.012s	sys 0.001s
W20260709 13:01:38.906549  3497 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.143s	user 0.008s	sys 0.007s
W20260709 13:01:38.931793  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 139909us
I20260709 13:01:38.942058  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:38.897229  3730 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.903905  3533 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:38.956816  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.928673  3568 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:38.957031  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:38.934845  3687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.003s	sys 0.003s
W20260709 13:01:39.000908  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.062s	sys 0.124s
W20260709 13:01:39.006129  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.062s	sys 0.124s
I20260709 13:01:38.940831  3735 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:38.965842  3680 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.443s	user 0.015s	sys 0.003s
W20260709 13:01:38.972054  3310 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.306s	user 0.005s	sys 0.011s
I20260709 13:01:38.940449  3572 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.952316  3403 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.958652  3616 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:39.019927  3687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.003s	sys 0.003s
W20260709 13:01:39.206281  3562 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.426s	user 0.010s	sys 0.006s
I20260709 13:01:38.961637  3497 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.004850  3681 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:39.209985  3744 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 13:01:39.037745  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.183111  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 176350us
I20260709 13:01:39.067917  3473 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:39.258404  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 299918us
I20260709 13:01:39.026993  3609 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.975962  3728 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:01:39.286435  1648 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:39.280939  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:39.263293  3310 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.616s	user 0.007s	sys 0.022s
W20260709 13:01:39.146561  3585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.006s	sys 0.011s
W20260709 13:01:39.287298  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.006s	sys 0.004s
I20260709 13:01:39.178961  3602 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:39.292625  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.007s	sys 0.004s
I20260709 13:01:39.068928  3692 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:39.298420  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 560652us
I20260709 13:01:39.226554  3557 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:39.320151  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.288533  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 346567us
I20260709 13:01:39.345281  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:39.360324  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 440893us
W20260709 13:01:39.295915  3585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.009s	sys 0.011s
I20260709 13:01:39.305205  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:39.359246  2374 raft_consensus.cc:2368] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4.
I20260709 13:01:39.382812  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:39.387737  1572 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:39.294740  3664 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.296725  3602 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:39.294770  3568 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:39.392220  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:39.295034  3473 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:38.962069  3730 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:39.397236  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 93117.3us
W20260709 13:01:39.400856  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.286943  3562 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.408556  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:39.272512  3680 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.425040  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:39.426185  1572 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:39.371785  3708 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.368279  3502 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:39.449501  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57422.4us
I20260709 13:01:39.469252  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:39.294402  3724 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.493s
W20260709 13:01:39.475013  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 129941us
I20260709 13:01:39.401856  3528 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.368460  3731 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:39.361651  3609 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.395749  3687 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.396044  3557 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.472577  3709 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.477579  3762 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.437215  3692 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:39.490348  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:39.477759  1647 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:39.530409  1572 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:39.531275  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:39.433981  3681 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.495360  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:39.506709  1642 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:39.396752  3726 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.518441  3762 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.519248  3730 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:39.520797  3753 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
W20260709 13:01:39.521694  3747 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 13:01:39.522452  3749 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:01:39.517055  3602 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.483752  3411 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:39.532003  1647 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:39.486596  3669 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:39.431768  3568 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.563246  3709 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:39.583743  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.577551  3692 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:39.515339  3731 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:39.690577  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 195339us
I20260709 13:01:39.684629  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1675) took 1388 ms. Trace:
I20260709 13:01:39.580235  3762 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:39.605624  3743 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.351s	user 0.000s	sys 0.002s
I20260709 13:01:39.524811  3687 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:39.715997  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.014s	sys 0.022s
W20260709 13:01:39.716703  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.015s	sys 0.022s
W20260709 13:01:39.676548  3757 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:01:39.694351  1575 rpcz_store.cc:276] 0709 13:01:38.295845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.653463 (+357618us) service_pool.cc:224] Handling call
0709 13:01:38.654025 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.691036 (+ 37011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.691087 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.741285 (+ 50198us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.011142 (+269857us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:39.011213 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.011355 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.684302 (+672947us) raft_consensus.cc:1685] finished
0709 13:01:39.684326 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.684528 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":732,"raft.queue_time_us":880,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":21179,"thread_start_us":196459,"threads_started":1,"wal-append.queue_time_us":700734}
I20260709 13:01:39.605464  3730 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:39.577131  3681 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:39.685297  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 201704us
W20260709 13:01:39.695631  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.197s (SENT)
I20260709 13:01:39.642023  3602 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.717499  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:39.484783  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 196843us
I20260709 13:01:39.622233  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:39.722225  1574 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.004s	sys 0.008s
I20260709 13:01:39.624152  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:39.695880  1614 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4.
I20260709 13:01:39.687925  3528 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.641690  3767 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:39.585862  3557 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:39.627305  3501 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.738552  1645 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:39.688084  3709 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:39.740156  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:39.743441  1647 raft_consensus.cc:2376] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5. Re-sending same reply.
I20260709 13:01:39.743606  1614 leader_election.cc:403] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:39.742189  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:39.743157  1574 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.004s	sys 0.008s
I20260709 13:01:39.718756  3750 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.746646  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:39.700534  3403 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:39.762233  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.725674  3562 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.759235  3767 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.627432  3411 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.740302  3669 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.772008  1572 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:39.770639  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.793380  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:39.794152  1572 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:39.772624  1575 raft_consensus.cc:2376] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3. Re-sending same reply.
I20260709 13:01:39.769730  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:39.793525  3568 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:39.741523  3730 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:39.776180  3759 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.169s	user 0.002s	sys 0.000s
W20260709 13:01:39.798811  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 103071us
I20260709 13:01:39.759701  1568 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:39.804859  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:39.806068  1575 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
W20260709 13:01:39.806260  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110710us
I20260709 13:01:39.807667  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:39.755625  3528 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.776424  3765 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:39.824442  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.777822  3709 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:39.785795  3403 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:39.892277  3710 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.207s	user 0.017s	sys 0.001s
I20260709 13:01:39.904829  3710 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.907073  3710 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.853044  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:39.912111  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:39.912938  1723 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
W20260709 13:01:39.913102  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 67551.6us
I20260709 13:01:39.853391  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:39.913959  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:39.914305  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1054) took 1055 ms. Trace:
I20260709 13:01:39.914683  1575 rpcz_store.cc:276] 0709 13:01:38.858786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.811383 (+952597us) service_pool.cc:224] Handling call
0709 13:01:39.914238 (+102855us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:39.915314  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:39.916574  3724 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.917405  3724 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:39.918102  3724 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.918467  3724 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:39.918860  3724 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.785580  3752 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:39.776891  3501 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.798002  3473 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:39.888895  3658 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 2.145s	user 0.018s	sys 0.000s
I20260709 13:01:39.853638  3739 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.885811  3557 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:39.891669  3685 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.800s	user 0.011s	sys 0.003s
I20260709 13:01:39.916249  1575 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:39.846786  3680 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.811100  3568 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.855516  3765 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.789031  3669 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:39.900511  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1040) took 1213 ms. Trace:
I20260709 13:01:39.773330  3750 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:39.921200  1722 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:39.921885  1720 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:01:40.045934  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.939493  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 200619us
I20260709 13:01:39.955053  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:40.069096  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:39.956378  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:01:40.070225  1642 raft_consensus.cc:2410] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:39.931041  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1049) took 1081 ms. Trace:
I20260709 13:01:40.070868  1575 rpcz_store.cc:276] 0709 13:01:38.849843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.915512 (+1065669us) service_pool.cc:224] Handling call
0709 13:01:39.915860 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.930626 (+ 14766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.930931 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.072530  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:40.075340  1575 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:40.076992  1572 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:39.885219  3748 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:01:40.083750  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:39.925928  3702 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 2.427s	user 0.009s	sys 0.018s
W20260709 13:01:39.945887  3766 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:01:39.923789  3473 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:39.920286  3572 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.057575  1649 rpcz_store.cc:276] 0709 13:01:38.686758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.688508 (+  1750us) service_pool.cc:224] Handling call
0709 13:01:38.688978 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.741109 (+ 52131us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:38.778076 (+ 36967us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.796750 (+1018674us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.802385 (+  5635us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.851758 (+ 49373us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.851980 (+   222us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:39.852031 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.852132 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.900076 (+ 47944us) raft_consensus.cc:1685] finished
0709 13:01:39.900123 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.900308 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141329,"prepare.queue_time_us":377767,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":106,"raft.queue_time_us":212061,"raft.run_cpu_time_us":471,"raft.run_wall_time_us":56854,"spinlock_wait_cycles":403840,"thread_start_us":895470,"threads_started":3,"wal-append.queue_time_us":589242}
I20260709 13:01:40.070098  1721 raft_consensus.cc:606] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:40.098981  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:39.951679  3671 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.177s	user 0.014s	sys 0.000s
I20260709 13:01:39.852865  3703 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.092166  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.094698  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:39.930755  2372 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:39.923295  3501 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:39.857465  3497 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:39.946923  3411 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:39.938926  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:39.946220  3528 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:39.945421  3764 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:01:40.124811  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.125608  1572 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:39.898415  2371 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 13:01:40.099860  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70401us
I20260709 13:01:39.939253  3771 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.103411  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:39.941715  3724 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.074486  3609 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:39.950140  3765 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:39.920749  3752 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.109562  3501 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:40.146739  3501 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.129550  2371 pending_rounds.cc:85] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 5
I20260709 13:01:40.147989  2371 pending_rounds.cc:107] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:01:40.148326  2371 raft_consensus.cc:2889] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.149376  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 344) took 2659 ms. Trace:
I20260709 13:01:40.149529  3501 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.149834  2371 rpcz_store.cc:276] 0709 13:01:37.489540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.501648 (+ 12108us) service_pool.cc:224] Handling call
0709 13:01:39.897371 (+2395723us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:39.897736 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.148970 (+251234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.149285 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5068160}
I20260709 13:01:40.116657  1575 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:39.800661  3692 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:40.122553  1567 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.436s	user 0.014s	sys 0.003s
W20260709 13:01:40.127856  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.131446  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.156507  1567 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:39.924652  3658 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:40.135290  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65810.5us
I20260709 13:01:40.136591  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:40.164861  1575 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:40.165424  1575 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 1
I20260709 13:01:40.165782  1575 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:40.166114  1575 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.167166  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1051) took 1309 ms. Trace:
I20260709 13:01:40.167619  1575 rpcz_store.cc:276] 0709 13:01:38.857886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.163762 (+1305876us) service_pool.cc:224] Handling call
0709 13:01:40.164298 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.166763 (+  2465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.167074 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.107800  3671 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.138938  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1045) took 2598 ms. Trace:
I20260709 13:01:40.104761  1649 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:40.105988  3473 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:40.145627  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 230422us
I20260709 13:01:40.147043  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.123646  3739 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.112622  3742 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.124429  3411 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:40.157558  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1669) took 1863 ms (client timeout 1907 ms). Trace:
W20260709 13:01:40.157548  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88540.2us
W20260709 13:01:40.196362  1567 rpcz_store.cc:269] 0709 13:01:38.294033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.307916 (+ 13883us) service_pool.cc:224] Handling call
0709 13:01:40.157465 (+1849549us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.137593  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1048) took 1288 ms. Trace:
I20260709 13:01:40.175040  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:40.176005  3752 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.177983  3592 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:40.178150  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46773.4us
I20260709 13:01:40.227458  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:40.176052  3609 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:40.179260  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1677) took 1853 ms. Trace:
W20260709 13:01:40.228595  3706 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.445s	user 0.010s	sys 0.008s
I20260709 13:01:40.229725  1569 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:01:40.230070  1569 pending_rounds.cc:85] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 1
I20260709 13:01:40.228755  1574 rpcz_store.cc:276] 0709 13:01:38.325675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.750570 (+424895us) service_pool.cc:224] Handling call
0709 13:01:38.785887 (+ 35317us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:39.774413 (+988526us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:39.775415 (+  1002us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.952377 (+176962us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:40.028062 (+ 75685us) log.cc:844] Serialized 52 byte log entry
0709 13:01:40.028361 (+   299us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:40.028426 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.028563 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.178969 (+150406us) raft_consensus.cc:1685] finished
0709 13:01:40.178996 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.179183 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11604,"prepare.queue_time_us":438,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":76479,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":54022,"thread_start_us":1146380,"threads_started":2,"wal-append.queue_time_us":354185}
I20260709 13:01:40.230298  1569 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:40.219201  3671 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.181154  3658 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.130307  1568 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:40.183292  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.187647  1573 rpcz_store.cc:276] 0709 13:01:37.540800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.547717 (+  6917us) service_pool.cc:224] Handling call
0709 13:01:37.600915 (+ 53198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.564783 (+963868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.138618 (+1573835us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54001257d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:40.138859 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899728640}
I20260709 13:01:40.142314  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:40.192890  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:40.144263  3687 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.197067  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:40.197772  1651 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:40.199927  3781 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:40.202495  3779 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.202656  3724 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.208099  1572 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:40.210115  3786 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.212078   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:01:40.214371  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.213984  3742 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.195604  3765 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.193908  3785 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:40.226344  3735 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.945s	user 0.012s	sys 0.001s
I20260709 13:01:40.230554  1569 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.231761  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.181397  3669 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:40.163277  3731 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.197541  1571 rpcz_store.cc:276] 0709 13:01:38.849505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.860784 (+ 11279us) service_pool.cc:224] Handling call
0709 13:01:38.899279 (+ 38495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.925485 (+ 26206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.925535 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.999498 (+ 73963us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.278672 (+279174us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:39.278727 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.278847 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.137228 (+858381us) raft_consensus.cc:1685] finished
0709 13:01:40.137262 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.137500 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322131,"raft.queue_time_us":34500,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":13632,"thread_start_us":104815,"threads_started":1,"wal-append.queue_time_us":606368}
I20260709 13:01:40.171715  3681 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.121779  3730 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.239884  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:40.247444  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55893us
I20260709 13:01:40.230553  3750 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.201480  3501 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:40.228636  3609 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.249224  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1053) took 1343 ms. Trace:
I20260709 13:01:40.273013  1575 rpcz_store.cc:276] 0709 13:01:38.858495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.195082 (+1336587us) service_pool.cc:224] Handling call
0709 13:01:40.196940 (+  1858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.197248 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.197278 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.197324 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:40.197364 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.199458 (+  2094us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.201959 (+  2501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:40.251652  1649 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:40.257359   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:40.237035  3592 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:40.253078  3739 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.245944  3779 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:40.276762  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45122.6us
I20260709 13:01:40.357838  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:40.358584  3779 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.358742  3788 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.253099  3765 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.360142  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:40.278779  3482 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:40.360940  3788 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.279727  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1050) took 1422 ms. Trace:
I20260709 13:01:40.335175  3669 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:40.356246  3592 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.362309  3669 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.362493  3473 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:40.362809  3592 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:40.362972  3669 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.335565  3681 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:01:40.368388  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1056) took 1503 ms. Trace:
I20260709 13:01:40.277844  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:40.273043  3785 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.360953  3765 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.250211  3687 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:40.262655  3579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.738s	user 0.020s	sys 0.081s
I20260709 13:01:40.361734  1576 rpcz_store.cc:276] 0709 13:01:38.857539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.923101 (+1065562us) service_pool.cc:224] Handling call
0709 13:01:39.923472 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.923888 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:39.923918 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.035421 (+111503us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.241139 (+205718us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:40.241180 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.241256 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.279398 (+ 38142us) raft_consensus.cc:1685] finished
0709 13:01:40.279429 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.279630 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12673,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"thread_start_us":139975,"threads_started":1,"wal-append.queue_time_us":190719}
W20260709 13:01:40.339362  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 173917us
W20260709 13:01:40.278520  3762 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
W20260709 13:01:40.398711  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 120958us
I20260709 13:01:40.275810  3609 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:40.312423  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.405934  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.249492  3671 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:40.431286  3671 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:40.407408  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:40.312253  3752 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.246248  3781 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.412914  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:01:40.411526  3579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.887s	user 0.024s	sys 0.082s
I20260709 13:01:40.406077  3473 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:40.276949  3776 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.434356  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:40.400146  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:40.353988  3403 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4.
W20260709 13:01:40.448279  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48182.5us
I20260709 13:01:40.272009  3735 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.449014  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:40.277189  3501 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.450078  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.250484  3786 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:40.461980  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.498944  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62196.9us
I20260709 13:01:40.508073  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.433069  3687 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:40.510671  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60680.4us
W20260709 13:01:40.513329  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79205.6us
I20260709 13:01:40.514832  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:40.436774  3765 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:40.435835  3559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.836s	user 0.020s	sys 0.051s
W20260709 13:01:40.521785  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.450913  3719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.207s	user 0.017s	sys 0.012s
I20260709 13:01:40.572959  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.573899  1576 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:40.437425  3710 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:40.578070  1642 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:40.452612  3692 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:40.449218  3739 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:40.506692  3482 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:40.415752  3762 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.502874  3473 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:40.434916  3752 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.502414  3693 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.592950  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.599769  1576 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:40.497014  3788 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:01:40.600734  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:40.601269  1576 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:40.576496  3559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.977s	user 0.022s	sys 0.061s
W20260709 13:01:40.579957  3719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.407s	user 0.019s	sys 0.026s
I20260709 13:01:40.572394  3579 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.500984  3776 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.583455  3728 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:40.584467  2373 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:40.572445  3528 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:01:40.577260  3765 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.590725  3482 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.658452  3579 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.668140  3482 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.669304  3786 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.669637  3579 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.670835  3752 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.576658  3750 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:01:40.672365  3750 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:40.672710  3750 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:40.673180  3785 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:40.673616  3750 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.674724  3781 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:40.674796  3750 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.608894  3739 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:01:40.684654  3765 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.690500  3762 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:40.692677  3473 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:40.694129  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.657706  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1029) took 2279 ms (client timeout 1732 ms). Trace:
W20260709 13:01:40.658180  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 143440us
I20260709 13:01:40.582489  3403 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:01:40.453617  3671 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.588120  3724 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:01:40.665762  3782 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:40.666693  3792 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:01:40.667330  3501 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:40.667565  3789 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:01:40.704978  1648 rpcz_store.cc:269] 0709 13:01:38.326164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.326294 (+   130us) service_pool.cc:224] Handling call
0709 13:01:40.605118 (+2278824us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54003fd754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:40.605241 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044256128}
I20260709 13:01:40.712064  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1060) took 1793 ms. Trace:
I20260709 13:01:40.829531  1573 rpcz_store.cc:276] 0709 13:01:38.919074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.248482 (+1329408us) service_pool.cc:224] Handling call
0709 13:01:40.248801 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.248954 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:40.451149 (+202195us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.711563 (+260414us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.711638 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:40.711704 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.711799 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.711969 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":106551,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":124,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":71791,"thread_start_us":355420,"threads_started":2}
I20260709 13:01:40.718752  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:40.723829  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.787451  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 222097us
I20260709 13:01:40.793169  3782 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:40.802635  1576 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:40.833604  1688 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Timed out: RequestConsensusVote RPC to 127.0.107.196:45309 timed out after 1.983s (SENT)
I20260709 13:01:40.838994  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.605849  3779 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:01:40.832070  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:40.896572  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 380614us
I20260709 13:01:40.936303  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:40.579260  3692 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:40.943818  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:40.944836  3725 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 2.528s	user 0.006s	sys 0.028s
I20260709 13:01:40.945586  3762 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:40.945403  3528 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:40.946116  3762 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:40.946148  3731 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.947232  3762 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:40.948151  3746 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.682s	user 0.002s	sys 0.002s
I20260709 13:01:40.947888  3731 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:40.949436  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1035) took 2613 ms. Trace:
I20260709 13:01:40.949723  3731 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:40.949839  1646 rpcz_store.cc:276] 0709 13:01:38.336189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.646413 (+310224us) service_pool.cc:224] Handling call
0709 13:01:38.684579 (+ 38166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.685145 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.685190 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.686003 (+   813us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.926771 (+240768us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:38.926842 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.926965 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.947391 (+2020426us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:40.949035 (+  1644us) raft_consensus.cc:1685] finished
0709 13:01:40.949068 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.949359 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":753,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":158,"spinlock_wait_cycles":14798080,"thread_start_us":240251,"threads_started":1,"wal-append.queue_time_us":541021}
I20260709 13:01:40.953179  3731 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:40.954521  3731 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.957415  3718 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:40.957836  3718 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:40.958668  3718 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:40.959672  3718 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:40.960515  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.376854  1574 rpcz_store.cc:276] 0709 13:01:38.865177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.230599 (+1365422us) service_pool.cc:224] Handling call
0709 13:01:40.230938 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.232975 (+  2037us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:40.233536 (+   561us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.367768 (+134232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.367840 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:40.367901 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.367997 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.368143 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96315,"prepare.queue_time_us":398,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":150,"thread_start_us":133755,"threads_started":1}
W20260709 13:01:40.964165  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 258863us
I20260709 13:01:40.858623  3782 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:40.975770  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 142217us
I20260709 13:01:40.932274  3690 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:41.034006  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:40.980938  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.988659  3709 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.905s	user 0.015s	sys 0.000s
I20260709 13:01:41.047933  1574 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:40.979588  3692 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.906319  3687 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.060318  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1055) took 2201 ms. Trace:
I20260709 13:01:41.063431  1567 rpcz_store.cc:276] 0709 13:01:38.859077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.197487 (+1338410us) service_pool.cc:224] Handling call
0709 13:01:40.198102 (+   615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.226787 (+ 28685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.226836 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.227343 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.271395 (+ 44052us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:40.271447 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.271523 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.060030 (+788507us) raft_consensus.cc:1685] finished
0709 13:01:41.060058 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.060241 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1596,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":43631,"threads_started":1,"wal-append.queue_time_us":187290}
I20260709 13:01:41.038771  3786 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:41.055231  3709 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:41.090883  3705 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 3.397s	user 0.006s	sys 0.021s
W20260709 13:01:41.099424  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 152666us
I20260709 13:01:41.082790  3731 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:41.086071  3690 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:41.084887  3735 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:41.101990  3592 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.739s	user 0.004s	sys 0.008s
W20260709 13:01:41.105646  3669 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.000s
I20260709 13:01:41.111536  1575 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:41.081781  3786 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.110682  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:41.106452  3701 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 3.452s	user 0.007s	sys 0.009s
I20260709 13:01:41.129230  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:41.132011  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.135699  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.139423  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:41.139577  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.106408  3690 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:41.130594  3730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.034s	sys 0.066s
W20260709 13:01:41.144380  1688 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Timed out: RequestConsensusVote RPC to 127.0.107.196:45309 timed out after 1.662s (SENT)
I20260709 13:01:41.144977  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:41.145582  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:41.153395  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:41.154660  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:41.155031  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 190130us
I20260709 13:01:41.156620  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:41.108394  3731 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:41.165658  1571 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:41.165403  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:41.167011  2377 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:41.144448  3690 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.178377  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:41.154377  3730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.034s	sys 0.067s
I20260709 13:01:41.135208  3669 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:41.160014  3674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.196s	user 0.025s	sys 0.026s
I20260709 13:01:41.133316  3592 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.133056  3800 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:01:41.231089  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:41.234252  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:41.178206  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:01:41.240170  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 75193.6us
I20260709 13:01:41.241726  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:41.244050  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87542.1us
W20260709 13:01:41.233515  3674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.273s	user 0.026s	sys 0.026s
I20260709 13:01:41.231026  1718 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:41.237406  3672 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:41.252679  3798 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:01:41.245407  3807 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:41.257038  3807 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:41.258255  3807 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:41.272730  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:41.239434  3528 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:41.274757  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.280110  3592 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:41.281013  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:41.282912  3609 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.852s	user 0.009s	sys 0.000s
I20260709 13:01:41.329025  3609 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.273563  3672 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:41.287729  3765 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.002s
I20260709 13:01:41.291436  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:41.391887  3664 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.394543  3609 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:41.395377  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.395939  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:41.396804   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:41.397686  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:41.398757  1718 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:41.399761  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:41.400974  3786 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:41.403003  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:41.403631  1571 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.405753  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.406466  1722 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:41.406836  3731 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:41.407212  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.407842  1722 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:41.407617  3731 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:41.409179  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.410317  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.410841  1723 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:41.411109  1688 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.993s (SENT)
I20260709 13:01:41.411739  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.414054  1723 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
W20260709 13:01:41.404078  1542 connection.cc:256] RPC call timeout handler was delayed by -1.3785584350000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:01:41.419749  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:41.414994  1571 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:41.414911  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.425920  1722 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:01:41.427373  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:41.428184  1571 raft_consensus.cc:2368] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6.
I20260709 13:01:41.431607  2380 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:41.439826  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:41.379398  3765 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.448400  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:41.451074  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:41.452693  1722 raft_consensus.cc:2410] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:41.456034  1722 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:41.436527  3482 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.005s
I20260709 13:01:41.464666  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:41.465974  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 167) took 3562 ms (client timeout 1683 ms). Trace:
W20260709 13:01:41.467449  1725 rpcz_store.cc:269] 0709 13:01:37.903465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.062690 (+159225us) service_pool.cc:224] Handling call
0709 13:01:41.465748 (+3403058us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:41.465895 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488769792}
I20260709 13:01:41.471496  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:41.475471  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:41.460759  3762 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:41.478076  1720 raft_consensus.cc:2368] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5.
I20260709 13:01:41.462428  3709 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:41.497633  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:41.498258  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 250980us
I20260709 13:01:41.498600  1571 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:41.469396  3528 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:41.471911  3814 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:41.487223  3762 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.512938  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:41.521713  3784 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 1.441s	user 0.007s	sys 0.008s
I20260709 13:01:41.523550  3765 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:41.525755  3782 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.492587  3807 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:41.529768  3669 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:41.470743  3482 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.501943  3528 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:41.573329  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.566828  3762 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:41.588366  3807 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:41.606935  3807 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.532594  3690 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.614066  3776 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:41.614486  3690 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:41.615801  1574 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:41.591929  3687 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:41.620117  1721 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:41.598264  3528 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:41.505575  3806 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.104s
W20260709 13:01:41.618660  3728 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.956s	user 0.010s	sys 0.017s
W20260709 13:01:41.630124  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 130989us
I20260709 13:01:41.632588  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:41.503502  3672 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:41.637091  3710 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.051s	user 0.019s	sys 0.002s
I20260709 13:01:41.636670  3728 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:41.660650  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.641054  3750 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.004s
I20260709 13:01:41.654687  3710 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:41.639840  3687 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:41.660895  3807 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.668325  3476 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:41.668699  1539 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4.
I20260709 13:01:41.670022  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:41.652513  3692 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.001s
I20260709 13:01:41.676957  3782 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:41.702709  2374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 256) took 3945 ms (client timeout 1983 ms). Trace:
W20260709 13:01:41.703399  2374 rpcz_store.cc:269] 0709 13:01:37.757089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.762555 (+  5466us) service_pool.cc:224] Handling call
0709 13:01:38.839557 (+1077002us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:41.702334 (+2862777us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54006d1ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:41.702554 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887800320}
W20260709 13:01:41.708608  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.713677  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:41.714475  1576 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:41.670784  3672 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.721678  3692 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:41.722179  3533 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:41.726468  3533 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:41.728863  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:41.729887  1572 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:41.685860  3687 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:41.737911  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 168) took 3455 ms. Trace:
I20260709 13:01:41.741047  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1763) took 4933 ms. Trace:
I20260709 13:01:41.741546  1724 rpcz_store.cc:276] 0709 13:01:36.807160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.900744 (+1093584us) service_pool.cc:224] Handling call
0709 13:01:37.901031 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.875743 (+974712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.740684 (+2864941us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b54003dd6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:41.740968 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176980352}
I20260709 13:01:41.742945  1573 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:41.651880  3664 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:41.745860  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:41.752888  1572 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:41.738479  1717 rpcz_store.cc:276] 0709 13:01:38.282701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.844395 (+561694us) service_pool.cc:224] Handling call
0709 13:01:38.845034 (+   639us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:38.934972 (+ 89938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.934997 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.957461 (+ 22464us) log.cc:844] Serialized 52 byte log entry
0709 13:01:39.408463 (+451002us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:39.486209 (+ 77746us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.486657 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.486847 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.737365 (+2250518us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b54001a6dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:41.737445 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.737628 (+   183us) raft_consensus.cc:1685] finished
0709 13:01:41.737649 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.737845 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12265,"prepare.queue_time_us":109090,"prepare.run_cpu_time_us":171,"prepare.run_wall_time_us":8880,"raft.queue_time_us":22774,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":36511,"spinlock_wait_cycles":210341376,"thread_start_us":259310,"threads_started":1,"wal-append.queue_time_us":663842,"wal-append.run_cpu_time_us":18829,"wal-append.run_wall_time_us":1514488}
I20260709 13:01:41.757731  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:41.763010  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.744779  3692 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:41.781932  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 96779.7us
I20260709 13:01:41.762493  3728 raft_consensus.cc:2764] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
W20260709 13:01:41.792245  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.798072  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.661s (SENT)
I20260709 13:01:41.798554  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:41.804662  3761 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:41.870569  3807 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:41.870760  3703 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.877009  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:41.877614  3703 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.814944  3762 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.814559  3616 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:41.895859  3775 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 2.103s	user 0.003s	sys 0.008s
I20260709 13:01:41.869232  3403 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.943657  3403 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:41.947335  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1042) took 3157 ms. Trace:
I20260709 13:01:41.947901  1643 rpcz_store.cc:276] 0709 13:01:38.789656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.789900 (+   244us) service_pool.cc:224] Handling call
0709 13:01:38.790307 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.790764 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.790795 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.793825 (+  3030us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.717224 (+923399us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:39.717268 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.717347 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.895058 (+2177711us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:41.946891 (+ 51833us) raft_consensus.cc:1685] finished
0709 13:01:41.946977 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.947256 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2646,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":187,"spinlock_wait_cycles":12957056,"thread_start_us":922841,"threads_started":1,"wal-append.queue_time_us":994474}
I20260709 13:01:41.908274  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:41.951555  3782 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:41.955022  3703 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:41.960675  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.683s (SENT)
I20260709 13:01:41.961628  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:41.963068  3761 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.976387  1576 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:42.060268  1576 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:01:42.093958  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 310396us
I20260709 13:01:42.095371  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:42.138973  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43698.4us
W20260709 13:01:42.140203  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 163935us
W20260709 13:01:42.134239  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.142072  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.929s (SENT)
W20260709 13:01:42.146178  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.973s (SENT)
I20260709 13:01:42.147145  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:42.140201  3728 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:42.149336  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.128s (SENT)
I20260709 13:01:42.150341  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:42.135406  3528 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:42.149082  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.147639  2376 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:42.156298  3688 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.156819  3688 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.150477  3716 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:42.150058  3728 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:42.162110  3728 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.162135  3687 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 13:01:42.167212  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.153405  3672 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:42.161468  3664 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:42.172991  3782 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.184602  3782 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.226320  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1046) took 4147 ms. Trace:
W20260709 13:01:42.224447  2372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 254) took 4489 ms (client timeout 1662 ms). Trace:
I20260709 13:01:42.234539  3800 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:42.235212  3800 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:42.236528  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.239373  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.240748  3559 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:42.241556  3559 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.243275  3482 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:42.244019  3559 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:42.244135  3807 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:42.244933  3559 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:42.245249  3807 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.245600  3559 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.247323  3782 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.248018  3559 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:42.248190  3528 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.226979  1570 rpcz_store.cc:276] 0709 13:01:38.079067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.119430 (+ 40363us) service_pool.cc:224] Handling call
0709 13:01:38.119923 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.453372 (+333449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.453416 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.454197 (+   781us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.885771 (+431574us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:38.885838 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.223631 (+3337793us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54002b0054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:42.223729 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.223810 (+    81us) raft_consensus.cc:1685] finished
0709 13:01:42.223830 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.226212 (+  2382us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3261,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":35195,"spinlock_wait_cycles":695090944,"thread_start_us":230882,"threads_started":1,"wal-append.queue_time_us":480123,"wal-append.run_cpu_time_us":16744,"wal-append.run_wall_time_us":1813151}
I20260709 13:01:42.251771  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.222515  3728 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.257304  3482 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:42.248309  3807 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.255297  3608 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:42.261101  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.275774  1570 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:42.288306  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.272718  3482 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:42.294724  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:42.232875  2372 rpcz_store.cc:269] 0709 13:01:37.731418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.741727 (+ 10309us) service_pool.cc:224] Handling call
0709 13:01:42.207931 (+4466204us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54001568d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.219687 (+ 11756us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4584118528}
I20260709 13:01:42.284607  3761 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:42.236294  3800 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.288313  3608 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.299958  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.293953  3692 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:42.305280  3733 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 3.808s	user 0.021s	sys 0.025s
W20260709 13:01:42.309175  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.304360  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.297181  3482 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.291421  3473 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:42.298682  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:42.319638  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.321218  1570 raft_consensus.cc:2368] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5.
I20260709 13:01:42.303243  3608 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.316960  3692 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.306504  3728 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.333041  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.321635  3800 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:42.342125  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.346066  3767 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.320729  3473 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.348133  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.322536  3807 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:42.349118  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1045) took 2942 ms (client timeout 3098 ms). Trace:
W20260709 13:01:42.351121  1645 rpcz_store.cc:269] 0709 13:01:39.406868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.407084 (+   216us) service_pool.cc:224] Handling call
0709 13:01:42.347442 (+2940358us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540038b3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.349046 (+  1604us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554484992}
I20260709 13:01:42.325560  3739 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:42.355558  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 180427us
I20260709 13:01:42.337200  3692 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:42.354081  3832 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.133s	user 0.000s	sys 0.002s
I20260709 13:01:42.361737  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.349889  3473 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.354769  3739 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.371177  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.374255  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.377612  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.379257  1643 raft_consensus.cc:2410] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:42.382283  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.384707  1650 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:42.387877  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.390215  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:42.391068  1647 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:42.391423  2378 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:42.392597  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.393299  1570 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:42.395866  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.396907  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:42.398911  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 112375us
W20260709 13:01:42.400431  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.401404  1616 connection.cc:256] RPC call timeout handler was delayed by -2.3309891960000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 13:01:42.399518  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 109348us
W20260709 13:01:42.402176  1616 connection.cc:256] RPC call timeout handler was delayed by -2.2787891959999911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 13:01:42.402609  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.385082  3742 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.356711  3807 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
W20260709 13:01:42.404196  1616 connection.cc:256] RPC call timeout handler was delayed by -2.1734891959999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
I20260709 13:01:42.403244  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.407186  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.406769  1570 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:42.408154  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.408562  1650 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:42.410781  1648 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:01:42.411255  1616 connection.cc:256] RPC call timeout handler was delayed by -1.54874094500002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:01:42.412477  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:42.412757  1616 connection.cc:256] RPC call timeout handler was delayed by -1.3839666320000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 13:01:42.413172  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.402231  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:42.403549  3835 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:42.413251  1616 connection.cc:256] RPC call timeout handler was delayed by -1.0042115620000232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1755s for a response
I20260709 13:01:42.417613  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:42.409704  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:42.416987  3778 raft_consensus.cc:993] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:01:42.413407  3502 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:42.416267  1645 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:42.415982  1570 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.418202  2378 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:42.418370  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:42.421978  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:42.439031  1644 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:42.420836  2372 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:42.424719  3836 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:42.417007  3507 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 13:01:42.430637  3791 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.838s	user 0.005s	sys 0.015s
I20260709 13:01:42.431572  1648 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:42.432374  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.444298  1650 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:42.432951  1651 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:42.432889  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1056) took 2729 ms (client timeout 2265 ms). Trace:
I20260709 13:01:42.433928  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:42.444531  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.445075  1649 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.438843  3736 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:01:42.446916  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1661) took 1308 ms. Trace:
I20260709 13:01:42.447300  1721 rpcz_store.cc:276] 0709 13:01:41.138596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.139204 (+   608us) service_pool.cc:224] Handling call
0709 13:01:42.446700 (+1307496us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b5400171bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.446842 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103668608}
I20260709 13:01:42.448387  3528 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:42.448413  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.448632  3782 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:42.436820  3669 raft_consensus.cc:1081] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:01:42.450090  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.435395  2375 raft_consensus.cc:1813] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Handling vote request from an unknown peer 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:42.450825  1576 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:42.451277  2375 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:42.452770  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1676) took 4142 ms. Trace:
I20260709 13:01:42.453265  1569 rpcz_store.cc:276] 0709 13:01:38.310575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.688233 (+377658us) service_pool.cc:224] Handling call
0709 13:01:38.735417 (+ 47184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.255169 (+1519752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.452537 (+2197368us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54001aa9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:42.452711 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3436036352}
I20260709 13:01:42.420964  3835 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:42.444825  1642 rpcz_store.cc:269] 0709 13:01:39.703613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.141950 (+438337us) service_pool.cc:224] Handling call
0709 13:01:42.432628 (+2290678us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b540005c3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.432810 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788734848}
I20260709 13:01:42.453299  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.458905  1649 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:42.459394  1576 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:42.443167  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.443684  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:42.454744  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:42.464821  1648 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:42.453984  3739 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.472793  1576 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:42.474236  1576 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:42.477325  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.479868  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:42.481360  2372 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:42.485049  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 257) took 4716 ms. Trace:
I20260709 13:01:42.485544  2373 rpcz_store.cc:276] 0709 13:01:37.769229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.770758 (+  1529us) service_pool.cc:224] Handling call
0709 13:01:37.906824 (+136066us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.666310 (+2759486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.484740 (+1818430us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b54005d25d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:42.484958 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5797675264}
I20260709 13:01:42.453081  3836 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.487247  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:42.488652  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45566.2us
I20260709 13:01:42.488770  3836 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:42.489744  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.490795  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.491146  3807 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.491912  3507 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:42.492501  3507 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:42.475369  3782 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:42.493603  3507 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.494530  3782 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:42.495464  3782 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.496770  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:42.497396  3572 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:42.497893  3572 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:42.498556  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.499418  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1083) took 1360 ms. Trace:
I20260709 13:01:42.499863  1574 rpcz_store.cc:276] 0709 13:01:41.138557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.142788 (+  4231us) service_pool.cc:224] Handling call
0709 13:01:42.499265 (+1356477us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400420ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.499370 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058706176}
I20260709 13:01:42.500403  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:42.501112  1642 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:42.501396  1572 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:42.502596  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.503408  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:42.506706  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.508916  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:42.498732  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1076) took 1990 ms (client timeout 2317 ms). Trace:
I20260709 13:01:42.509864  2378 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:01:42.510515  1573 rpcz_store.cc:269] 0709 13:01:40.508715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.831712 (+322997us) service_pool.cc:224] Handling call
0709 13:01:42.498555 (+1666843us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54000271d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.498679 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075229440}
I20260709 13:01:42.499680  3559 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:42.513653  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.512701  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.515415  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:42.516112  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.516767  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.486927  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:42.518951  1650 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:42.542873  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:42.515638  3608 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.488094  1568 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:42.522703  1648 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:42.523823  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.524564  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.524791  1644 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:42.525336  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.527125  3836 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:42.529309  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.521920  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:42.530404  3841 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:42.531637  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.533435  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:42.566530  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 67181.2us
I20260709 13:01:42.535913  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.521988  3502 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:42.539580  1642 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:42.567651  1724 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:42.541163  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.544301  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:42.551014  1647 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:42.499017  3572 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.549427  3482 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.516525  3669 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:42.570955  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.571478  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.582617  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.571719  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.582930  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.586520  1721 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:42.587606  1722 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.590317  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.590878  1716 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:42.591676  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.518442  3496 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:01:42.589995  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:42.595651  1724 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:42.598138  3608 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:42.560503  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90431.8us
I20260709 13:01:42.605666  1722 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:42.610054  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.612435  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:42.619592  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.620806  1724 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:42.541296  3693 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:01:42.623976  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.653743  1717 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.647352  1721 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:42.622427  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:42.632459  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:42.612798  3835 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:42.657372  1616 leader_election.cc:400] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6.
I20260709 13:01:42.664193  1616 leader_election.cc:403] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:01:42.610157  3482 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:42.657878  1724 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:42.666281  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:42.592509  3836 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.635088  1716 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:42.621410  3670 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:42.642247  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:42.679904  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.661557  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 137092us
I20260709 13:01:42.683174  1722 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:01:42.655586  3762 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:42.699860  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.697041  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.703842  1722 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:42.682188  3670 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.666857  3846 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:42.715622  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1072) took 2203 ms (client timeout 2217 ms). Trace:
I20260709 13:01:42.715910  1724 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:42.716131  1575 rpcz_store.cc:269] 0709 13:01:40.507708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.507819 (+   111us) service_pool.cc:224] Handling call
0709 13:01:42.710375 (+2202556us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54003638d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.710589 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385686784}
I20260709 13:01:42.716826  1716 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:42.722630  1724 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
W20260709 13:01:42.724960  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.727332  3692 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.728542  1725 raft_consensus.cc:2368] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:42.729912  3836 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:42.730297  3692 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:42.731495  3846 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:42.733170  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.717569  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.735548  1575 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:01:42.746152  1717 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:42.765543  1724 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:42.752839  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 184174us
I20260709 13:01:42.706640  3782 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:42.710023  3528 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:42.717278  3670 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:42.778666  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:42.779940  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:42.780964  3807 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:42.783288  3800 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:42.783927  3800 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:42.785041  3800 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.789314  1570 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:42.766286  1718 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:42.797940  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:42.798923  1570 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:42.803617  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 137447us
I20260709 13:01:42.632306  3502 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:42.766049  3846 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:42.788950  3762 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:01:42.856714  3841 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:42.866881  2371 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:42.893234  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1682) took 1471 ms. Trace:
I20260709 13:01:42.906396  1572 rpcz_store.cc:276] 0709 13:01:41.422308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.876755 (+454447us) service_pool.cc:224] Handling call
0709 13:01:42.501142 (+624387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.501955 (+   813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.892952 (+390997us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54000271d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:42.893161 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435220352}
I20260709 13:01:42.921738  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:42.806167  3528 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:01:42.986022  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:42.987462  2378 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:42.964547  1570 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:43.011225  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.805110  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:43.038441  1575 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:43.032629  3616 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:43.224685  3616 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.226303  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:42.891795  3836 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:43.050348  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:43.082445  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.083163  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 303284us
I20260709 13:01:43.227352  3692 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:43.052829  2373 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:43.231071  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.087827  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.234122  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:43.235033  3837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.011s	sys 0.037s
W20260709 13:01:43.235581  3837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.011s	sys 0.037s
I20260709 13:01:43.036374  3843 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:43.236261  3507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.018s	sys 0.030s
I20260709 13:01:42.926156  3841 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.239341  3841 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:43.046612  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 345) took 1936 ms. Trace:
I20260709 13:01:43.240016  2371 rpcz_store.cc:276] 0709 13:01:41.110120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.112839 (+  2719us) service_pool.cc:224] Handling call
0709 13:01:42.791911 (+1679072us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49f8c 00007fc830c49ef6 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:42.795275 (+  3364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.926991 (+131716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.041692 (+114701us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3783552}
W20260709 13:01:43.066798  3473 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.005s
I20260709 13:01:43.241389  3473 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.243129  1724 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 13:01:43.109176  3852 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:01:43.108976  3869 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:43.054167  3848 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:43.244874  3869 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:43.245179  3848 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:43.245951  3869 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.246145  3848 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.247447  3869 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:43.054109  1572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:01:43.248826  1572 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:43.249297  1572 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:43.249405  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.250174  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.140054  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.147836  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.253250  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.248828  1570 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:01:43.159503  3799 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 2.115s	user 0.004s	sys 0.044s
W20260709 13:01:43.179040  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 371624us
W20260709 13:01:43.196678  3572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.014s	sys 0.026s
W20260709 13:01:43.283473  3572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.014s	sys 0.026s
I20260709 13:01:43.229557  3692 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:43.236693  3507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.018s	sys 0.030s
W20260709 13:01:43.201581  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 396698us
I20260709 13:01:43.144196  3693 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:43.284555  3692 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.284891  3693 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.285928  3731 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:43.251629  3836 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.287163  3693 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:43.288586  3836 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:43.278519  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.313s (SENT) [suppressed 19 similar messages]
I20260709 13:01:43.236029  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.294647  1616 leader_election.cc:400] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5.
I20260709 13:01:43.295624  1616 leader_election.cc:403] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:01:43.295795  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.286401  3731 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:43.289742  3854 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.240s	user 0.001s	sys 0.001s
I20260709 13:01:43.244560  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:43.309365  1575 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:43.292582  3807 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:43.244093  3473 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:43.252908  3861 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:01:43.297504  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74796.9us
I20260709 13:01:43.300642  2378 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:43.284623  3403 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:43.252959  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.227286  3616 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:43.322376  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:43.287209  3859 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:01:43.277421  3843 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:43.366930  3843 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.362983  1572 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.297019  3762 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:43.361728  1570 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:43.361888  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:43.297629  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:43.362608  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.276822  3855 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:43.372913  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 45067.3us
I20260709 13:01:43.232204  2373 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
W20260709 13:01:43.352490  1568 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.039s	sys 0.061s
I20260709 13:01:43.378940  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:43.380987  1572 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:43.385766  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:43.399147  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 101651us
I20260709 13:01:43.391760  2378 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:43.400398  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:43.376111  3799 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:01:43.386821  1568 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.040s	sys 0.061s
I20260709 13:01:43.436805  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:43.444826  1570 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:43.440670  3871 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:43.441641  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.441882  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.442619  3855 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:43.443249  3670 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.649s	user 0.013s	sys 0.000s
I20260709 13:01:43.386713  3688 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:43.445824  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:43.461136  1642 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:43.446275  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.461688  1649 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:01:43.450106  1688 leader_election.cc:336] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.732s (SENT)
I20260709 13:01:43.453694  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:43.455757  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:43.443111  3507 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:01:43.459623  1648 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:43.444208  3865 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:01:43.404089  3473 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:43.377589  3528 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:01:43.404703  3502 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:43.464018  1575 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:43.467031  1723 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:43.482439  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:43.460305  3877 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:01:43.485795  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:43.492259  1642 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:43.460178  3800 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:43.476926  3688 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.399207  3731 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:43.498387  2380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 346) took 2363 ms (client timeout 1965 ms). Trace:
W20260709 13:01:43.507779  2380 rpcz_store.cc:269] 0709 13:01:41.135241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.135459 (+   218us) service_pool.cc:224] Handling call
0709 13:01:43.498152 (+2362693us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54005e0e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:43.498307 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591893888}
I20260709 13:01:43.511530  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:43.459827  2375 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:43.516952  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:43.493367  3841 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.525002  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.527772  2377 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:43.462335  3670 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.494493  3473 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:43.537886  3875 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:01:43.539225  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1714) took 1176 ms. Trace:
I20260709 13:01:43.539040  3473 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.539731  1645 rpcz_store.cc:276] 0709 13:01:42.362935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.363165 (+   230us) service_pool.cc:224] Handling call
0709 13:01:43.538950 (+1175785us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400306954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:43.539139 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93899520}
I20260709 13:01:43.541456  1717 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 1. Current term is 3. Ops: []
W20260709 13:01:43.529300  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.528934  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.528934  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:43.535385  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 169) took 5251 ms. Trace:
I20260709 13:01:43.537681  3670 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:43.462164  3855 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:43.528810  3841 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:43.549548  2380 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:01:43.561905  2380 pending_rounds.cc:85] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 5
I20260709 13:01:43.562347  2380 pending_rounds.cc:107] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:01:43.562681  2380 raft_consensus.cc:2889] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.543795  1719 rpcz_store.cc:276] 0709 13:01:38.283529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.365165 (+2081636us) service_pool.cc:224] Handling call
0709 13:01:40.365507 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.398904 (+ 33397us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:40.501907 (+103003us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.596434 (+ 94527us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.601292 (+  4858us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.601431 (+   139us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:40.603397 (+  1966us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.603594 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.603662 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.167705 (+1564043us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b18003c8380. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:43.158610 (+990905us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:43.535066 (+376456us) raft_consensus.cc:1685] finished
0709 13:01:43.535095 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.535298 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1812,"prepare.queue_time_us":103772,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":81,"raft.queue_time_us":4113,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":70412,"spinlock_wait_cycles":77749504,"thread_start_us":160612,"threads_started":2,"wal-append.queue_time_us":385359}
I20260709 13:01:43.530308  3731 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:43.564908  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 352) took 1758 ms. Trace:
I20260709 13:01:43.492895  3502 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:43.613152  3872 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.242s	user 0.001s	sys 0.001s
I20260709 13:01:43.607133  3855 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:43.646659  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.513105  3688 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:43.567564  3528 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.564810  3703 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:43.660207  3728 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.976672  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1688) took 1365 ms. Trace:
I20260709 13:01:43.976866  3728 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.977130  1724 rpcz_store.cc:276] 0709 13:01:42.610984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.789181 (+178197us) service_pool.cc:224] Handling call
0709 13:01:42.800039 (+ 10858us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.243622 (+443583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.811304 (+567682us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54001a7cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:43.976595 (+165291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018379648}
I20260709 13:01:43.978685  3664 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:43.699116  3878 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:01:43.979305  3664 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:43.719017  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1099) took 1031 ms. Trace:
W20260709 13:01:43.771246  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 326094us
W20260709 13:01:43.814965  1539 connection.cc:256] RPC call timeout handler was delayed by -4.4712330719999613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 13:01:43.780125  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.982270  1539 connection.cc:256] RPC call timeout handler was delayed by -4.319133072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 13:01:43.766184  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.980448  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 365857us
W20260709 13:01:43.983456  1539 connection.cc:256] RPC call timeout handler was delayed by -3.6981330719999619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2863s for a response
W20260709 13:01:43.984829  1539 connection.cc:256] RPC call timeout handler was delayed by -3.6909330719999502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2871s for a response
W20260709 13:01:43.985062  1539 connection.cc:256] RPC call timeout handler was delayed by -2.5281330720000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.41630000000000006s for a response
I20260709 13:01:43.985555  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:43.830229  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1102) took 1131 ms. Trace:
I20260709 13:01:43.992322  1567 rpcz_store.cc:276] 0709 13:01:42.698663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.102470 (+403807us) service_pool.cc:224] Handling call
0709 13:01:43.102889 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.289348 (+186459us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:43.289708 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.373976 (+ 84268us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.374467 (+   491us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.374535 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:43.374620 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.601091 (+226471us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b1800157360. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:43.663621 (+ 62530us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b1800157360. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419
0709 13:01:43.829797 (+166176us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54001aaed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:43.829859 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.830126 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":993,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":387,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":8862,"spinlock_wait_cycles":476284928,"thread_start_us":83332,"threads_started":1,"wal-append.queue_time_us":79565}
W20260709 13:01:43.750252  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1671) took 2568 ms (client timeout 1705 ms). Trace:
I20260709 13:01:43.980396  1569 rpcz_store.cc:276] 0709 13:01:42.687912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.688113 (+   201us) service_pool.cc:224] Handling call
0709 13:01:42.707333 (+ 19220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.707782 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:43.224102 (+516320us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.442272 (+218170us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b4800002dd8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:01:43.468236 (+ 25964us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.485719 (+ 17483us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.485904 (+   185us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:43.485939 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.486019 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.689534 (+203515us) raft_consensus.cc:1685] finished
0709 13:01:43.718538 (+ 29004us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.718874 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7505,"prepare.queue_time_us":458846,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":1343,"raft.queue_time_us":384,"raft.run_cpu_time_us":12320,"raft.run_wall_time_us":731592,"spinlock_wait_cycles":61381120,"thread_start_us":732215,"threads_started":2,"wal-append.queue_time_us":321816}
W20260709 13:01:44.000200  1720 rpcz_store.cc:269] 0709 13:01:41.181594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.246249 (+ 64655us) service_pool.cc:224] Handling call
0709 13:01:43.749840 (+2503591us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540010e0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:43.750118 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463915904}
I20260709 13:01:43.837666  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1686) took 1272 ms. Trace:
I20260709 13:01:44.003355  1717 rpcz_store.cc:276] 0709 13:01:42.607648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.765038 (+157390us) service_pool.cc:224] Handling call
0709 13:01:42.765460 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.541913 (+776453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.836932 (+295019us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54001702d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:43.837543 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784293760}
I20260709 13:01:43.573057  2379 rpcz_store.cc:276] 0709 13:01:41.806138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.806341 (+   203us) service_pool.cc:224] Handling call
0709 13:01:41.806891 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.904248 (+ 97357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.904296 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.905027 (+   731us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.159440 (+254413us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.159501 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.545307 (+1385806us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b54004acc54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:43.545522 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.545581 (+    59us) raft_consensus.cc:1685] finished
0709 13:01:43.545595 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.564804 (+ 19209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2201,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":116,"spinlock_wait_cycles":222227328,"thread_start_us":253842,"threads_started":1,"wal-append.queue_time_us":315304,"wal-append.run_cpu_time_us":4309,"wal-append.run_wall_time_us":1188245}
W20260709 13:01:43.997603  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.044185  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1109) took 1270 ms. Trace:
I20260709 13:01:44.044966  1575 rpcz_store.cc:276] 0709 13:01:42.774134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.488426 (+714292us) service_pool.cc:224] Handling call
0709 13:01:43.500977 (+ 12551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:43.502607 (+  1630us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:43.526356 (+ 23749us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.673548 (+147192us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:43.674448 (+   900us) log.cc:844] Serialized 52 byte log entry
0709 13:01:43.674641 (+   193us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:43.727535 (+ 52894us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.728037 (+   502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.728155 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.043670 (+315515us) raft_consensus.cc:1685] finished
0709 13:01:44.043715 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.044060 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24077,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":1356,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":13956,"thread_start_us":111865,"threads_started":1,"wal-append.queue_time_us":494503}
W20260709 13:01:44.053331  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 67834.8us
W20260709 13:01:44.054224  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.975394  3888 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
W20260709 13:01:43.699429  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.686434  3881 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:01:43.822175  2372 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:43.684636  3876 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:01:43.909392  3869 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.653s	user 0.014s	sys 0.003s
I20260709 13:01:43.664055  3836 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:43.825309  2371 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:43.685571  3688 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:43.835505  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:43.773509  3502 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:44.099472  3890 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:43.843822  2376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 347) took 2679 ms (client timeout 1907 ms). Trace:
I20260709 13:01:44.178970  3841 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:44.181182  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1103) took 1482 ms. Trace:
W20260709 13:01:44.185554  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 785271us
W20260709 13:01:44.186421  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.242794  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.272650  2342 connection.cc:256] RPC call timeout handler was delayed by -1.1672288159999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28140000000000004s for a response
I20260709 13:01:44.256676  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:44.273056  2342 connection.cc:256] RPC call timeout handler was delayed by -1.1672288159999766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28140000000000004s for a response
W20260709 13:01:44.194700  3782 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.408s	user 0.013s	sys 0.001s
I20260709 13:01:44.057983  3528 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.274420  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:44.275559  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.286548  1724 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:44.271371  1576 rpcz_store.cc:276] 0709 13:01:42.698965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.217830 (+518865us) service_pool.cc:224] Handling call
0709 13:01:43.218893 (+  1063us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:43.367202 (+148309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.367248 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:43.368047 (+   799us) log.cc:844] Serialized 52 byte log entry
0709 13:01:43.491965 (+123918us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:43.492012 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.492083 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.180770 (+688687us) raft_consensus.cc:1685] finished
0709 13:01:44.180910 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.181106 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147461,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"thread_start_us":123400,"threads_started":1,"wal-append.queue_time_us":148557}
I20260709 13:01:44.291569  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:44.294512  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:44.057564  3703 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.275559  3688 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.270102  2375 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:01:44.278028  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 223195us
I20260709 13:01:44.285930  3782 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6.
I20260709 13:01:44.318218  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:44.334672  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1098) took 1648 ms. Trace:
I20260709 13:01:44.335733  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:44.335100  1574 rpcz_store.cc:276] 0709 13:01:42.686113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.686276 (+   163us) service_pool.cc:224] Handling call
0709 13:01:42.686616 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.697180 (+ 10564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.697209 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.697743 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.225066 (+527323us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:43.225112 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.225191 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.334340 (+1109149us) raft_consensus.cc:1685] finished
0709 13:01:44.334370 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.334588 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15582,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":19442,"thread_start_us":466873,"threads_started":1,"wal-append.queue_time_us":538811}
I20260709 13:01:43.979305  3728 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.236361  3836 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.350921  1723 pending_rounds.cc:85] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 5
I20260709 13:01:44.351274  1723 pending_rounds.cc:107] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:01:44.351536  1723 raft_consensus.cc:2889] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.326470  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:44.353052  1719 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.328095  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:44.310981  3669 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.722s	user 0.011s	sys 0.023s
I20260709 13:01:44.346218  1717 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:44.356139  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 269835us
I20260709 13:01:44.347932  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.300832  3898 raft_consensus.cc:2764] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:44.338809  2374 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:44.357538  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.298022  3658 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:01:44.369045  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:44.370023  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.803s (SENT)
I20260709 13:01:44.357842  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.312747  2375 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:44.382965  1717 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:44.385028  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.390283  1716 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:01:44.357100  3669 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.771s	user 0.012s	sys 0.023s
W20260709 13:01:44.271114  2376 rpcz_store.cc:269] 0709 13:01:41.137566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.140599 (+  3033us) service_pool.cc:224] Handling call
0709 13:01:43.787536 (+2646937us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54006fed54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:43.800768 (+ 13232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469335936}
W20260709 13:01:44.384289  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 48625us
I20260709 13:01:44.302424  3688 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:44.385063  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.861s (SENT)
I20260709 13:01:44.397886  1719 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:44.398581  3897 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:44.382762  3895 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:01:44.399767  3892 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:01:44.400311  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1788) took 1860 ms. Trace:
I20260709 13:01:44.401871  3860 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:44.386746  3728 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:44.404579  3669 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:44.406288  3869 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.418107  3836 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.419718  3703 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.383158  3841 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.427805  3782 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:01:44.429642  3528 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.431291  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1106) took 1731 ms. Trace:
I20260709 13:01:44.432757  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:44.463007  1570 rpcz_store.cc:276] 0709 13:01:42.699796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.456657 (+756861us) service_pool.cc:224] Handling call
0709 13:01:43.467222 (+ 10565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.467584 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.467618 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.468164 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.968403 (+500239us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:43.968473 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.968580 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.431013 (+462433us) raft_consensus.cc:1685] finished
0709 13:01:44.431036 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.431219 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":9317,"thread_start_us":470359,"threads_started":1,"wal-append.queue_time_us":796994}
I20260709 13:01:44.303390  3890 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:01:44.354889  3664 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:44.434794  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49786.8us
I20260709 13:01:44.433966  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 270) took 1516 ms. Trace:
I20260709 13:01:44.468309  2378 rpcz_store.cc:276] 0709 13:01:42.917869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.520549 (+602680us) service_pool.cc:224] Handling call
0709 13:01:43.541798 (+ 21249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.734830 (+193032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.734884 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.770938 (+ 36054us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.314970 (+544032us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:44.315035 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.315139 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.433616 (+118477us) raft_consensus.cc:1685] finished
0709 13:01:44.433638 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.433906 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72850,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":49786,"thread_start_us":546549,"threads_started":2,"wal-append.queue_time_us":616683}
I20260709 13:01:44.442245  1721 rpcz_store.cc:276] 0709 13:01:42.539861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.671499 (+131638us) service_pool.cc:224] Handling call
0709 13:01:42.692997 (+ 21498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.711565 (+ 18568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.711595 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.712103 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.843066 (+1130963us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b18003c8260. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:01:43.996014 (+152948us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:43.996053 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.996116 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.399996 (+403880us) raft_consensus.cc:1685] finished
0709 13:01:44.400017 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.400248 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17725,"raft.queue_time_us":17862,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":41470,"spinlock_wait_cycles":5042432,"thread_start_us":152709,"threads_started":1,"wal-append.queue_time_us":1686259}
W20260709 13:01:44.440062  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 140572us
W20260709 13:01:44.447528  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 173199us
I20260709 13:01:44.473918  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:44.474273  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:44.446630  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:44.457751  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.433573  3903 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:44.435565  2374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 350) took 2061 ms (client timeout 1738 ms). Trace:
I20260709 13:01:44.439536  3897 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:44.480489  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:44.486959  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 92640.6us
I20260709 13:01:44.479066  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 176) took 4727 ms. Trace:
I20260709 13:01:44.498066  1651 rpcz_store.cc:276] 0709 13:01:39.743523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.235490 (+491967us) service_pool.cc:224] Handling call
0709 13:01:42.432766 (+2197276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.444790 (+ 12024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.471084 (+2026294us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b540005c3d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:44.471256 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5052849024}
I20260709 13:01:44.487092  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:44.487594  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.500157  1721 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:44.439266  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:44.451242  3860 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.494050  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:44.495448  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 273) took 1448 ms. Trace:
I20260709 13:01:44.512456  2377 rpcz_store.cc:276] 0709 13:01:43.047692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.533429 (+485737us) service_pool.cc:224] Handling call
0709 13:01:43.533859 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.534268 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.534300 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.534877 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.671513 (+136636us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:43.916594 (+245081us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.917189 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.917323 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.495238 (+577915us) raft_consensus.cc:1685] finished
0709 13:01:44.495258 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.495386 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":823364,"raft.queue_time_us":80625,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":5743,"thread_start_us":362189,"threads_started":2,"wal-append.queue_time_us":704838}
I20260709 13:01:44.478081  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:44.504626  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1112) took 1349 ms. Trace:
I20260709 13:01:44.505762  2376 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:01:44.460728  3869 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.506263  2372 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:01:44.506707  3669 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:44.508464  3731 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.847s	user 0.014s	sys 0.000s
I20260709 13:01:44.515612  1573 rpcz_store.cc:276] 0709 13:01:43.154916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.562182 (+407266us) service_pool.cc:224] Handling call
0709 13:01:44.467291 (+905109us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:44.467573 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.504121 (+ 36548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.504169 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.504227 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:44.504306 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.504414 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.504547 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25582,"raft.queue_time_us":43222,"spinlock_wait_cycles":14752000}
W20260709 13:01:44.521759  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43759.5us
I20260709 13:01:44.523963  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:44.488586  3897 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.531948  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:44.496656  3728 raft_consensus.cc:2764] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:44.489686  3782 consensus_queue.cc:1243] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 5.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:44.497442  3528 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:44.541111  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.541399  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.541952  1576 raft_consensus.cc:2410] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:44.540015  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.542230  1567 raft_consensus.cc:2410] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:44.542805  1574 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:44.493232  2371 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:44.544107  1574 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:44.547272  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.550133  1576 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:44.550755  1567 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: [3.6-4.7]
I20260709 13:01:44.518952  3658 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:44.554565  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:44.555155  2378 raft_consensus.cc:2376] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6. Re-sending same reply.
I20260709 13:01:44.555459  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:44.556049  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.557762  1575 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:44.557947  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.556872  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:44.558741  1576 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:44.497289  3836 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:01:44.528146  3895 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:44.559536  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.566370  1573 raft_consensus.cc:2410] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:44.544204  3528 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.560439  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:44.567502  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.567901  2380 raft_consensus.cc:2376] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5. Re-sending same reply.
I20260709 13:01:44.568140  1575 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:44.561074  2377 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.554396  1718 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:44.570792  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.571993  1570 raft_consensus.cc:2410] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:44.561728  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:44.510026  3703 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:01:44.562852  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:44.478155  2374 rpcz_store.cc:269] 0709 13:01:42.371629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.373723 (+  2094us) service_pool.cc:224] Handling call
0709 13:01:43.835144 (+1461421us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:44.391845 (+556701us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4277632}
I20260709 13:01:44.476469  3841 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.547425  3664 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:44.546627  3907 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:44.564481  3782 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:44.557173  3909 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:44.565924  3895 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:44.576985  3528 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.560091  3731 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 13:01:44.620932  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1774) took 2090 ms (client timeout 1888 ms). Trace:
W20260709 13:01:44.623821  1725 rpcz_store.cc:269] 0709 13:01:42.531041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.531310 (+   269us) service_pool.cc:224] Handling call
0709 13:01:44.620645 (+2089335us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54001a86d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:44.620839 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371959552}
W20260709 13:01:44.624769  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63102.2us
I20260709 13:01:44.581341  3664 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:44.622886  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.627281  1570 raft_consensus.cc:2410] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:01:44.623008  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:44.563359  3908 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:01:44.580762  3658 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:01:44.629642  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:44.655279  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.670399  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 133487us
I20260709 13:01:44.694685  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:44.694977  1575 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:44.622462  3739 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:44.620713  3895 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:44.634326  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.622512  3703 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.696470  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1113) took 1513 ms. Trace:
I20260709 13:01:44.665683  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.699959  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:44.700240  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.700743  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.700834  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.727131  1651 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:44.727747  1643 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:44.701375  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.701777  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.693681  3913 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:44.702921  1649 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:44.700032  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 186) took 1214 ms. Trace:
I20260709 13:01:44.626610  3782 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:44.721553  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.722289  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.734705  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.735504  1567 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:44.736581  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.723418  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.703481  3860 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.726694  1569 rpcz_store.cc:276] 0709 13:01:43.182984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.000973 (+817989us) service_pool.cc:224] Handling call
0709 13:01:44.124321 (+123348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.496214 (+371893us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:44.534700 (+ 38486us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.696079 (+161379us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.696146 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:44.696198 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.696273 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.696410 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24878,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4422,"raft.queue_time_us":212073,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":5302,"thread_start_us":454169,"threads_started":2,"wal-append.queue_time_us":149618}
I20260709 13:01:44.729580  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.729377  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.731302  1650 rpcz_store.cc:276] 0709 13:01:43.486136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.838893 (+352757us) service_pool.cc:224] Handling call
0709 13:01:43.839583 (+   690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:43.839954 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.839992 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:43.848924 (+  8932us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.433326 (+584402us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:44.433366 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.433441 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:44.699834 (+266393us) raft_consensus.cc:1685] finished
0709 13:01:44.699854 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.699981 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":450575,"threads_started":1,"wal-append.queue_time_us":603447}
I20260709 13:01:44.737908  1573 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:44.741477  1642 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:44.626489  3664 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:44.703754  3890 consensus_queue.cc:1243] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer da3b57321031416cba9db75450ef302a log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:01:44.704229  3910 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:01:44.694124  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.740119  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:44.703850  3900 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:01:44.712989  3841 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:44.744683  1575 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:44.711583  3739 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.742542  3917 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:44.730662  3913 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:44.755733  1722 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.759169  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:44.747923  3782 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:44.752825  3739 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:44.776607  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.782948  1648 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 2.
I20260709 13:01:44.779358  2375 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.732221  3703 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.781704  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.784092  1718 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:44.758661  3869 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:44.789619  3897 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.797006  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.800199  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:44.800911  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.804857  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:44.805568  1575 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:44.806864  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.807510  1648 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:44.809070  2375 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.815996  1647 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:44.815912  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:44.817395  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.818167  1643 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:44.826143  1648 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:44.826421  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.826855  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.827008  1721 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:44.827509  3917 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:44.830962  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.785050  3528 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.837798  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:44.837453  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.841840  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:44.842552  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.848896  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.849234  1643 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:44.850046  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.852279  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.858858  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.864990  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.866204  1648 raft_consensus.cc:2410] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:44.867602  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:44.868860  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.873739  1648 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.876554  1651 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.876924  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:44.877480  1721 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:44.877861  1721 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:44.880285  3926 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.886346  3926 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.889017  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.888928  3895 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.879895  1721 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.890610  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:44.872462  3658 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:44.891091  1721 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.891232  1649 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.891665  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.904662  3703 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.905440  3528 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.784057  3913 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.906464  3913 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:44.894018  3890 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:44.894747  3915 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:44.913033  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1683) took 2348 ms. Trace:
I20260709 13:01:44.913442  1721 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:44.913424  3915 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.914903  3664 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:44.917052  3911 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 13:01:44.917472  3915 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.917713  3664 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:44.905653  3897 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:44.914891  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.919437  1570 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:44.837643  3917 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.919948  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.921052  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.922439  1721 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.913468  1722 rpcz_store.cc:276] 0709 13:01:42.564449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.718067 (+153618us) service_pool.cc:224] Handling call
0709 13:01:42.718418 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.756176 (+2037758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.912791 (+156615us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54003dd1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:44.912970 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4685279488}
I20260709 13:01:44.924888  1649 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:44.925675  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.925992  1722 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.926294  1649 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
W20260709 13:01:44.894021  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0647287699999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
I20260709 13:01:44.812597  3782 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.902711  2375 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:01:44.907354  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.895582  3926 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:44.906456  3762 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.523s	user 0.007s	sys 0.009s
I20260709 13:01:44.909251  3890 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.895254  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.936470  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:44.928557  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:44.928380  1722 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.938858  1722 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:44.945945  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:44.946449  3871 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:44.946799  3871 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:44.908705  3658 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:44.949129  3658 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:44.950258  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:44.951074  1716 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:44.952831  3895 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.959072  3890 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:44.944168  3837 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:44.964911  3837 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:44.965767  3837 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:44.940083  3762 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5.
I20260709 13:01:44.969481  1644 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:44.933750  2375 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 5
I20260709 13:01:44.971601  2375 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:01:44.971909  2375 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.972661  3688 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:44.999706  3658 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:45.000069  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:45.001384  3898 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:45.001752  3898 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.002060  3898 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:44.972718  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 279) took 1665 ms. Trace:
I20260709 13:01:45.005043  2375 rpcz_store.cc:276] 0709 13:01:43.307749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.567445 (+259696us) service_pool.cc:224] Handling call
0709 13:01:44.237405 (+669960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.778975 (+541570us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5400472820. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833cec06f 00007fc833d64f60 00007fc833d667de 00007fc833d70b49 00007fc833d71416 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62
0709 13:01:44.972425 (+193450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.972657 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37444352}
I20260709 13:01:45.022329  3853 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:45.022769  3853 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:45.023008  3693 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:45.023414  3693 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:45.026628  3898 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.029870  3927 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:45.030227  3898 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.030335  3927 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:45.020187  3739 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.038033  3927 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.039132  3923 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:45.039462  3923 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:45.039525  3927 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:45.040030  3923 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.041206  3923 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:44.927237  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0647287699999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 13:01:45.073915  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0611287699999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 13:01:45.074656  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0611287699999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:01:44.922323  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1110) took 2144 ms. Trace:
W20260709 13:01:45.078886  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 201888us
I20260709 13:01:45.083164  1649 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:01:45.083055  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.814s (SENT)
W20260709 13:01:45.084755  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.085541  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.086154  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.087950  1570 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:45.089267  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:45.089562  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.097352  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:45.098434  1716 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:45.075073  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0476287700000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 13:01:45.101043  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0476287700000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:01:45.102094  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:45.104885  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:45.107144  1572 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.108613  1723 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.105293  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:45.110963  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.112074  3664 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:45.113405  2379 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:01:45.113068  3890 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.113479  3664 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.023734  3853 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.114527  3890 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:45.115150  3890 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.115729  3853 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.116884  3855 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.103716  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:45.100136  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:45.116956  3890 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.079656  1568 rpcz_store.cc:276] 0709 13:01:42.777559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.522404 (+744845us) service_pool.cc:224] Handling call
0709 13:01:44.061658 (+539254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.430085 (+368427us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:44.430507 (+   422us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.613748 (+183241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.615336 (+  1588us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:44.621091 (+  5755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.922089 (+300998us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400026cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:44.922132 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.922267 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2431,"prepare.queue_time_us":3326,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":31078,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":9636,"spinlock_wait_cycles":730059520,"thread_start_us":136580,"threads_started":1,"wal-append.queue_time_us":219851}
I20260709 13:01:45.135915  3909 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:45.136270  1642 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:45.126250  3871 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.138881  1572 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:45.140374  3890 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.140328  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.141896  1646 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:45.143216  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.140771  1576 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:45.153949  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.156458  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.158445  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.158957  3871 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.164610  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:45.169335  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.181962  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.69523e+06us
I20260709 13:01:45.198696  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:45.199898  3502 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.200817  3502 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.200989  3693 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.169341  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:45.203747  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.206444  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1111) took 2100 ms. Trace:
I20260709 13:01:45.206871  1571 rpcz_store.cc:276] 0709 13:01:43.105637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.547670 (+442033us) service_pool.cc:224] Handling call
0709 13:01:45.205152 (+1657482us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:45.205565 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.205977 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.206002 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.206061 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:45.206096 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.206156 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.206262 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7686912}
I20260709 13:01:45.203894  3502 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:45.208752  3915 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:45.209406  1650 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:45.209630  3915 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.212179  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:45.213356  3502 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:45.213815  3502 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:45.215145  3502 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:45.216701  3926 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:45.217494  3926 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.220129  3890 raft_consensus.cc:2764] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:45.189327  3664 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:45.213588  3927 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:45.222633  1571 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.235662  1572 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.222973  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:45.191329  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 88001.6us
W20260709 13:01:45.224460  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 116866us
I20260709 13:01:45.232869  1574 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:45.205104  3841 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:45.248103  1574 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.248836  3841 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:45.249354  3841 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.249814  1574 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.251541  1574 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.240676  3927 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.253187  3927 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:45.232658  3502 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.256596  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.257793  1649 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:45.258685  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.259238  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.260740  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.261586  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.262944  1647 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:45.262501  1649 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:45.268752  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 169198us
I20260709 13:01:45.269948  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:45.272143  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:45.272035  1649 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:45.281834  3502 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:45.282527  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:45.283838  3928 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:45.285158  3917 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:45.285689  3917 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.223976  1568 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.254892  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.220315  3926 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:45.301396  1716 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:45.227699  3693 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.222295  3890 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.286171  3917 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.313328  1570 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:45.313089  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.313565  3739 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.318861  1717 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:45.319041  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.320128  1650 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:45.326437  1574 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:45.329784  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.330528  1650 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.330595  1643 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.331436  3915 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.331977  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.334488  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.335393  1649 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:45.336321  1650 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
W20260709 13:01:45.337081  3903 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:01:45.333395  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.338169  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:45.338538  1575 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:45.339890  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:45.342620  1643 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:45.346401  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.347697  3837 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.348066  1651 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:45.350044  3890 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.358705  3915 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:45.361145  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:45.368785  1644 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:45.360762  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54670.7us
I20260709 13:01:45.377370  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.378252  1646 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:45.379407  3923 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:45.391093  3559 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.391887  3861 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:45.392709  3861 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.393443  3861 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.374331  3739 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:45.396572  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.397181  3739 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.399618  3739 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:45.391839  3559 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.405881  3559 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:45.386107  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68449.1us
W20260709 13:01:45.390596  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50894.7us
I20260709 13:01:45.378635  3855 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:45.382180  3926 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:45.384804  1568 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:45.443233  1568 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.444489  1568 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.445730  1568 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.440198  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.435549  3918 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.439347  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:45.441167  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54180.9us
I20260709 13:01:45.441637  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:45.451154  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.453550  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.461678  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.387286  3903 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:45.472321  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:45.458129  3915 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:45.479241  3918 consensus_queue.cc:1243] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:45.465790  3926 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.491000  3926 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.491832  1646 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:45.496114  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.460839  3917 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.497997  1571 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:45.499378  3917 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:45.500556  3907 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:45.501317  3907 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:45.491559  3927 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.502573  3907 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.506006  3664 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:45.506557  3664 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.506942  3664 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.515348  3927 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:45.533305  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:45.552572  3739 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:45.553224  3739 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:45.554628  3739 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.539134  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:45.533849  3890 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.515908  3903 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.566144  3688 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:45.569293  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50725.8us
I20260709 13:01:45.584825  1724 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:45.587715  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.589246  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:45.589505  1472 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 216) took 18992 ms (client timeout 15000 ms). Trace:
I20260709 13:01:45.570166  3927 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:01:45.592240  3927 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:45.592620  3927 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:45.593623  3927 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.594852  3927 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:45.589874  1472 rpcz_store.cc:269] 0709 13:01:26.596918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.601424 (+  4506us) service_pool.cc:224] Handling call
0709 13:01:27.845109 (+1243685us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:01:45.589264 (+17744155us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:39.773158 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:39.773906 (+   748us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:39.773965 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:01:39.773992 (+    27us) tablet.cc:662] Decoding operations
0709 13:01:39.775242 (+  1250us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:39.775394 (+   152us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:39.775432 (+    38us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:01:39.776293 (+   861us) tablet.cc:701] Row locks acquired
0709 13:01:39.782004 (+  5711us) write_op.cc:260] PREPARE: finished
0709 13:01:39.804510 (+ 22506us) write_op.cc:270] Start()
0709 13:01:39.856064 (+ 51554us) write_op.cc:276] Timestamp: P: 1783602099797730 usec, L: 0
0709 13:01:39.935353 (+ 79289us) op_driver.cc:348] REPLICATION: starting
0709 13:01:39.936392 (+  1039us) log.cc:844] Serialized 2094 byte log entry
0709 13:01:41.463229 (+1526837us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:41.760934 (+297705us) op_driver.cc:464] REPLICATION: finished
0709 13:01:42.030780 (+269846us) write_op.cc:301] APPLY: starting
0709 13:01:42.030924 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:42.031191 (+   267us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:42.031242 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:42.137215 (+105973us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:42.159141 (+ 21926us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:01:42.160586 (+  1445us) write_op.cc:312] APPLY: finished
0709 13:01:42.191688 (+ 31102us) log.cc:844] Serialized 47 byte log entry
0709 13:01:42.201675 (+  9987us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:42.204770 (+  3095us) write_op.cc:454] Released schema lock
0709 13:01:42.207819 (+  3049us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:42.223477 (+ 15658us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:42.224941 (+  1464us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5527,"reported_tablets":22,"spinlock_wait_cycles":1943388032,"child_traces":[["op",{"apply.queue_time_us":269459,"apply.run_cpu_time_us":6549,"apply.run_wall_time_us":187543,"num_ops":5,"prepare.queue_time_us":393782,"prepare.run_cpu_time_us":11574,"prepare.run_wall_time_us":307942,"raft.queue_time_us":134094,"raft.run_cpu_time_us":43861,"raft.run_wall_time_us":272372,"replication_time_us":1825382,"spinlock_wait_cycles":6550400,"thread_start_us":695763,"threads_started":4,"wal-append.queue_time_us":143929,"wal-append.run_cpu_time_us":24342,"wal-append.run_wall_time_us":3117033}]]}
I20260709 13:01:45.604677  1643 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:45.573909  3915 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:45.609668  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.610682  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.578850  3739 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.615796  3688 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:45.620146  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.621225  1567 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.624127  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:45.632086  1571 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.625557  1572 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:45.634069  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:45.635792  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:45.636684  3853 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:45.636636  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.637667  1570 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:45.638432  3909 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:45.644788  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.645689  2375 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:01:45.647033  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.645673  1717 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:45.652482  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:45.653250  2375 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:45.654984  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.655779  1717 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:45.656116  1576 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:45.652968  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:45.658947  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:45.660051  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.660658  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:45.660943  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:45.661911  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.668082  2373 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:45.670550  3841 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.671808  1717 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:01:45.673511  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68976.3us
I20260709 13:01:45.674459  3903 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.675128  3915 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.675549  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:45.676690  3941 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:45.676358  3915 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:45.677230  3941 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.678207  3941 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.679020  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:45.680259  1573 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:45.680996  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:45.682447  1569 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:45.685039  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.686642  3923 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:45.690450  1572 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:45.691272  1572 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.692380  3861 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:45.694159  3943 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:01:45.695416  1569 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:45.686992  1568 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:45.700475  1571 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:45.659922  1567 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:45.717700  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.722304  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:45.732728  1648 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:45.702531  3890 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:45.736676  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:45.738775  1576 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.739629  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.740271  1575 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
W20260709 13:01:45.737144  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 163448us
I20260709 13:01:45.735598  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.742682  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:45.746372  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45241.3us
I20260709 13:01:45.732808  1572 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.753865  1717 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:45.755708  3782 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.756428  3782 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.757886  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:45.758644  1718 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:45.758834  3782 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.746868  3909 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.772707  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.774719  1571 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:45.744992  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:45.784961  1471 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 4, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.786264  3903 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:45.752600  3890 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:45.787727  1471 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:01:45.792166  1471 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:45.794373  1720 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:45.797969  1572 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.787632  3909 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.789600  3890 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.764873  3915 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:45.795754  1471 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.813269  1576 raft_consensus.cc:2410] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:01:45.818602  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65318.8us
I20260709 13:01:45.835510  1471 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 4, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:45.863968  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:45.866677  3941 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:45.867364  3941 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:45.868480  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.868657  3941 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.870319  3941 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.872534  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:45.874174  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.877318  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 100965us
I20260709 13:01:45.884734  1471 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 3 to 5, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:45.900596  3664 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:45.901420  3841 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:45.902937  1570 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:45.904989  3664 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.913563  3909 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:45.919096  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.919766  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.920560  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.921132  1644 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.921271  1651 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.922178  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:45.923107  1648 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:45.923957  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.925412  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:45.926795  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:45.909754  1471 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:45.930914  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.928176  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:45.935724  1571 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:45.936017  2376 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:45.938033  1573 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:01:45.942093  3945 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.212s	user 0.003s	sys 0.002s
W20260709 13:01:45.950196  3693 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
I20260709 13:01:45.951668   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:45.952292  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 55978.2us
I20260709 13:01:45.910192  3861 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:45.956766  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:45.958061  3950 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:01:45.960254  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.961866  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:45.955440  1471 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:45.968638  1646 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:45.971920  1644 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:01:45.971660  3739 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:45.973815  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:45.977036  1643 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:45.968248  1688 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.985s (SENT)
W20260709 13:01:45.977672  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 84286.6us
W20260709 13:01:45.987643  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.998208  1572 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:46.005165  1471 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 3, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.007766  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.009055  1643 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:46.012439  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:46.013289  2373 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:46.018570  1539 leader_election.cc:400] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:46.018914  1539 leader_election.cc:403] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.021095  3903 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:46.022341  3502 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:46.023032  3507 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.020146  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:46.028568  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:46.029634  3786 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:46.030263  3786 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:46.031337  3786 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.033651  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:46.035153  3502 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:46.008304  1688 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Timed out: RequestConsensusVote RPC to 127.0.107.196:45309 timed out after 1.985s (SENT)
I20260709 13:01:46.038633  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:46.040812  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.037634  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:46.044539  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:46.008798  3861 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:01:45.999902  3915 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:46.053635  1571 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:46.055433  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.058828  1570 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:46.036811  3853 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:46.062654  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.043920  1471 catalog_manager.cc:5697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:45.980520  3871 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 13:01:46.069615  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.068483  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 60237.4us
W20260709 13:01:46.074198  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.074194  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.075021  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.075775  1716 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:46.077518  1646 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:46.009414  3928 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:46.084872  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.067466  3853 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.084695  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:46.092830  1646 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:46.085146  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:46.098208  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.071928  3507 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:46.106678  1723 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:46.068336  3963 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:46.059110  3837 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:01:46.111042  1471 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 3, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:46.119397  1717 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:46.122944  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.123096  3837 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:46.123672  3837 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:46.119568  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.126714  1643 raft_consensus.cc:2410] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:46.126833  1716 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:46.097018  3853 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.149868  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:46.115898  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:46.143370  3841 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:46.157342  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.137378  1471 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:01:46.169480  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51770.6us
I20260709 13:01:46.164891  1646 raft_consensus.cc:2376] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3. Re-sending same reply.
I20260709 13:01:46.175410  1471 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:46.158252  3963 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:46.190917  1646 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:01:46.191704  1646 pending_rounds.cc:85] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 3
I20260709 13:01:46.192938  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.194371  1647 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:46.198690  1648 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:46.200757  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:46.201180  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.201829  1651 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:46.209393  1471 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 3, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:46.212296  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.213491  1645 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:46.217723  3837 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.236912  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:46.192162  1646 pending_rounds.cc:107] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:01:46.185999  3915 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.224481  3658 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.321007  3658 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.323436  3658 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.169219  2375 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:46.314122  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.324671  1643 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:01:46.317438  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.307785  1471 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:46.139990  3952 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:01:46.319824  1646 raft_consensus.cc:2889] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:46.337030  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.337742  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.338111  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.338472  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.316627  3965 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.088s	user 0.000s	sys 0.004s
I20260709 13:01:46.345012  1642 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:46.352126  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.353660  1651 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:46.360807  3918 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:46.361388  3918 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:46.365454  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46157.1us
W20260709 13:01:46.367769  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.375213  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.384419  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:46.386969  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:46.393728  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.393977  3971 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:46.394469  1688 connection.cc:256] RPC call timeout handler was delayed by -1.624024211999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32930000000000004s for a response
I20260709 13:01:46.394786  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.395746  3970 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.396034  3853 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.396699  3974 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.397135  3969 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:46.397234  3974 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.397708  3969 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.398285  3853 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:46.399216  3961 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.005s
I20260709 13:01:46.399857  3974 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.380856  3739 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 13:01:46.394825  1688 connection.cc:256] RPC call timeout handler was delayed by -1.3462721409999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:01:46.411355  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1473721409999484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 13:01:46.411823  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1473721409999484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 13:01:46.412226  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0294721409999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 13:01:46.412684  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0006721409999386s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
I20260709 13:01:46.396358  3970 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.408289  3693 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:46.425696  3971 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:46.461648  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.404291  3890 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:01:46.463425  3871 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:01:46.467231  3739 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.467898  3739 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.468420  3507 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:46.468504  3972 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:46.468925  3507 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:46.470057  3507 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.470942  3871 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:46.471385  3871 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:46.471050  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 101358us
I20260709 13:01:46.473482  3969 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.464949  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.457747  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:46.441592  3693 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:46.489470  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:46.497320  3909 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:46.496583  3693 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.490263  1645 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:46.498302  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.471324  3507 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.497792  3909 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.461896  3971 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.507331  3782 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:46.504491  3786 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.011s
W20260709 13:01:46.516661  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.498950  3693 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:46.509593  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 112760us
I20260709 13:01:46.472605  3871 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.525830  3871 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.512449  3909 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.530580  3970 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.531531  3970 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.407666  3918 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.533229  3918 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.514739  3971 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.500273  3853 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:46.536849  3853 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.408520  3963 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.537508  3853 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.470059  3739 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.541210  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.542871  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.544561  3915 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.544636  3890 raft_consensus.cc:2764] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:01:46.545265  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:46.553406  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47861.5us
I20260709 13:01:46.555745  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.556313  3976 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:46.557226  3976 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.558136  3962 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.554359  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:46.557552  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.560137  3907 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:46.560580  3907 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.560905  3907 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.562055  3915 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.563349  3915 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.563874  3915 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.573172  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.588675  3915 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:46.589133  3903 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.580791  3984 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.591487  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.592893  3664 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:46.592072  3984 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.593405  3664 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:46.594511  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.594467  3941 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:46.596596  3941 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.597342  3941 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:46.597857  3941 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:46.599262  3941 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:46.600534  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.610776  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.613140  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:46.614076  1645 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:46.597487  3984 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:46.623512  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78390.2us
I20260709 13:01:46.594529  3664 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.624784  3664 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.624944  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.625926  3786 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.629142  3941 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.627975  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:46.622826  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 125924us
I20260709 13:01:46.651520  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:46.652560  2379 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:46.656236  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.656667  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65141.3us
W20260709 13:01:46.702859  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55237.6us
W20260709 13:01:46.708199  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 210013us
I20260709 13:01:46.655609  3890 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.731669  2379 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:46.745419  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:46.748054  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 120160us
I20260709 13:01:46.711549  3976 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:46.753353  3994 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:46.757084  2373 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:01:46.709147  3952 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.760391  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.758285  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:46.762703  1648 raft_consensus.cc:606] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:46.772641  3999 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.773329  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:46.776916  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:46.777712  1724 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.780809  3909 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.775743  1718 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:46.781251  1719 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:46.787845  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:46.788236  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.789000  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:46.774499  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 368) took 1115 ms. Trace:
I20260709 13:01:46.794255  3973 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:46.794605  2375 rpcz_store.cc:276] 0709 13:01:45.659413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.660283 (+   870us) service_pool.cc:224] Handling call
0709 13:01:46.774205 (+1113922us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540064f354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:46.774316 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655841024}
I20260709 13:01:46.798017  3853 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.792579  1716 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:46.792179  3996 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:46.808645  3973 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.809376  3973 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.810180  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:52714
I20260709 13:01:46.810972  1573 raft_consensus.cc:606] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.811331  1573 raft_consensus.cc:614] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:46.790668  3999 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.812610  3972 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:46.813517  3972 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:46.814599  3971 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.783378  3909 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.815205  3972 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.819541  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:01:46.819823  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59564.4us
I20260709 13:01:46.820407  1648 raft_consensus.cc:606] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.820804  3996 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:46.821007  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.821305  3972 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:46.822098  3996 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.829447  3973 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.838480  1724 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:46.829751  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:46.829892  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.831188  3957 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:46.834144  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.787390  3963 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.822710  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:46.842079  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.843498  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.844142  3739 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:46.845105  3502 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.846426  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:46.847700  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:46.860980  1723 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.840051  1648 raft_consensus.cc:606] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:46.861613  1648 raft_consensus.cc:614] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:46.780128  3907 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.866025  3739 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.868081  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.866595  3664 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.864473  3786 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:46.875397  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52824.6us
I20260709 13:01:46.876624  3999 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:46.877265  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:46.877849  3999 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:46.878321  3999 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:46.875536  3871 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:46.883886  3992 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.884450  3664 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.875828  3786 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:46.872265  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59667.5us
I20260709 13:01:46.880043  1569 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:46.890271  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.893911  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.903087  1724 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
W20260709 13:01:46.904719  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.554s (SENT)
I20260709 13:01:46.905366  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.906580  1575 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:46.871918  3952 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.908625  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:46.911923  3658 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.884047  3999 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.906150  3853 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.920727  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.925465  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:46.926831  1645 raft_consensus.cc:606] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:46.927323  1645 raft_consensus.cc:614] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:46.910346  3871 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:46.917610  3957 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:46.941354  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.921588  3971 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.947427   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:46.943606  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:46.949955  3999 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.966873  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:46.967648  1724 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:46.959421   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:46.936414  3507 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:46.979493  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.953038  3739 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:46.953158  3871 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:46.954706  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:46.960937  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.898061  3992 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:46.965289  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:46.966612  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:46.995317  3918 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:46.998512  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:46.999313  1725 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.999845  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:47.000530  1648 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:47.000756  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56421.5us
I20260709 13:01:47.002184  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.003436  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.005618  3890 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.008929  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.007411  1648 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:47.011049  3907 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.014230  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:46.978329  3853 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 13:01:47.020233  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.984465  3507 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:47.029737  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.996764  3971 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:47.035578  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92093.7us
I20260709 13:01:47.012199  3786 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.048956  1724 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:47.051713  1721 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:47.053593  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:47.054598  1718 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.060854  3786 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.061285  3923 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:47.035905  3971 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.033247  3918 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.069473  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:47.030072  3688 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:47.030535  3507 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.034477  3890 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.113991  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:47.016997  3996 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
W20260709 13:01:47.112602  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 210299us
I20260709 13:01:47.144698  1723 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:47.071640  3971 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.071089  3837 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:47.136870  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61769.1us
W20260709 13:01:47.135757  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.135627  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.214196  1725 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:47.222074  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 322690us
I20260709 13:01:47.067548  3923 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.063241  3786 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:47.233021  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 124007us
I20260709 13:01:47.221843  3941 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.074827  3992 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.244937  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.236992  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:47.136641  3973 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.113505  3782 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:47.199276  3970 raft_consensus.cc:993] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: : Instructing follower 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:47.216513  3871 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:47.258581  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.160081  3688 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:47.250727  3853 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.160442  3890 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.295077  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:47.302083  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65200.1us
I20260709 13:01:47.275375  3970 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:47.282153  4019 raft_consensus.cc:1081] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Signalling peer 58cdb682fbf54792a702a7a455f265a2 to start an election
I20260709 13:01:47.275988  3853 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:47.319325  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.243736  3923 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.266611  3973 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.274976  3871 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.324121  3853 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.349947  3973 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.332412  3782 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:47.460942  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.428887  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.486519  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 167342us
W20260709 13:01:47.466521  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 149368us
W20260709 13:01:47.482776  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 153349us
I20260709 13:01:47.521627  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.335825  3970 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:47.382723  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:47.380872  4015 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.172s	user 0.000s	sys 0.002s
W20260709 13:01:47.388217  3972 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.005s
W20260709 13:01:47.465062  4020 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:01:47.355206  4026 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:47.487425  3871 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.485494  3861 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:47.364172  3941 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:47.386395  3907 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:47.538136  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.486716  4027 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:47.522243  3926 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.525007  3890 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.344485  3962 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.200s
I20260709 13:01:47.460680  3669 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.202s
W20260709 13:01:47.575251  3997 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:01:47.542851  3992 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:47.570878  3861 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:47.610163  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88693us
I20260709 13:01:47.576254  4031 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:01:47.352998  3688 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:47.653980  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 116507us
I20260709 13:01:47.582947  3907 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.608238  3972 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:47.662688  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.625118  3992 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:47.662984  3923 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.656251  3837 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.684823  3861 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.685171  3837 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.685824  3992 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.686411  3861 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.688941  4019 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:47.689360  4025 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:01:47.691908  4031 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.666107  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.693478  4031 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:47.693976  4031 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:47.694046  1716 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:01:47.695055  4031 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.669521  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:47.671790  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.696851  1645 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:47.705299  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.706641  3837 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:47.683362  3507 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.709306  1717 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:47.674826  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58423.3us
I20260709 13:01:47.710768  3907 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.599165  3926 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.713478  3507 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:47.714532  3918 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:47.714725  3926 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.715730  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:47.716720  3909 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:47.717485  3909 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:47.717830  4032 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:01:47.599431  3941 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.719518  3909 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.721513  3909 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.722199  3941 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:47.723120  3941 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:47.724777  3739 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:01:47.684131  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 69225.6us
W20260709 13:01:47.729875  1718 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.675s	user 0.005s	sys 0.013s
I20260709 13:01:47.730429  1718 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:47.730556  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:47.672187  3972 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.673528  3890 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.732625  3972 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.733144  3890 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.734164  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:47.735829  4014 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:47.736799  3502 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:47.658309  3970 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.628103  3782 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.744685  3837 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:47.748214  1573 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:01:47.753324  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:47.754664  1645 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:47.758594  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:47.683014  3853 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.723875  3941 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:47.658766  3957 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:47.764468  1574 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:01:47.668120  4020 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:01:47.771433  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:47.761592  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:47.773957  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:47.776872  1572 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.763746  3658 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:47.767180  3890 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.782563  3658 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.782727  2376 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:47.806275  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:47.807009  1569 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.786219  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.807791  1718 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:47.786723  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.809535  3658 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.798411  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.792891  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:47.784114  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:47.852396  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:47.835008  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:47.856272  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:47.835780  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:47.844575  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74486.8us
I20260709 13:01:47.867343  1721 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:47.861157  1570 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:47.868038  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 92512.5us
W20260709 13:01:47.897781  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 179665us
W20260709 13:01:47.898252  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47671.6us
I20260709 13:01:47.914422  1572 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:47.799674  3971 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.558s	user 0.014s	sys 0.000s
I20260709 13:01:47.934509  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:47.789698  3890 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.928782  1719 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:01:47.946897  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 81133.1us
I20260709 13:01:47.853318  1720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:01:47.952723  1720 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.953298  1720 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.953635  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:47.893996  3907 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.976708  3907 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.959939  3976 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:47.983774  3976 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:47.984650  3786 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:47.986009  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:47.987516  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 57524us
I20260709 13:01:47.988732  4046 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:47.989308  4046 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:47.989717  4046 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:47.991503  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.930452  4014 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:01:47.993654  4013 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:47.957134  3890 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:47.998221  3890 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:47.998517  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:47.999969  4013 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:47.948827  3971 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.002197  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:48.015548  3669 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.015810  3971 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.018848  3976 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:48.026221  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 191387us
I20260709 13:01:48.029389  1573 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:48.030575  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.961477  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:47.848858  3957 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.964107  2376 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:47.969249  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:47.945847  3739 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.003830  3782 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:47.945423  3837 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.042685  3923 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.042928  3782 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.043315  3837 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:48.043447  3923 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:47.972057  3853 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.035946  1574 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:47.999753  3907 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:48.003842  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.039654  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140314us
I20260709 13:01:48.039860  3909 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:48.041991  3957 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:48.023725  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:48.064416  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.045610  4019 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:48.037004  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:48.061136  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:48.066141  1721 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.045857  3837 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.003191  4031 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.064142  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:48.076035  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:48.077453  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:48.076112  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:48.059540  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.067390  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:48.076097  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:48.082350  1576 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.003680  3871 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.077062  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.080639  1646 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:48.085484  1724 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:48.082883  1645 raft_consensus.cc:2376] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3. Re-sending same reply.
I20260709 13:01:48.088485  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:48.088683  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.064740  3974 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.097416  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.067283  3957 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.113759  1569 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:48.111961  1642 raft_consensus.cc:2368] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5.
I20260709 13:01:48.081095  3853 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.042752  3739 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.071712  3669 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:48.139001  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.080276  3973 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:48.164636  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 65805.5us
I20260709 13:01:48.113775  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:48.167407  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78837.9us
I20260709 13:01:48.079210  2375 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:48.084733  4019 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:48.185792  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1041218380000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 13:01:48.092113  3782 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.164149  3918 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:01:48.189708  3853 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.184808  3861 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:01:48.106415  3871 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.171703  3957 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.206054  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:48.209380  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.209816  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:48.185519  3973 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.163367  3974 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:48.197183  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:48.199839  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 86201.9us
I20260709 13:01:48.283568  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:48.263120  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88028us
I20260709 13:01:48.205009  3971 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.263343  4049 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:48.199332  3890 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.226501  1720 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.241389  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.310715  1642 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:48.198690  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55254
I20260709 13:01:48.311910  1723 raft_consensus.cc:606] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:01:48.312319  1723 raft_consensus.cc:614] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:48.206172  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.237285  1648 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:48.210767  4039 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.165s
I20260709 13:01:48.207165   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:48.295686  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.250118  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:48.313776  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73875us
I20260709 13:01:48.292165  1576 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:48.384130  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:48.246263  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:48.196491  4021 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.388659  1720 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:48.393116  1569 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:48.357986  3688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.015s	sys 0.033s
I20260709 13:01:48.398473  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:48.398656  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 103141us
I20260709 13:01:48.409711  1643 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:48.196333  3972 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.300441  4049 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:48.396386  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 190387us
I20260709 13:01:48.442713  1721 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:48.315690  4014 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:48.434330  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.258702  3658 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:48.453217  1724 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:01:48.452035  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 62689.7us
I20260709 13:01:48.360657  3923 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:48.406932  3688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.015s	sys 0.033s
I20260709 13:01:48.454424  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:48.397502  3992 raft_consensus.cc:2764] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:48.404590  3971 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:48.431383  3837 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:48.451942  4049 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.431835  3918 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.431178  2376 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:48.520586  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74342.3us
I20260709 13:01:48.521903  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:48.405254  3890 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.530917  3890 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.531152  3658 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.531490  3890 raft_consensus.cc:476] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 13:01:48.532040  3890 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:48.501513  3970 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:48.528599  3853 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.528972  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:48.532254  3941 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.533376  3973 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.534664  1570 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:01:48.527449  4044 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:01:48.536330  1474 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 5 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.528606  3739 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.545531  1573 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:48.500964  3971 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.500886  3837 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.560245  1474 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.560117  3971 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:48.431934  4021 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.549391  3974 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:48.568193  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:48.569175  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:48.563134  3837 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:48.563318  4013 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.574335  1474 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:48.579308  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.500s (SENT)
I20260709 13:01:48.565905  3941 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.549329  3890 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:48.581852  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:48.584296  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 203) took 18692 ms (client timeout 15000 ms). Trace:
W20260709 13:01:48.586959  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.599860  1474 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 3 to 4, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 4 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:48.601744  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 79969us
I20260709 13:01:48.596609  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:48.581336  4052 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:48.618152  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 355013us
W20260709 13:01:48.626349  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.661s (SENT)
I20260709 13:01:48.627064  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:48.632632  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.717s (SENT)
I20260709 13:01:48.633140  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:48.642876  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:48.595160  3871 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.644552  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:48.650851  1723 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:48.651207  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:48.647513  1724 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:01:48.589993  1471 rpcz_store.cc:269] 0709 13:01:29.891783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.892797 (+  1014us) service_pool.cc:224] Handling call
0709 13:01:48.584133 (+18691336us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:46.556147 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:46.558154 (+  2007us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:46.558253 (+    99us) write_op.cc:435] Acquired schema lock
0709 13:01:46.558283 (+    30us) tablet.cc:662] Decoding operations
0709 13:01:46.744304 (+186021us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:46.751285 (+  6981us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:46.752392 (+  1107us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:01:46.771609 (+ 19217us) tablet.cc:701] Row locks acquired
0709 13:01:46.771673 (+    64us) write_op.cc:260] PREPARE: finished
0709 13:01:46.772092 (+   419us) write_op.cc:270] Start()
0709 13:01:46.772284 (+   192us) write_op.cc:276] Timestamp: P: 1783602106771999 usec, L: 0
0709 13:01:46.772304 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:01:46.884871 (+112567us) log.cc:844] Serialized 6017 byte log entry
0709 13:01:47.342805 (+457934us) op_driver.cc:464] REPLICATION: finished
0709 13:01:47.772050 (+429245us) write_op.cc:301] APPLY: starting
0709 13:01:47.772294 (+   244us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:47.772709 (+   415us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:47.772843 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:47.775055 (+  2212us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:47.775536 (+   481us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:01:47.775588 (+    52us) write_op.cc:312] APPLY: finished
0709 13:01:47.776059 (+   471us) log.cc:844] Serialized 107 byte log entry
0709 13:01:47.892158 (+116099us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:47.943903 (+ 51745us) write_op.cc:454] Released schema lock
0709 13:01:47.981028 (+ 37125us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:47.981695 (+   667us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:47.981731 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":832712,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":415786,"apply.run_cpu_time_us":6290,"apply.run_wall_time_us":210115,"num_ops":15,"prepare.queue_time_us":89555,"prepare.run_cpu_time_us":24789,"prepare.run_wall_time_us":385376,"raft.queue_time_us":143562,"raft.run_cpu_time_us":43897,"raft.run_wall_time_us":300252,"replication_time_us":553162,"thread_start_us":326679,"threads_started":4,"wal-append.queue_time_us":132500}]]}
I20260709 13:01:48.649044  1474 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.630438  3923 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:48.658068  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 61628.7us
I20260709 13:01:48.640487  3974 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:48.641891  3957 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.648437  3941 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:48.662339  1474 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 3 to 5, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 5 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:48.667462  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 233228us
I20260709 13:01:48.668756  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.653426  3688 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.655593  3972 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.684772  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.686023  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:48.686941  1724 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:48.689730   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:48.674958  4052 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:48.670419  3507 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:48.674233  4046 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.684s	user 0.016s	sys 0.000s
I20260709 13:01:48.675944  3970 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.694459  1474 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:48.720909  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:48.722968  4064 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:48.723729  4064 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:48.704723  4050 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.145s	user 0.000s	sys 0.002s
I20260709 13:01:48.743140  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:48.745332  1643 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:48.746364  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.747080  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.748070  1646 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:48.749724  1648 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:48.701532  3973 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:48.713112  1474 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a reported cstate change: config changed from index 2 to 3, term changed from 1 to 4, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to <none>, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:48.758876  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.759572  1643 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:48.759821  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 82401us
I20260709 13:01:48.760937  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:48.761819  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:48.671787  3688 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:48.762425  1643 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.786058  1642 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:01:48.708297  3739 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:48.752370  3871 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.794322  1642 pending_rounds.cc:85] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 1
W20260709 13:01:48.786440  4054 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.217s	user 0.000s	sys 0.002s
I20260709 13:01:48.808720  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:48.809681  1642 pending_rounds.cc:107] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:01:48.828773  1642 raft_consensus.cc:2889] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.790575  4052 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.707558  4046 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.785759  3507 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:48.789268  3658 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:48.804975  1474 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 3 to 4, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:48.803381  4021 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.868726  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:48.803239  3973 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:48.827011  3976 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.793s	user 0.006s	sys 0.008s
I20260709 13:01:48.882565  1643 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:48.463644  4019 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.890162  1575 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:48.891217  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:01:48.893175  4058 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 13:01:48.894287  4061 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:01:48.898505  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:48.899278  4049 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.901024  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:48.903062  3688 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.911085  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1743) took 1069 ms. Trace:
I20260709 13:01:48.872850  4052 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.917999  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:48.907981  1723 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:48.918656  1721 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:49.038687  1718 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:48.919386  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:48.919771  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.920459  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.920538  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:48.921157  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:49.040313  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:01:49.040781  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 119733us
I20260709 13:01:49.040992  1642 raft_consensus.cc:606] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:48.922058  3992 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:48.922250  4031 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.003s
I20260709 13:01:48.922883  3782 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.922788  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:48.924968  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:48.926394  4056 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:01:48.926807  3976 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:48.885844  3890 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:48.928980  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:48.929183  3999 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.929989  3853 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.827355  3871 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.877357  4064 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:48.933854  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:49.095152  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 161397us
I20260709 13:01:49.096068  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:48.934217  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:48.935199  4040 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:48.871059  3739 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:48.936012  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:48.936383  4055 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:01:48.922673  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88335.4us
I20260709 13:01:48.941112  2376 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:48.941342  4057 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.237s	user 0.000s	sys 0.002s
I20260709 13:01:48.941738  3996 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:49.116555  3996 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.117226  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1751) took 1251 ms. Trace:
I20260709 13:01:49.118207  3739 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.119302  3996 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:48.942999  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with delete_type TABLET_DATA_TOMBSTONED (TS 4f737473248e4c059fa74af3b8e1f77e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44738
I20260709 13:01:48.945107  4046 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:48.943174  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.952085  3962 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:48.956626  3507 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.957232  3973 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:48.961515  2377 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:48.961890  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:48.962570  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:48.963537  4013 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:48.966754  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:48.972906  4060 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:48.976739  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76030.1us
I20260709 13:01:48.977106  4063 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:49.169196  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.908s (SENT)
I20260709 13:01:49.169392  4063 consensus_queue.cc:1243] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:48.984009  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:48.986296  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:49.169744  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:48.988744  3972 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.011449  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.013671  1474 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:49.028473  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.036846  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1749) took 1244 ms. Trace:
I20260709 13:01:49.037343  1573 rpcz_store.cc:276] 0709 13:01:47.841895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.842051 (+   156us) service_pool.cc:224] Handling call
0709 13:01:48.910859 (+1068808us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54000c2954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:48.910993 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389842688}
I20260709 13:01:49.048565  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.051613  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.055117  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:49.056198  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.049883  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:49.068390  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.065106  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:48.908032  3971 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:49.054893  4018 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:01:49.060606  4068 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:49.069950  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.061241  3992 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:49.070564  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1169) took 1060 ms. Trace:
I20260709 13:01:49.144449  2376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 291) took 1010 ms. Trace:
I20260709 13:01:49.203156  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.073429  3976 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.078787  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1754) took 1005 ms. Trace:
I20260709 13:01:49.064180  1717 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:49.080680  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.207160  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.208092  1721 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:49.081841  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.046352  4014 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.091473  3853 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:49.091682  3999 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.110309  1649 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 3: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:49.173539  3658 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:49.109331  1722 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:49.117762  1567 rpcz_store.cc:276] 0709 13:01:47.866243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.024772 (+158529us) service_pool.cc:224] Handling call
0709 13:01:48.025260 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.025763 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.025794 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.026349 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.491291 (+464942us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:48.541394 (+ 50103us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.541796 (+   402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.541907 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.116967 (+575060us) raft_consensus.cc:1685] finished
0709 13:01:49.116987 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.117157 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49841,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":20195,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":17404,"thread_start_us":435790,"threads_started":1,"wal-append.queue_time_us":554877}
I20260709 13:01:48.948611  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.109491  4030 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:49.067472  3782 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.109941  4021 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.109786  4040 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:49.169783  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1753) took 1097 ms. Trace:
W20260709 13:01:49.172495  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 76538.6us
I20260709 13:01:49.172648  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:49.173946  1571 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:49.174865  1643 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:49.176251  1725 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:49.177802  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.208123  4046 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:49.189929  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.190080  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.191335  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.245283  1718 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:49.174819  1716 rpcz_store.cc:276] 0709 13:01:47.785306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.785436 (+   130us) service_pool.cc:224] Handling call
0709 13:01:47.785970 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.942602 (+156632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.942639 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:47.943201 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.489751 (+546550us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:48.489829 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.489951 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.029915 (+539964us) raft_consensus.cc:1685] finished
0709 13:01:49.029938 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.030112 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156115,"raft.queue_time_us":159800,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":31085,"thread_start_us":489302,"threads_started":1,"wal-append.queue_time_us":606668}
I20260709 13:01:49.142202  3688 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:49.176812  1642 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:49.248867  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.249282  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.249897  1647 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:49.250396  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.250865  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.251658  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.071959  2379 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:01:49.199308  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.253720  3971 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:49.196463  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 146758us
I20260709 13:01:49.208371  4068 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.203034  2376 rpcz_store.cc:276] 0709 13:01:48.115239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.177252 (+ 62013us) service_pool.cc:224] Handling call
0709 13:01:49.121896 (+944644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:49.113092  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.256538  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.256942  1570 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:49.221045  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:49.222862  1719 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:49.258457  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.141417  3507 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.259039  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.259603  1719 raft_consensus.cc:1813] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Handling vote request from an unknown peer 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:49.260215  1719 raft_consensus.cc:2368] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:01:49.260972  3507 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:49.197420  1568 rpcz_store.cc:276] 0709 13:01:48.010182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.040433 (+ 30251us) service_pool.cc:224] Handling call
0709 13:01:48.040892 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.041328 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.041357 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.041856 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.458833 (+416977us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:48.458897 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.459004 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.070273 (+611269us) raft_consensus.cc:1685] finished
0709 13:01:49.070296 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.070458 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29710,"raft.queue_time_us":23511,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":11703,"thread_start_us":416474,"threads_started":1,"wal-append.queue_time_us":628820}
I20260709 13:01:49.216993  4014 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.263365  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.223541  3739 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.264485  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.265625  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:49.231325  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.266157  1568 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:49.266633  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:49.232808  3853 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:49.234540  1572 rpcz_store.cc:276] 0709 13:01:48.073044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.086557 (+ 13513us) service_pool.cc:224] Handling call
0709 13:01:48.086981 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.175855 (+ 88874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.175900 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.185263 (+  9363us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.572218 (+386955us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:48.659709 (+ 87491us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.659975 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.660065 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.169526 (+509461us) raft_consensus.cc:1685] finished
0709 13:01:49.169551 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.169717 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":343584,"prepare.queue_time_us":54921,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":1819,"raft.queue_time_us":111158,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":3468,"thread_start_us":373425,"threads_started":1,"wal-append.queue_time_us":467019}
I20260709 13:01:49.267795  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.267951  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.268160  3992 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:49.268795  3992 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:49.269418  4030 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:49.269294  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.246940  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.270979  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:49.193921  3972 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.271692  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.173689  4060 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.272503  1573 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 2.
I20260709 13:01:49.273209  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.273977  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:49.274587  1573 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.275609  1569 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:49.259541  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.252331  1716 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:49.277385  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.280961  4040 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:49.282359  1717 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:49.282642  4040 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.264751  4014 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:49.284795  4013 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.266722  1575 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:49.285754  1724 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:49.270236  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:49.207360  1574 rpcz_store.cc:276] 0709 13:01:48.074002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.074147 (+   145us) service_pool.cc:224] Handling call
0709 13:01:48.074769 (+   622us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:48.091884 (+ 17115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.091919 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:48.117133 (+ 25214us) log.cc:844] Serialized 78 byte log entry
0709 13:01:48.598503 (+481370us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:48.640159 (+ 41656us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.893076 (+252917us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b1800364cc0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:48.901097 (+  8021us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b1800364cc0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:01:48.901318 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.901633 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.952312 (+ 50679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.952424 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.078471 (+126047us) raft_consensus.cc:1685] finished
0709 13:01:49.078496 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.078710 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55054,"prepare.queue_time_us":369762,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":2032,"raft.queue_time_us":115991,"raft.run_cpu_time_us":569,"raft.run_wall_time_us":25492,"spinlock_wait_cycles":10716160,"thread_start_us":397025,"threads_started":1,"wal-append.queue_time_us":429914}
I20260709 13:01:49.288256  1718 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:49.208822  3837 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.000s
I20260709 13:01:49.283789  1717 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:49.271904  1646 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:49.257247  1720 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:49.252264  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.272410  1567 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:49.248989  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:49.359987  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.788s (SENT)
I20260709 13:01:49.360709  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:49.296684  1716 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:49.270532  4030 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.276420  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.362234  1723 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:49.266007  3782 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.215263  1474 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 5, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 5 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.183552  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.260982  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:49.367791  1719 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:49.367875  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.356743  1724 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:49.350996  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.370482  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:49.257630  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.349996  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.369012  1647 raft_consensus.cc:1813] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Handling vote request from an unknown peer 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:49.374126  1647 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:49.348809  1722 raft_consensus.cc:2368] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:49.381388  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:49.383040  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 135220us
I20260709 13:01:49.376657  4063 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:49.384510  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.387760  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.256429  4068 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.389396  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:49.388295  1650 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:49.381394  1569 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:49.390621  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.388403  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:49.391686  1575 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:49.392747  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:49.392793  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.247651  4052 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.393780  1575 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:49.394892  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.391784  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:49.389010  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.378973  3837 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.396977  1574 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:49.397230  4049 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.397946  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.398785  1574 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:49.398936  3837 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.399608  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.400641  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:49.401248  1642 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.402691  3999 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.354398  4046 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:49.404368  4046 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:49.405117  3688 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.405706  3688 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:49.404377  1651 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:49.409775  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.411021  1651 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:49.413932  1651 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:49.415407  1645 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:49.413254  1574 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:49.419351  1647 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:49.400763  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:49.429800  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.430879  1567 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:49.274958  4060 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.436133  1651 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 3. Current term is 4. Ops: [1.3-3.4]
I20260709 13:01:49.430054  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.385967  3871 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.386070  3962 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:49.247404  4019 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.183s
I20260709 13:01:49.269321  3992 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.536617  1568 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
W20260709 13:01:49.565330  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.538154  1573 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
W20260709 13:01:49.494763  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 106470us
I20260709 13:01:49.548373  1718 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:01:49.489890  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 120787us
I20260709 13:01:49.544988  1567 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:01:49.587932  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 366994us
I20260709 13:01:49.516335  1576 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.591353  1571 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:49.444530  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.593371  1570 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:49.597779  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:49.601217  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:49.664039  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62922.2us
I20260709 13:01:49.665336  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:49.448606  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.791s (SENT) [suppressed 42 similar messages]
I20260709 13:01:49.609512  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:49.627703  1573 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:49.636289  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.678900  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81238.8us
I20260709 13:01:49.680078  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:49.637204  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:01:49.513554  3837 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:49.520198  3971 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.519625  4075 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:01:49.443833  4049 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:49.643693  1567 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:49.585899  3972 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.537176  4080 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:49.704444  4049 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.551792  3658 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:01:49.682876  1719 raft_consensus.cc:606] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:49.705946  1719 raft_consensus.cc:614] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:49.686058  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:49.668358  1522 auto_rebalancer.cc:893] Removing replace marker failed: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:49.707437  4049 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.682698  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:49.587471  3962 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:49.682757  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 93716.9us
I20260709 13:01:49.708853  3782 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:49.709335  3782 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.705073  4080 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.710160  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:49.707772  1522 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:01:49.709759  3782 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:49.711354  1522 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:01:49.709789  4068 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:01:49.709811  3962 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:49.683383  3996 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:49.725251  1576 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:49.774343  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.678813  4041 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:49.696228  4078 tablet_replica.cc:333] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:01:49.701052  3971 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:49.781560  3999 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:49.710472  4080 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:49.686184  3837 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.690469  3890 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:49.690694  4031 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:49.697448  4046 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:49.795113  1642 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
W20260709 13:01:49.806162  4083 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:01:49.679029  4066 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.151s
W20260709 13:01:49.772778  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 235825us
I20260709 13:01:49.857604  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:49.807729  3999 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.858832  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.711068  4049 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:01:49.811345  3837 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:49.813663  4031 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.813560  4046 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.813972  3890 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.845424  3861 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.856200  4076 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:01:49.889825  1648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.011s	sys 0.018s
I20260709 13:01:49.871690  3999 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:49.890290  1648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.011s	sys 0.018s
W20260709 13:01:49.889878  4036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.000s	sys 0.017s
I20260709 13:01:49.877033  4031 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.882979  4046 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:49.905135  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 195908us
W20260709 13:01:49.894011  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 183942us
I20260709 13:01:49.886678  3890 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.886512  3861 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:49.925653  4036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.001s	sys 0.018s
I20260709 13:01:49.932806  3507 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:49.940487  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77849.5us
I20260709 13:01:49.943306  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:49.961890  1573 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:49.923729  4060 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:49.953038  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:49.890360  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 441829us
W20260709 13:01:49.889415  2377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.009s	sys 0.009s
W20260709 13:01:49.994194  2377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.009s	sys 0.013s
W20260709 13:01:49.889931  4018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.010s	sys 0.007s
I20260709 13:01:49.935340  3962 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:50.077751  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1765) took 1373 ms. Trace:
I20260709 13:01:50.098994  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:50.099290  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 163470us
I20260709 13:01:49.997150  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:50.015641  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:01:50.078102  1571 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:50.060817  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 178771us
W20260709 13:01:50.008553  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.098886  1569 rpcz_store.cc:276] 0709 13:01:48.704603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.898230 (+193627us) service_pool.cc:224] Handling call
0709 13:01:50.077425 (+1179195us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54000276d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:50.077665 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564842368}
W20260709 13:01:50.104496  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 168432us
W20260709 13:01:50.018716  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 308605us
I20260709 13:01:50.108788  1567 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:49.996271  4018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.015s	sys 0.008s
I20260709 13:01:49.941299  3992 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.117901  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.119166  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:50.120571  1573 pending_rounds.cc:85] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
W20260709 13:01:50.075292  4056 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:50.122516  1573 pending_rounds.cc:107] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 4.5
W20260709 13:01:50.002890  4086 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.300s	user 0.001s	sys 0.001s
I20260709 13:01:50.119629  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:50.132889  1573 raft_consensus.cc:2889] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:50.102846  3962 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.116607  4066 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:50.135370  3962 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:50.136478  4021 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:50.138273  4021 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.138373  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:50.138893  4092 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.139230  2379 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:50.139520  4092 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:50.141280  4090 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:01:50.141587  4014 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:50.141971  4014 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:50.142825  4094 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:01:50.142891  4014 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.144311  4014 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:50.145450  4040 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.862s	user 0.014s	sys 0.000s
I20260709 13:01:50.149638  1575 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:01:50.150944  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 198) took 1222 ms. Trace:
I20260709 13:01:50.151280  1575 rpcz_store.cc:276] 0709 13:01:48.928970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.394601 (+465631us) service_pool.cc:224] Handling call
0709 13:01:50.150751 (+756150us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b540027cc54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:50.150887 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733862784}
I20260709 13:01:50.145954  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1760) took 1449 ms. Trace:
I20260709 13:01:50.118197  4068 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:01:50.152264  1574 rpcz_store.cc:276] 0709 13:01:48.696252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.434994 (+738742us) service_pool.cc:224] Handling call
0709 13:01:49.435305 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.435723 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.435767 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.489510 (+ 53743us) log.cc:844] Serialized 26 byte log entry
0709 13:01:49.697919 (+208409us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:49.697960 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.698029 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.145685 (+447656us) raft_consensus.cc:1685] finished
0709 13:01:50.145711 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.145881 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8302,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":40726,"thread_start_us":184030,"threads_started":1,"wal-append.queue_time_us":344297}
I20260709 13:01:50.154632  4092 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:49.961324  3507 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:50.146229  3871 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:50.162010  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 163844us
I20260709 13:01:50.164124  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:50.187187  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69383.1us
I20260709 13:01:50.187021  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:50.188865  2379 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:50.187875  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:50.202884  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:50.215771  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.143s (SENT)
I20260709 13:01:50.216516  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.223951  1539 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:50.117072  4095 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 13:01:50.237689  3971 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.237947  3853 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:50.238461  3853 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:50.239327  3507 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.240499  4068 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:50.240902  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:50.241214  3507 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.242057  4101 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:50.242576  4101 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.242940  4101 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:50.214833  3996 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.244590  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.244915  3996 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.165277  3871 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.246417  3871 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.199501  4066 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.236883  2379 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:50.138839  4021 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.260281  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 114866us
W20260709 13:01:50.258831  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 56092.8us
I20260709 13:01:50.190594  4036 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.263852  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:50.160665  4089 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 13:01:50.241381  4100 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.277580  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.274044  1649 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:50.239667  3853 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.279424  4099 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:01:50.279587  3853 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:50.285238  4097 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:01:50.285542  1642 raft_consensus.cc:606] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:50.286077  1642 raft_consensus.cc:614] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:50.190989  4040 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
W20260709 13:01:50.274806  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 49046.4us
I20260709 13:01:50.288125  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.298081  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1188) took 1308 ms. Trace:
I20260709 13:01:50.298699  1571 rpcz_store.cc:276] 0709 13:01:48.989932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.660946 (+671014us) service_pool.cc:224] Handling call
0709 13:01:50.077716 (+416770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.101137 (+ 23421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.297788 (+196651us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54000276d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:50.297990 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874258176}
I20260709 13:01:50.240963  4068 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.260833  3984 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.265048  4060 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.332712  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.319623  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.330377  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.272367  3996 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:50.347165  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.643s (SENT)
I20260709 13:01:50.347891  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:50.346505  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1139) took 1573 ms. Trace:
I20260709 13:01:50.350287  1648 rpcz_store.cc:276] 0709 13:01:48.754804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.754971 (+   167us) service_pool.cc:224] Handling call
0709 13:01:48.755424 (+   453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:49.098448 (+343024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.098495 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:49.190220 (+ 91725us) log.cc:844] Serialized 52 byte log entry
0709 13:01:49.923835 (+733615us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:49.931310 (+  7475us) log.cc:844] Serialized 14 byte log entry
0709 13:01:49.938811 (+  7501us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.110611 (+171800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.148298 (+ 37687us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b180038caa0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:50.328121 (+179823us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400133b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:50.328181 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.328225 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:50.328253 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.328439 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195855,"prepare.queue_time_us":116821,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":112,"raft.queue_time_us":178215,"raft.run_cpu_time_us":753,"raft.run_wall_time_us":74117,"spinlock_wait_cycles":840536064,"thread_start_us":681698,"threads_started":1,"wal-append.queue_time_us":825075}
W20260709 13:01:50.356777  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.359035  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1758) took 1416 ms. Trace:
I20260709 13:01:50.359967  1649 rpcz_store.cc:276] 0709 13:01:48.942501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.396979 (+454478us) service_pool.cc:224] Handling call
0709 13:01:49.397308 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.279048 (+881740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.358559 (+ 79511us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54001bbcd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:50.358920 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2015966592}
I20260709 13:01:50.372085  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:50.323596  4068 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:50.378648  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46033.7us
I20260709 13:01:50.380946  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.329622  4079 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:50.385532  1721 raft_consensus.cc:2410] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:50.375648  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.326973  4095 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:50.265086  4066 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.345701  4060 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:50.346443  3861 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.345978  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 387) took 1124 ms. Trace:
I20260709 13:01:50.322678  3837 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.419490  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 156327us
W20260709 13:01:50.401022  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.725s (SENT)
I20260709 13:01:50.414813  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.420570  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:50.354326  4108 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.385895  4079 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.441357  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:50.440549  3976 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.449453  1718 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:50.452579  4018 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:50.386065  3972 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:01:50.459940  3837 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.440378  4079 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:50.394179  4095 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:50.499140  4018 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.469753  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:50.506417  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54715us
I20260709 13:01:50.507191  4098 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:50.508119  4102 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:50.397451  4110 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:50.520032  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 275366us
W20260709 13:01:50.520437  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79225.8us
I20260709 13:01:50.413571  4036 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:50.528779  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 240751us
I20260709 13:01:50.539209  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:50.540647  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:50.541855  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.541802  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 283166us
I20260709 13:01:50.442658  4040 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:50.544665  1570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.015s	sys 0.034s
I20260709 13:01:50.544757  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.545833  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:50.546564  1570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.016s	sys 0.035s
I20260709 13:01:50.546711  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:50.550066  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.566486  1574 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:50.566994  1574 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:50.569548  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:50.567804  1574 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.571645  1574 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.590471  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1189) took 1600 ms. Trace:
I20260709 13:01:50.540540  3837 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:50.598338  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:50.548557  4049 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.528774  4098 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.548470  4089 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.613329  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:50.602195  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52250.3us
I20260709 13:01:50.631723  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.618392  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:50.538074  3973 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:01:50.555311  4018 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:50.629688  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 80669.9us
I20260709 13:01:50.632954  1571 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:50.595320  4040 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.595008  1570 rpcz_store.cc:276] 0709 13:01:48.990229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.661475 (+671246us) service_pool.cc:224] Handling call
0709 13:01:49.685462 (+ 23987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.685670 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:49.955561 (+269891us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.589372 (+633811us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.589443 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:50.589507 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.590204 (+   697us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.590388 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":477323,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"thread_start_us":875566,"threads_started":3}
I20260709 13:01:50.606695  3837 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.639674  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.643829  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.652583  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:50.653537  1568 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:01:50.663147  1567 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.002s
I20260709 13:01:50.663628  1567 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:50.638384  3861 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:50.638881  4021 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.641970  3837 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:50.645993  4095 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.415913  2373 rpcz_store.cc:276] 0709 13:01:49.203808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.206130 (+  2322us) service_pool.cc:224] Handling call
0709 13:01:49.256784 (+ 50654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.257265 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.257305 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.257822 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:01:49.577667 (+319845us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:49.585928 (+  8261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.599855 (+ 13927us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.275446 (+675591us) raft_consensus.cc:1685] finished
0709 13:01:50.287371 (+ 11925us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.327555 (+ 40184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3472,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":306053,"threads_started":1,"wal-append.queue_time_us":331034}
W20260709 13:01:50.680989  4109 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:01:50.681870  4116 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.682845  4116 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.684496  4123 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.685173  4123 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.687803  4121 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:50.692441  4113 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.248s
I20260709 13:01:50.693403  4100 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:50.696612  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
W20260709 13:01:50.703588  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.668385  3688 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.262s	user 0.011s	sys 0.006s
W20260709 13:01:50.710428  3962 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.004s
I20260709 13:01:50.710690  4126 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.712217  4135 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.713125  4102 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:50.713595  4013 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.428s	user 0.009s	sys 0.007s
W20260709 13:01:50.674367  3890 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.741s	user 0.015s	sys 0.005s
I20260709 13:01:50.639470  4018 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.687403  4123 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:50.714375  4014 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.560s	user 0.017s	sys 0.002s
W20260709 13:01:50.753422  4115 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:01:50.705894  4078 raft_consensus.cc:2243] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:50.754412  3890 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.755446  4119 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:01:50.756835  3890 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.760959  4066 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.761340  4116 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.715415  3871 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.763250  4106 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.446s	user 0.001s	sys 0.001s
I20260709 13:01:50.763325  4135 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.719559  3688 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:50.766738  1719 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:50.696516  3861 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:50.764045  3871 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:50.775975  3861 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:50.779227  4111 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:01:50.779426  3837 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.752934  4013 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:50.741318  4088 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.768172  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:50.773913  3999 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.843s	user 0.006s	sys 0.006s
I20260709 13:01:50.774158  3972 raft_consensus.cc:993] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: : Instructing follower da3b57321031416cba9db75450ef302a to start an election
I20260709 13:01:50.774657  4018 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:50.774792  3507 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.004s
I20260709 13:01:50.782004  4088 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.790616  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1193) took 1409 ms. Trace:
I20260709 13:01:50.796249  1572 rpcz_store.cc:276] 0709 13:01:49.381733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.680931 (+299198us) service_pool.cc:224] Handling call
0709 13:01:49.682246 (+  1315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.684660 (+  2414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.684689 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.712833 (+ 28144us) log.cc:844] Serialized 26 byte log entry
0709 13:01:49.893297 (+180464us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:49.893359 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.893472 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.762385 (+868913us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:50.790275 (+ 27890us) raft_consensus.cc:1685] finished
0709 13:01:50.790304 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.790541 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1926,"raft.queue_time_us":2494,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":77319,"spinlock_wait_cycles":195032832,"thread_start_us":48264,"threads_started":1,"wal-append.queue_time_us":462370}
I20260709 13:01:50.727378  3962 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.714116  4060 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.787119  4031 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.826s	user 0.006s	sys 0.006s
I20260709 13:01:50.816664  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.873756  4031 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.810947  4088 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:50.878296  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.803418  4080 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.956s	user 0.006s	sys 0.006s
W20260709 13:01:50.879890  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 248301us
I20260709 13:01:50.880903  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:50.881429  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:50.881794  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:50.885500  4126 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:50.853922  3782 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.142s	user 0.011s	sys 0.002s
I20260709 13:01:50.887416  1721 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:01:50.888260  1723 raft_consensus.cc:606] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:50.888435  1717 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:50.888679  1723 raft_consensus.cc:614] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:50.809665  3972 raft_consensus.cc:1081] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Signalling peer da3b57321031416cba9db75450ef302a to start an election
I20260709 13:01:50.885546  1718 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:50.888988  1539 leader_election.cc:400] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:01:50.896441  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.912815  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:50.889451  4119 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.117s	user 0.003s	sys 0.001s
I20260709 13:01:50.879199  4078 raft_consensus.cc:2272] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:50.888908  4088 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.919147  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:50.879449  4108 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:50.795734  3999 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.916595  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.827575  4079 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:50.920172  4068 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.009s
I20260709 13:01:50.890702  4080 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.915328  4040 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.914883  4126 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.894767  3782 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:50.920742  4052 raft_consensus.cc:2764] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:01:50.935847  4046 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.989s	user 0.012s	sys 0.004s
W20260709 13:01:50.948182  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 312247us
I20260709 13:01:50.949954  1725 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:50.949761  1539 leader_election.cc:400] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:50.968187  1688 leader_election.cc:400] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5.
W20260709 13:01:50.953954  4101 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.711s	user 0.013s	sys 0.001s
I20260709 13:01:50.927718  4088 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:50.960865  4132 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:01:50.961961  4079 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:50.962414  4133 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:50.939584  4087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.018s	sys 0.010s
W20260709 13:01:50.952473  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 334956us
I20260709 13:01:50.968082  1539 leader_election.cc:403] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:50.953442  4120 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:51.146534  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:51.026677  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:51.112036  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:50.968935  1688 leader_election.cc:403] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:50.918036  4134 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:01:50.929350  4108 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.025161  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:51.491211  4079 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.574543  4079 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.190147  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 207) took 1070 ms. Trace:
I20260709 13:01:51.576498  1573 rpcz_store.cc:276] 0709 13:01:50.119875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.120016 (+   141us) service_pool.cc:224] Handling call
0709 13:01:50.120361 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:50.158158 (+ 37797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.158200 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:50.159147 (+   947us) log.cc:844] Serialized 52 byte log entry
0709 13:01:50.414808 (+255661us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:50.518651 (+103843us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.635184 (+116533us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.816832 (+181648us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b4800002dd8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:01:50.893764 (+ 76932us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b180044d6a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:01:51.189556 (+295792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.189709 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.189782 (+    73us) raft_consensus.cc:1685] finished
0709 13:01:51.189805 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.190051 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76768,"prepare.queue_time_us":80125,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":106,"raft.queue_time_us":221076,"raft.run_cpu_time_us":2785,"raft.run_wall_time_us":118729,"spinlock_wait_cycles":8719872,"thread_start_us":577036,"threads_started":3,"wal-append.queue_time_us":361469}
I20260709 13:01:51.036859  3837 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.155507  1651 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:50.970485  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:51.020591  1720 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:51.241125  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.063963  1717 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:01:51.068758  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.585686  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 636199us
W20260709 13:01:51.588410  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 350863us
I20260709 13:01:51.735435  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:51.218823  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64791.3us
I20260709 13:01:51.737056  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:51.732924  1572 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:51.737692  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1791) took 1038 ms. Trace:
W20260709 13:01:51.318681  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.738572  1717 rpcz_store.cc:276] 0709 13:01:50.699075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.740389 (+ 41314us) service_pool.cc:224] Handling call
0709 13:01:51.737626 (+997237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:51.739574  4133 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.116180  3688 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:51.304708  4128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.007s	sys 0.013s
I20260709 13:01:51.741652  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:51.574172  1570 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:50.939850  4098 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:51.738227  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:51.116263  4060 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.710899  4078 ts_tablet_manager.cc:1916] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:01:51.035909  4122 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.263s	user 0.007s	sys 0.000s
I20260709 13:01:51.030052  3739 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.115631  4146 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:01:50.946982  4142 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:51.775416  3973 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:51.037729  4095 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:51.121910  3782 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.778724  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:51.151613  4031 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.797219  1642 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:51.798543  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 830484us
I20260709 13:01:51.779343  4098 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5.
I20260709 13:01:51.694169  4064 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:51.477257  4088 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:51.758385  4128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.007s	sys 0.021s
I20260709 13:01:51.028942  3962 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:50.941642  4068 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.797191  4060 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.191951  3992 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:51.667992  2373 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c"
dest_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:41544
I20260709 13:01:51.749816  4141 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:51.609287  3861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.002s	sys 0.021s
I20260709 13:01:51.749575  4135 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:51.784107  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:51.786350  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:51.840694  1643 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:51.163247  4052 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.641024  4092 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:51.241312  3890 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.786759  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:51.844283  1648 raft_consensus.cc:2368] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:01:51.844413  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:50.973277  3972 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.785389  1647 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:51.578403  4079 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:51.783449  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:51.034920  3999 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:51.849346  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66047.3us
W20260709 13:01:51.786701  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1777) took 1455 ms (client timeout 1742 ms). Trace:
I20260709 13:01:51.240794  4126 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:51.241223  4100 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.800329  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:51.765836  4133 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.846915  3890 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.845279  4052 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:50.967175  4040 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.855000  3972 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.861423  4052 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:51.119437  4080 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:51.265147  4120 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.862636  4040 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:51.845597  4092 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.864926  4120 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:51.796770  3739 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.868671  4080 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:51.869336  4080 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:51.870165  3739 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.782974  4148 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:51.870432  4080 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.871503  3962 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:51.807416  4064 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:50.977681  4046 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.201979  4101 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.792265  4142 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:51.240427  4132 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:51.820385  4060 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:51.839351  4150 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.079s	user 0.004s	sys 0.000s
I20260709 13:01:51.845201  1643 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:51.828464  4141 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:51.830480  3861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.004s	sys 0.026s
I20260709 13:01:51.828948  4108 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.790271  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:51.854319  1720 rpcz_store.cc:269] 0709 13:01:50.325950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.327437 (+  1487us) service_pool.cc:224] Handling call
0709 13:01:51.780716 (+1453279us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b5400222c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:51.780875 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577341696}
I20260709 13:01:51.841800  4135 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.858234  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:51.862630  3972 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.802172  4149 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:51.867100  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 66890.3us
I20260709 13:01:51.795230  3973 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.900774  3973 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:51.904618  4149 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:51.907656  4149 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:51.909484  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51383us
I20260709 13:01:51.910602  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:51.910902  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:51.912093  4148 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:51.872756  4100 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.909559  4133 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:51.913836  1643 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:01:51.855203  3688 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:51.914777  4100 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.871980  3962 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:51.914662  4108 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:51.916075  3962 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.917167  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:51.913743  4041 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:51.922173  1650 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:01:51.918867  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:51.923390  1651 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:51.885155  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:51.920948  1569 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:51.919498  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:51.926481  1648 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:51.880848  4132 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:51.926587  1645 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [3.3-6.4]
I20260709 13:01:51.877571  4142 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:51.897887  4141 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:51.918138  2373 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:51.921486  4135 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:51.921597  4120 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.941727  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:51.944301  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1791) took 1347 ms (client timeout 1619 ms). Trace:
W20260709 13:01:51.945011  1569 rpcz_store.cc:269] 0709 13:01:50.596598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.598087 (+  1489us) service_pool.cc:224] Handling call
0709 13:01:51.944066 (+1345979us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54000c2e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:51.944219 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039879424}
I20260709 13:01:51.926651  3962 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:51.940886  2373 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:51.944226  4098 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:51.930630  4148 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:51.968767  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:01:51.975951  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 198030us
W20260709 13:01:51.977257  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 216083us
I20260709 13:01:51.980156  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:51.981266  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:51.977948  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 239804us
I20260709 13:01:51.983403  2377 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:01:51.951694  4087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.680s	user 0.027s	sys 0.028s
I20260709 13:01:51.947800  3890 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.996309  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:51.983508  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.005121  1539 leader_election.cc:400] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:01:52.005607  1539 leader_election.cc:403] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:51.952176  4060 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:51.971701  4092 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:52.038995  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55205.5us
I20260709 13:01:52.039413  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:51.947819  4120 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.041467  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:51.987077  3861 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:52.034323  4148 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:51.987195  4040 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:52.003644  4133 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:51.994652  4100 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.990078  4153 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.239s	user 0.003s	sys 0.001s
I20260709 13:01:52.051689  1539 leader_election.cc:400] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:01:52.084640  1539 leader_election.cc:403] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:52.052740  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.063633  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:52.064886  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:52.086609  1569 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.066011  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:52.046775  4155 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:01:52.069028  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:52.003942  4068 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.049582  3782 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:52.033814  3992 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:52.049146  3999 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.030885  4091 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.037330  3890 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:51.979650  4088 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
W20260709 13:01:52.101356  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 113786us
I20260709 13:01:52.107033  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:52.093365  1575 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:52.098090  2379 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:52.103382  1572 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:52.109519  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 132532us
W20260709 13:01:52.104809  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55022.9us
I20260709 13:01:52.061687  4060 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:52.067880  3739 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:52.073462  4148 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.123312  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.126865  1571 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 2.
I20260709 13:01:52.125108  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.128310  1575 raft_consensus.cc:2410] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:52.071455  4087 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:01:52.146281  2375 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:52.151470  1569 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:52.146652  3782 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:52.169577  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.170027  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1796) took 1191 ms. Trace:
I20260709 13:01:52.170430  1716 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:52.170497  1567 rpcz_store.cc:276] 0709 13:01:50.979094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.979339 (+   245us) service_pool.cc:224] Handling call
0709 13:01:50.979924 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.024808 (+ 44884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.024845 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.161285 (+136440us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.598258 (+436973us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:51.895786 (+297528us) log.cc:844] Serialized 14 byte log entry
0709 13:01:51.896152 (+   366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.896274 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.940605 (+ 44331us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:52.169654 (+229049us) raft_consensus.cc:1685] finished
0709 13:01:52.169777 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.169952 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372990,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":12699,"raft.queue_time_us":46178,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":23975,"spinlock_wait_cycles":9174912,"thread_start_us":608729,"threads_started":2,"wal-append.queue_time_us":479727}
I20260709 13:01:52.127152  4158 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.101485  3992 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.106743  4126 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:51.972270  4132 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.113530  4134 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:52.247870  1720 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:52.164207  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.164804  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.250248  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:52.166081  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.167292  4052 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:52.162410  4155 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:01:52.167531  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.172925  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.129312  4148 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.140657  4046 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.094452  4040 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.175853  4120 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:52.094916  4133 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.176795  4098 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.151762  4030 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:52.120872  4162 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:52.128059  3739 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:52.192998  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81581us
I20260709 13:01:52.099558  4100 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.206493  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.207448  1716 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:52.241484  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.919s (SENT)
I20260709 13:01:52.305922  1723 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:52.306455  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65052.9us
W20260709 13:01:52.256258  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 134443us
I20260709 13:01:52.307852  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:52.264423  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.264446  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.264792  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.311025  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.265407  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.266597  1718 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.263569  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.115136  3999 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.269296  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.313678  1721 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:52.313833  1724 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:52.159271  4068 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
I20260709 13:01:52.183907  4101 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.208283  4161 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:52.245220  3992 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.121367  4091 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.333164  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.288218  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.289122  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.335325  1569 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:52.287400  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 194469us
I20260709 13:01:52.320966  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:52.341441  4068 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:52.289659  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.341940  4068 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:52.342551  1647 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:52.344743  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.345400  1570 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.345891  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.335076  1568 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:52.346758  1567 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:52.347378  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.348263  1648 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:52.297443  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.352756  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.354526  1720 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:52.356524  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.357478  1717 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:52.338493  2377 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:52.351440  1645 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:52.363673  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.266816  4149 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:01:52.373963  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.319478  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.268018  4052 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.289829  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.300818  4098 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.300766  4064 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:52.317498  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.373963  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.365304  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:52.470299  1645 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:52.465833  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.400410  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:52.464886  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.465456  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:52.473546  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.473557  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.474385  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.472470  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:52.466187  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.300912  3739 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.476672  1716 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:52.464432  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.491014  1648 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:52.478480  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.480226  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.244869  4158 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.305090  4046 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:52.397794  4165 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 13:01:52.514190  1650 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:52.515743  1718 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:01:52.486683  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 113336us
I20260709 13:01:52.403257  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:52.497723  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.498123  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.321925  4080 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:52.500787  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.505440  4134 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:52.495672  1570 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:01:52.343257  4068 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.465078  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.512135  4040 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.494211  1724 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
W20260709 13:01:52.426684  3973 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 13:01:52.379279  4157 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:52.486251  4177 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:01:52.384735  4174 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.469174  3962 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.008s
I20260709 13:01:52.469821  4149 consensus_queue.cc:1243] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:52.399689  4161 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:52.536288  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 228536us
I20260709 13:01:52.366662  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:52.497437  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:52.514008  4064 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:52.519914  1649 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:52.524680  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:52.526160  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 415631us
I20260709 13:01:52.422247  4091 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:52.516273  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 207812us
I20260709 13:01:52.532280  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.534874  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.536975  1725 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:52.519430  1575 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:52.506246  1723 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:01:52.518461  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.378988  4101 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 13:01:52.539820  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:52.300679  4162 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:52.622061  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 83360.1us
I20260709 13:01:52.517156  4170 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:52.624413  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.507555  4176 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:52.521173  4134 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.523823  4087 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:52.626576  1644 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:01:52.622397  4142 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.680s	user 0.013s	sys 0.000s
I20260709 13:01:52.633510  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.668397  1723 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:52.623328  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:52.521061  4068 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:52.622028  2371 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:52.518808  4046 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.619647  4175 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:52.618801  2374 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:52.621433  4064 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:52.653000  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134818us
I20260709 13:01:52.665035  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:52.633671  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:52.680018  4060 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.034s	sys 0.074s
I20260709 13:01:52.691519  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:52.691864  4060 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.034s	sys 0.074s
I20260709 13:01:52.696108  4064 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:52.523044  4157 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.521591  3973 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.765046  4157 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:52.766891  3973 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:52.646159  2373 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.699s	user 0.013s	sys 0.000s
I20260709 13:01:52.768172  2373 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.769780  1646 raft_consensus.cc:2368] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:01:52.688457  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:52.672752  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.773377  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:52.774711  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.776230  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.533996  4080 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.780696  4080 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.622723  4040 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.781481  4040 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.686370  3999 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:52.737967  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46583.6us
I20260709 13:01:52.539801  4174 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:01:52.784214  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:52.785364  1575 raft_consensus.cc:2410] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:52.786784  1651 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:01:52.725435  4095 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.928s	user 0.013s	sys 0.001s
I20260709 13:01:52.684715  4142 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.747869  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:01:52.757728  4148 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.638911  4182 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:52.789172  1575 pending_rounds.cc:85] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 2
I20260709 13:01:52.791631  1643 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:52.791764  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.811939  1575 pending_rounds.cc:107] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:01:52.812363  1575 raft_consensus.cc:2889] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:52.713129  4176 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:52.818471  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:52.795763  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 301491us
I20260709 13:01:52.820483  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:52.661352  4130 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.661113  4134 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.799597  1723 raft_consensus.cc:606] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:01:52.804704  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.824707  1723 raft_consensus.cc:614] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:52.809003  1722 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:52.721531  4162 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.750418  4060 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:52.824309  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:52.802652  4188 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:01:52.828925  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:52.842406  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.809273  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:52.719666  4158 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.797019  4184 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:52.713580  4175 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.853812  1723 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:52.853927  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.799224  4171 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:52.646970  4161 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.670698  4087 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.664911  4170 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.859540  1723 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:52.839540  3890 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.006s
I20260709 13:01:52.848737  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:52.826524  4079 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.868315  3992 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.001s
I20260709 13:01:52.839593  4112 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:52.802335  3973 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:01:52.836900  3688 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:01:52.865367  4141 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.930s	user 0.013s	sys 0.001s
I20260709 13:01:52.876896  4098 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:52.877348  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:52.875499  1723 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:52.856678  3999 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.856715  4092 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:52.872892  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68528.6us
I20260709 13:01:52.903487  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:52.884055  4158 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.845898  4190 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:52.906435  4190 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.907225  4175 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.909457  4190 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.910346  4158 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.891418  3992 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.856374  4162 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:52.912905  4162 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.913837  3992 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.914733  4046 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.868799  4159 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:52.915670  4162 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.916093  4159 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.916620  4159 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:52.917608  1723 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:01:52.918814  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:01:52.919910  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.884255  1644 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:52.895833  4079 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.921850  1725 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:01:52.919392  1723 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:01:52.922518  4170 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.886610  4185 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:52.923506  4185 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.923990  4185 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:52.924254  2371 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:52.925629  2377 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:52.926189  4132 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.927320  4060 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.929576  3999 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:52.880800  4052 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:52.905565  4183 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 13:01:52.521130  4173 consensus_queue.cc:1175] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3b57321031416cba9db75450ef302a" responder_term: 7 status { last_received { term: 3 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:52.882951  4087 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:52.921698  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.905014  4092 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:52.933436  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.920006  1572 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:01:52.935326  1725 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:01:52.871093  4191 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.890861  3890 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
W20260709 13:01:52.962576  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.971340  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:52.980029  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 170896us
I20260709 13:01:52.866066  4184 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.941217  4174 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:52.951566  4098 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.988216  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:53.002658  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 154023us
I20260709 13:01:53.003695  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:53.004570  1572 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:52.993572  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:52.941256  4162 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:52.898646  3688 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.006204  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:53.009443  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:52.872848  4171 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.005905  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:53.010471  4092 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:53.016631  4157 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.017052  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:53.019182  1644 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:53.024554  4142 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.026297  4171 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:53.027115  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:53.027895  4148 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:53.028167  1573 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:53.030949  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:53.031912  1573 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:53.011623  3890 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:53.032743  3890 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.033214  3890 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:53.034214  3688 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.035460  4079 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:53.037660  4184 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:53.038761  4196 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:01:53.011241  4098 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:52.979435  4191 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:52.982448  3999 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:52.904004  4141 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.041931  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:53.044790  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:53.045192  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:53.065299  1571 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:53.073905  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:53.075078  1572 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:53.041849  4164 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.865s	user 0.007s	sys 0.002s
I20260709 13:01:53.072551  1573 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:53.019285  4157 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.008289  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:53.146817  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.202294  1688 connection.cc:256] RPC call timeout handler was delayed by -2.804954437000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 13:01:53.182057  1576 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:01:53.371104  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 467740us
W20260709 13:01:53.125887  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.374997  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.046607  4174 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:53.304540  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.083030  1574 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:01:53.375337  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:53.090672  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:53.382460  1575 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:53.384755  1572 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:53.385995  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:53.386946  1572 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:53.397385  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1783) took 1599 ms. Trace:
I20260709 13:01:53.399818  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:53.047470  4148 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:53.404806  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:53.414862  4132 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:53.422106  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1778) took 1638 ms (client timeout 1739 ms). Trace:
I20260709 13:01:53.416460  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:53.415097  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 406897us
I20260709 13:01:53.415540  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1789) took 1094 ms. Trace:
I20260709 13:01:53.426914  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:53.428931  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.430153  1722 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:53.416320  1572 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:53.397908  1642 rpcz_store.cc:276] 0709 13:01:51.798082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.801258 (+  3176us) service_pool.cc:224] Handling call
0709 13:01:51.801750 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:51.802165 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.802195 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:51.802758 (+   563us) log.cc:844] Serialized 52 byte log entry
0709 13:01:52.091909 (+289151us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:52.091954 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.092038 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.040980 (+948942us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:53.154372 (+113392us) raft_consensus.cc:1685] finished
0709 13:01:53.154397 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.397294 (+242897us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1667,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":1263,"spinlock_wait_cycles":7900416,"thread_start_us":271690,"threads_started":1,"wal-append.queue_time_us":362789}
I20260709 13:01:53.433658  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:53.234890  3972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.019s	sys 0.007s
W20260709 13:01:53.162053  4126 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.903s	user 0.011s	sys 0.004s
W20260709 13:01:53.422817  1646 rpcz_store.cc:269] 0709 13:01:51.783492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.783671 (+   179us) service_pool.cc:224] Handling call
0709 13:01:53.416739 (+1633068us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:53.422018 (+  5279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2137348864}
I20260709 13:01:53.176620  4079 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.202783  4182 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:53.424027  1651 rpcz_store.cc:276] 0709 13:01:52.321305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.321479 (+   174us) service_pool.cc:224] Handling call
0709 13:01:53.415233 (+1093754us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400306954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:53.415341 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675972352}
I20260709 13:01:53.439173  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 243) took 1266 ms. Trace:
W20260709 13:01:53.474548  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65006.6us
I20260709 13:01:53.439921  2377 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:53.347477  3973 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:53.087126  3861 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:53.474828  1721 rpcz_store.cc:276] 0709 13:01:52.172788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.813994 (+641206us) service_pool.cc:224] Handling call
0709 13:01:52.814253 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.899392 (+ 85139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.899423 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:52.899465 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:52.899510 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.438826 (+539316us) spinlock_profiling.cc:243] Waited 84.9 ms on lock 0x7b54001716d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:53.438900 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.439083 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195318656}
I20260709 13:01:53.486647  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:53.487401  1721 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:53.161300  4200 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.040196  4185 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.456035  4126 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:01:53.444933  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1792) took 1055 ms. Trace:
I20260709 13:01:53.444977  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.127295  3688 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:53.447188  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.418723  4157 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:01:53.127415  3999 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:53.515739  3890 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.528851  3999 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.541525  1572 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:53.465157  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65514.8us
I20260709 13:01:53.176632  4078 ts_tablet_manager.cc:1929] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:01:53.266986  3782 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:53.467926  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:53.441732  3972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.021s	sys 0.007s
I20260709 13:01:53.015828  4162 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:53.470162  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.470767  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.471752  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:53.487458  4161 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:01:53.570283  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.571410  1646 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:53.475752  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:53.572553  4060 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:53.478212  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.485126  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.380739  4159 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.492717  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.493149  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:53.178747  4064 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.505409  1722 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:53.154573  4098 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:53.501811  1650 rpcz_store.cc:276] 0709 13:01:52.389381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.389563 (+   182us) service_pool.cc:224] Handling call
0709 13:01:53.444736 (+1055173us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b54003fd754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:53.444855 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90736512}
I20260709 13:01:53.500429  4135 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.510708  4203 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:53.470019  4079 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.485728  3861 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:53.511723  4197 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:53.421649  4148 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.512398  1573 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:53.512678  3739 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.994s	user 0.009s	sys 0.005s
I20260709 13:01:53.505288  4200 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.506513  4068 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.513031  4182 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.513622  1567 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:01:53.473801  4132 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:53.516680  3688 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.492451  3973 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.492177  4204 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:53.519089  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.466069  4041 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:53.546972  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:53.562026  1721 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:01:53.562784  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 147419us
I20260709 13:01:53.567902  4158 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.574028  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.417590  4174 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:53.573168  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97580us
I20260709 13:01:53.596910  1646 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:01:53.848161  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1207) took 1211 ms. Trace:
I20260709 13:01:53.869197  3861 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.870204  1725 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:53.871182  3739 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.825899  4189 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:53.873457  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 326698us
I20260709 13:01:53.873905  4189 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:53.573124  4126 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.875082  4189 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.875401  4126 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.875671  1642 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:53.878595  4204 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:53.880358  3972 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:53.880950  3739 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.883315  4189 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:53.868515  1568 rpcz_store.cc:276] 0709 13:01:52.636830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.846667 (+1209837us) service_pool.cc:224] Handling call
0709 13:01:53.847277 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.847615 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.847657 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.847726 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:53.847782 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.847874 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.848061 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.874997  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:53.886315  4178 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:53.886955  4178 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:53.884181  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.887830  1642 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:01:53.884735  4204 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:53.888163  4178 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.889079  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.889168  4204 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.890079  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1153) took 1095 ms. Trace:
I20260709 13:01:53.890581  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:53.890635  1646 rpcz_store.cc:276] 0709 13:01:52.794631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.595801 (+801170us) service_pool.cc:224] Handling call
0709 13:01:53.596517 (+   716us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.886400 (+289883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.890006 (+  3606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.883121  4197 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:53.852916  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.893574  4068 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.893424  4197 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.894960  3972 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.897022  4174 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.791831  4040 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:53.902084  1721 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:53.624974  4060 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.904870  1720 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:53.906370  4079 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:53.628943  1651 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:01:53.906652  4132 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.907768  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1154) took 1112 ms. Trace:
I20260709 13:01:53.909971  2377 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:53.910392  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.908283  1651 rpcz_store.cc:276] 0709 13:01:52.794893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.627854 (+832961us) service_pool.cc:224] Handling call
0709 13:01:53.628556 (+   702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.907461 (+278905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.907700 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.911526  1647 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:53.569340  3782 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:53.914032  2377 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:53.915771  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1206) took 1279 ms. Trace:
I20260709 13:01:53.916617  3782 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:53.635622  4078 log.cc:1199] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 4f737473248e4c059fa74af3b8e1f77e: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-0-root/wals/b837a5ec9a0d4f2a8635dedf45d51aa2
I20260709 13:01:53.883602  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:53.926391  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:53.926667  4078 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44738 (request call id 73) took 5003 ms. Trace:
I20260709 13:01:53.919330  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:53.928068  4098 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:53.928799  2377 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:53.929118  4098 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:53.929957  4098 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:53.931471  4098 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:53.603947  4161 consensus_queue.cc:1243] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:53.927809  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:53.936228  2374 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:53.936676  4206 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:53.940332  2373 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:53.941130  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41544 (request call id 395) took 2361 ms. Trace:
I20260709 13:01:53.941510  2373 rpcz_store.cc:276] 0709 13:01:51.580123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.580310 (+   187us) service_pool.cc:224] Handling call
0709 13:01:53.829924 (+2249614us) spinlock_profiling.cc:243] Waited 94.2 ms on lock 0x7b54004acc54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d65a3d 00007fc838d35cd0 00007fc8322ba254 00007fc8322ba1e2 00007fc8322ba111 00007fc8322ba08c 00007fc8322b9342 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:53.941027 (+111103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216761600}
I20260709 13:01:53.943322  3890 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:53.937350  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1152) took 1143 ms. Trace:
I20260709 13:01:53.709522  4141 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.922403  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:53.917404  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.970299  1721 raft_consensus.cc:2410] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:53.920449  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.933040  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.973588  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.884146  4135 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:53.916172  1573 rpcz_store.cc:276] 0709 13:01:52.636530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.480746 (+844216us) service_pool.cc:224] Handling call
0709 13:01:53.512147 (+ 31401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.915480 (+403333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.915714 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.897607  3972 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:53.945154  1644 rpcz_store.cc:276] 0709 13:01:52.794376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.065854 (+271478us) service_pool.cc:224] Handling call
0709 13:01:53.066372 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.066649 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.066677 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.067255 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.163008 (+ 95753us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:53.935805 (+772797us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.936953 (+  1148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.937041 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.937089 (+    48us) raft_consensus.cc:1685] finished
0709 13:01:53.937123 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.937284 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":567815,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":95,"raft.queue_time_us":111,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"thread_start_us":236216,"threads_started":2,"wal-append.queue_time_us":435370}
I20260709 13:01:53.967177  2377 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:01:53.944126  3890 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:53.986367  3890 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:53.983275  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55602.3us
I20260709 13:01:53.989360  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:53.992326  3890 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:53.994091  3890 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.001612  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:54.002440  1721 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.003505  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:54.004168  1725 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:53.994292  1647 raft_consensus.cc:2410] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:01:53.993001  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.018806  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:54.022039  1722 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:54.027743  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:53.911173  3973 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.032172  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:54.043666  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:54.048000  1725 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:53.944499  4159 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:53.906415  4148 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:53.875409  4041 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:54.061715  1721 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:53.994980  3890 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:54.065817  1614 connection.cc:256] RPC call timeout handler was delayed by -1.1350169769999638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
W20260709 13:01:54.069651  1614 connection.cc:256] RPC call timeout handler was delayed by -1.0855169769999407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
I20260709 13:01:54.151866  1650 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:54.054409  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.017161  4135 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:54.181661  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 109743us
W20260709 13:01:54.184890  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 214) took 1891 ms (client timeout 2102 ms). Trace:
I20260709 13:01:54.065395  4199 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:54.063874  4161 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.064481  4148 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:54.186007  1571 rpcz_store.cc:269] 0709 13:01:52.293852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.364954 (+ 71102us) service_pool.cc:224] Handling call
0709 13:01:54.077222 (+1712268us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54004209d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:54.184762 (+107540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361738112}
I20260709 13:01:54.063047  4041 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:54.164316  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1164) took 1315 ms. Trace:
I20260709 13:01:54.065634  4162 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.076758  4218 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.271245  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 88016.6us
W20260709 13:01:54.298626  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 309425us
I20260709 13:01:54.278731  4204 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:54.302861  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:54.304368  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.309626  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:54.311914  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:54.316115  2372 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.316375  4206 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:54.318043  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:54.319787  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:54.313616  1651 rpcz_store.cc:276] 0709 13:01:52.848537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.911574 (+1063037us) service_pool.cc:224] Handling call
0709 13:01:53.911953 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.912238 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:53.912660 (+   422us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.153357 (+240697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.153424 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.153497 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.163937 (+ 10440us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.164199 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83981,"prepare.queue_time_us":216,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":53,"raft.queue_time_us":87860,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":55238,"thread_start_us":152208,"threads_started":1}
I20260709 13:01:54.313616  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.329095  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:54.330899  4218 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.274369  4141 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:54.321043  4217 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:01:54.327788  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1842) took 1981 ms (client timeout 2209 ms). Trace:
I20260709 13:01:54.334213  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:54.334429  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:54.369333  1721 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:54.333783  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:54.194968  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:54.371497  1725 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:01:54.237756  1574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1806) took 2123 ms (client timeout 1805 ms). Trace:
W20260709 13:01:54.372609  1574 rpcz_store.cc:269] 0709 13:01:52.063827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.065678 (+  1851us) service_pool.cc:224] Handling call
0709 13:01:54.076301 (+2010623us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:54.187363 (+111062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054608768}
I20260709 13:01:54.374399  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:54.375247  1574 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
W20260709 13:01:54.352837  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.191007  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:54.381624  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47901.6us
I20260709 13:01:54.382182  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:54.382488  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:54.382819  1721 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:54.368997  1717 rpcz_store.cc:269] 0709 13:01:52.346007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.358283 (+ 12276us) service_pool.cc:224] Handling call
0709 13:01:54.327461 (+1969178us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:54.327691 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1928469248}
I20260709 13:01:54.385659  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:53.927412  4078 rpcz_store.cc:276] 0709 13:01:48.922891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.924152 (+  1261us) service_pool.cc:224] Handling call
0709 13:01:49.547609 (+623457us) ts_tablet_manager.cc:1151] Deleting tablet b837a5ec9a0d4f2a8635dedf45d51aa2
0709 13:01:49.568935 (+ 21326us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:01:50.837545 (+1268610us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400281ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d76362 00007fc83776134c 00007fc838d8460a 00007fc838db52d0 00007fc838d9b83d 00007fc838d9b7e9 00007fc838d9b779 00007fc838d9b741 00007fc838d9a96d 000000000038f354
0709 13:01:52.816736 (+1979191us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:53.162915 (+346179us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:53.926562 (+763647us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":627036,"spinlock_wait_cycles":545057024,"tablet-delete.queue_time_us":374475,"thread_start_us":266707,"threads_started":1}
W20260709 13:01:54.369939  4130 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.535s	user 0.015s	sys 0.001s
W20260709 13:01:54.385353  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 142608us
I20260709 13:01:54.367740  4098 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:54.387847  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 54842.4us
I20260709 13:01:54.386610   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:54.386721  1574 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.387311  4148 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:54.189880  4159 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:54.187346  4221 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:54.426986  1719 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:54.448279  1574 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:54.439935  1721 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:54.390967  4199 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:54.189864  4197 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.216s
I20260709 13:01:54.367157  4161 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.452086  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:54.448566  1719 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:54.393432  4162 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:54.450278  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 396230us
I20260709 13:01:54.377063  4201 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:01:54.455639  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:54.501241  1574 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:54.501503  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 80629us
W20260709 13:01:54.415045  3861 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.002s
I20260709 13:01:54.404814  3782 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:54.412823  3999 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.377939  4141 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.405900  4130 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:54.502393  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:54.484855  1719 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.537679  1719 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.538101  1574 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:01:54.513190  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.539428  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:54.519338  1644 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:54.438730  4148 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:54.438282  3972 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:54.533250  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:54.460861  2372 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:54.463440  4199 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:54.556797  1648 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:54.405320  4215 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:01:54.545635  1574 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:01:54.631779  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.561684  1569 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.534085  4232 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:54.546450  3999 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.643603  1569 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:54.629236  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1818) took 1963 ms. Trace:
I20260709 13:01:54.645231  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:54.646168  1569 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:01:54.558032  4079 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.030s	sys 0.019s
I20260709 13:01:54.646202  1573 rpcz_store.cc:276] 0709 13:01:52.665685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.975174 (+1309489us) service_pool.cc:224] Handling call
0709 13:01:54.042546 (+ 67372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.042869 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:54.435124 (+392255us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.628768 (+193644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.628834 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.628904 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.628986 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.629152 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160,"thread_start_us":570039,"threads_started":2}
I20260709 13:01:54.650693  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:54.651901  1573 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.647442  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:54.652213  4079 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.654121  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:54.559458  4148 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.655822  4079 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.666839  3999 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.676492  1573 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:54.681125  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1215) took 1840 ms. Trace:
I20260709 13:01:54.683919  1573 rpcz_store.cc:276] 0709 13:01:52.840444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.679973 (+1839529us) service_pool.cc:224] Handling call
0709 13:01:54.680515 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.680735 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.680760 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.680808 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.680857 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.680915 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.681039 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.685362  4079 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:54.687238  1573 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:54.546875  3782 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.453423  4159 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.640829  4232 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.696974  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1216) took 1851 ms. Trace:
I20260709 13:01:54.546319  4182 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.700958  4229 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:01:54.703387  1574 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:54.705796  1573 rpcz_store.cc:276] 0709 13:01:52.845925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.686595 (+1840670us) service_pool.cc:224] Handling call
0709 13:01:54.687022 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.694867 (+  7845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.696748 (+  1881us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.719494  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1824) took 1868 ms. Trace:
I20260709 13:01:54.720389  1574 rpcz_store.cc:276] 0709 13:01:52.850901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.694875 (+1843974us) service_pool.cc:224] Handling call
0709 13:01:54.703161 (+  8286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.719030 (+ 15869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.719419 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.720881  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1217) took 1859 ms. Trace:
I20260709 13:01:54.620445  3837 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:54.520432  4162 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.543176  3861 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.721292  1573 rpcz_store.cc:276] 0709 13:01:52.861723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.719419 (+1857696us) service_pool.cc:224] Handling call
0709 13:01:54.719998 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.720470 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.720512 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.720561 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:54.720603 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.720664 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.720819 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.706558  3999 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.738878  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1219) took 1861 ms. Trace:
I20260709 13:01:54.633169  4199 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:54.729678  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1218) took 1857 ms. Trace:
W20260709 13:01:54.727077  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94047.1us
I20260709 13:01:54.749512  1574 rpcz_store.cc:276] 0709 13:01:52.872263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.724265 (+1852002us) service_pool.cc:224] Handling call
0709 13:01:54.728945 (+  4680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.729190 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.729229 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.729289 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:54.729343 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.729407 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.729583 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.626621  4130 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:54.621841  4214 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:54.739588  1573 rpcz_store.cc:276] 0709 13:01:52.877808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.737803 (+1859995us) service_pool.cc:224] Handling call
0709 13:01:54.738120 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.738452 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.738479 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.738527 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:54.738565 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.738631 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.738808 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:54.741287  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 238996us
I20260709 13:01:54.756796  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1221) took 1878 ms. Trace:
W20260709 13:01:54.757899  4219 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
W20260709 13:01:54.542707  4230 consensus_queue.cc:1175] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd6d377efe514bc58ae49c3f4a107232" responder_term: 7 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:54.725654  4182 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:54.761734  4226 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:01:54.618567  3972 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.465032  4221 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.779404  1574 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:54.780486  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1220) took 1902 ms. Trace:
I20260709 13:01:54.781189  1574 rpcz_store.cc:276] 0709 13:01:52.878229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.751640 (+1873411us) service_pool.cc:224] Handling call
0709 13:01:54.778993 (+ 27353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.779933 (+   940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.780392 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.757679  1573 rpcz_store.cc:276] 0709 13:01:52.878545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.755914 (+1877369us) service_pool.cc:224] Handling call
0709 13:01:54.756254 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.756474 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.756499 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.756544 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:54.756580 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.756632 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.756742 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.718298  4232 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.767004  1651 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:54.783875  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.747159  4201 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:54.790257  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1224) took 1802 ms. Trace:
I20260709 13:01:54.790797  1573 rpcz_store.cc:276] 0709 13:01:52.987722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.789118 (+1801396us) service_pool.cc:224] Handling call
0709 13:01:54.789539 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.789812 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.789844 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.789895 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.789944 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.790021 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.790175 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.761830  4159 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.763239  3837 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:54.797470  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1225) took 1798 ms. Trace:
I20260709 13:01:54.738055  3999 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.753907  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:01:54.804389  1614 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.750s (SENT) [suppressed 9 similar messages]
I20260709 13:01:54.809976  1573 rpcz_store.cc:276] 0709 13:01:52.999460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.793096 (+1793636us) service_pool.cc:224] Handling call
0709 13:01:54.793573 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.796609 (+  3036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.796672 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.796795 (+   123us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:54.796886 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.797074 (+   188us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.797341 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.816098  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1223) took 1831 ms. Trace:
W20260709 13:01:54.787349  4223 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.246s	user 0.001s	sys 0.001s
I20260709 13:01:54.811568  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.809048  1571 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 3.
I20260709 13:01:54.817210  1574 rpcz_store.cc:276] 0709 13:01:52.985169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.792971 (+1807802us) service_pool.cc:224] Handling call
0709 13:01:54.815470 (+ 22499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.815724 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.815748 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.815795 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.815826 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.815878 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.816031 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.825214  1574 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:54.826002  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1227) took 1809 ms. Trace:
I20260709 13:01:54.819742  1573 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:54.778996  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.826402  1574 rpcz_store.cc:276] 0709 13:01:53.043972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.824642 (+1780670us) service_pool.cc:224] Handling call
0709 13:01:54.825027 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.825717 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.825942 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.788966  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 375) took 1340 ms. Trace:
I20260709 13:01:54.834312  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1226) took 1822 ms. Trace:
I20260709 13:01:54.834858  1573 rpcz_store.cc:276] 0709 13:01:53.011879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.814849 (+1802970us) service_pool.cc:224] Handling call
0709 13:01:54.819524 (+  4675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.833899 (+ 14375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.834224 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.838418  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.841841  1573 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:54.809391  4218 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.766810  4130 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.810140  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.857440  1574 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:54.766922  4041 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:54.858577  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.862118  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1228) took 1817 ms. Trace:
I20260709 13:01:54.770167  4214 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:54.866875  1574 rpcz_store.cc:276] 0709 13:01:53.044825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.828704 (+1783879us) service_pool.cc:224] Handling call
0709 13:01:54.857076 (+ 28372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.861509 (+  4433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.862042 (+   533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.833361  4141 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:54.846256  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1229) took 1778 ms. Trace:
I20260709 13:01:54.875526  1573 rpcz_store.cc:276] 0709 13:01:53.067416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.836436 (+1769020us) service_pool.cc:224] Handling call
0709 13:01:54.841510 (+  5074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.842608 (+  1098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.845595 (+  2987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.878664  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:54.880559  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.855594  3890 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:54.810747  4201 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.896674  2373 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:54.904635  3782 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:54.903859  3861 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.925210  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1165) took 1521 ms. Trace:
I20260709 13:01:54.933395  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1817) took 2271 ms. Trace:
I20260709 13:01:54.934208  1568 rpcz_store.cc:276] 0709 13:01:52.662066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.885798 (+1223732us) service_pool.cc:224] Handling call
0709 13:01:53.898170 (+ 12372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.276133 (+377963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.276169 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.276227 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:54.276273 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.932971 (+656698us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54003101d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:54.933038 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.933280 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868572928}
I20260709 13:01:54.936419  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1808) took 1013 ms. Trace:
I20260709 13:01:54.937238  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1176) took 1064 ms. Trace:
I20260709 13:01:54.938124  4041 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.907456  4221 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:54.907114  4182 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.941066  3972 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:54.907863  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.942625  2376 raft_consensus.cc:2376] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5. Re-sending same reply.
W20260709 13:01:54.943316  2376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 370) took 2645 ms (client timeout 1534 ms). Trace:
I20260709 13:01:54.887916  3837 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.909385  4162 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.910175  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:54.955492  2380 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:01:54.956138  2380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 372) took 2657 ms (client timeout 2209 ms). Trace:
I20260709 13:01:54.958544  2375 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:01:54.927241  1649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1148) took 2135 ms (client timeout 2353 ms). Trace:
W20260709 13:01:54.960063  1649 rpcz_store.cc:269] 0709 13:01:52.791373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.791494 (+   121us) service_pool.cc:224] Handling call
0709 13:01:54.926948 (+2135454us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400389fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:54.927125 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3165481728}
I20260709 13:01:54.905805  4148 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.961500  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:54.962985  4148 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:54.913331  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1808) took 2844 ms. Trace:
W20260709 13:01:54.965750  2342 connection.cc:256] RPC call timeout handler was delayed by -1.9026966880000487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 13:01:54.966964  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:54.937856  1644 rpcz_store.cc:276] 0709 13:01:53.872474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.995470 (+122996us) service_pool.cc:224] Handling call
0709 13:01:54.049541 (+ 54071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.545540 (+495999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.936971 (+391431us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:54.937174 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079787392}
I20260709 13:01:54.912151  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 239) took 2741 ms. Trace:
I20260709 13:01:54.925966  1650 rpcz_store.cc:276] 0709 13:01:53.404235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.922350 (+518115us) service_pool.cc:224] Handling call
0709 13:01:53.922802 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.152573 (+229771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.924871 (+772298us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54003fc854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:54.925125 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352027008}
I20260709 13:01:54.972237  1719 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:01:54.956615  2380 rpcz_store.cc:269] 0709 13:01:52.298591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.298817 (+   226us) service_pool.cc:224] Handling call
0709 13:01:54.909987 (+2611170us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:54.956039 (+ 46052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50675584}
I20260709 13:01:54.962293  1649 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:01:54.908752  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:54.969877  1724 rpcz_store.cc:276] 0709 13:01:52.171039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.691990 (+520951us) service_pool.cc:224] Handling call
0709 13:01:52.752683 (+ 60693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.786300 (+ 33617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.786338 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:52.786387 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:52.786424 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.911814 (+2125390us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b540010dbd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:54.911872 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.912042 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76761344}
I20260709 13:01:54.965030  1576 rpcz_store.cc:276] 0709 13:01:52.068976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.078399 (+  9423us) service_pool.cc:224] Handling call
0709 13:01:52.078863 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.362815 (+1283952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.912543 (+1549728us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003106d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:54.912941 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536226048}
I20260709 13:01:54.944270  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:54.937271  1722 rpcz_store.cc:276] 0709 13:01:53.923504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.033360 (+109856us) service_pool.cc:224] Handling call
0709 13:01:54.033804 (+   444us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:54.366767 (+332963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.366821 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:54.414706 (+ 47885us) log.cc:844] Serialized 78 byte log entry
0709 13:01:54.745401 (+330695us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:54.745447 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.745524 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.935904 (+190380us) raft_consensus.cc:1685] finished
0709 13:01:54.935938 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.936298 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103600,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":55590,"thread_start_us":516281,"threads_started":2,"wal-append.queue_time_us":389059}
I20260709 13:01:54.984930  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:54.988147  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55387.4us
I20260709 13:01:54.990760  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:54.945528  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:55.001761  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56350.4us
I20260709 13:01:55.002859  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.001724  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.004884  1719 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:55.005600  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.002950  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:55.008510  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.008734  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:54.943755  2376 rpcz_store.cc:269] 0709 13:01:52.297406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.301266 (+  3860us) service_pool.cc:224] Handling call
0709 13:01:54.907652 (+2606386us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:54.943212 (+ 35560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55122176}
I20260709 13:01:54.884704  4197 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:01:55.043638  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1829) took 1030 ms. Trace:
I20260709 13:01:54.901228  4215 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:54.909744  4095 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:55.043681  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:55.052546  1568 rpcz_store.cc:276] 0709 13:01:54.013418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.986321 (+972903us) service_pool.cc:224] Handling call
0709 13:01:55.043014 (+ 56693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.043224 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.043263 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.043315 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:55.043354 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.043413 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.043543 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.046998  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1830) took 1033 ms. Trace:
I20260709 13:01:55.054268  1576 rpcz_store.cc:276] 0709 13:01:54.013806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.002267 (+988461us) service_pool.cc:224] Handling call
0709 13:01:55.046079 (+ 43812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.046358 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.046436 (+    78us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.046507 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:55.046566 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.046657 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.046893 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.910429  3999 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:54.914696  4201 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:55.017560  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:54.984143  4041 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:55.006697  3972 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:55.051506  4204 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:55.021188  4030 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:55.068084  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.089787  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 219) took 1037 ms. Trace:
I20260709 13:01:55.092007  1568 rpcz_store.cc:276] 0709 13:01:54.052081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.059237 (+1007156us) service_pool.cc:224] Handling call
0709 13:01:55.089069 (+ 29832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.089316 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.089355 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.089417 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:55.089475 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.089558 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.089708 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.081485  1576 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:55.083113  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:55.096220  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:55.085716  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 246) took 2687 ms. Trace:
I20260709 13:01:55.042191  3782 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:55.098510  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1204) took 2462 ms. Trace:
W20260709 13:01:55.099143  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1851) took 1650 ms (client timeout 2096 ms). Trace:
I20260709 13:01:55.056310  4095 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:55.057958  4101 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:55.104499  1572 rpcz_store.cc:276] 0709 13:01:52.635726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.438226 (+802500us) service_pool.cc:224] Handling call
0709 13:01:53.438600 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.542129 (+103529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.098196 (+1556067us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400364cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:55.098433 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236208896}
W20260709 13:01:55.103907  1720 rpcz_store.cc:269] 0709 13:01:53.448757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.467594 (+ 18837us) service_pool.cc:224] Handling call
0709 13:01:55.098516 (+1630922us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400294ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:55.099053 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788296448}
I20260709 13:01:55.107582  1720 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:55.108943  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:55.112160  1720 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:55.115917  1720 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:55.061096  3861 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:55.122186  1723 rpcz_store.cc:276] 0709 13:01:52.398358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.922752 (+524394us) service_pool.cc:224] Handling call
0709 13:01:55.084640 (+2161888us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:01:55.084910 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.085192 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.085239 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.085300 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:55.085357 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.085437 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.085617 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27495936}
I20260709 13:01:55.079716  2372 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:55.117810  1720 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.055826  4203 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:01:55.102543  3999 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:55.187239  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.196127  1720 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:55.192590  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140593us
W20260709 13:01:55.206351  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.153s (SENT)
I20260709 13:01:55.227919  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:55.226717  1644 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:55.235746  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:55.182390  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:55.244818  1643 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:55.185062  3972 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:55.258663  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.266541  4159 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:55.267114  4224 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
I20260709 13:01:55.267526  4159 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:54.832212  2371 rpcz_store.cc:276] 0709 13:01:53.444388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.452331 (+  7943us) service_pool.cc:224] Handling call
0709 13:01:53.504765 (+ 52434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.511358 (+  6593us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:53.571794 (+ 60436us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.017641 (+445847us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.173108 (+155467us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.357720 (+184612us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:54.367129 (+  9409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.384635 (+ 17506us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.766592 (+381957us) raft_consensus.cc:1685] finished
0709 13:01:54.766615 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.784678 (+ 18063us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14120,"prepare.queue_time_us":51712,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":15997,"raft.queue_time_us":26490,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":30626,"thread_start_us":391270,"threads_started":1,"wal-append.queue_time_us":686661}
I20260709 13:01:55.268989  4215 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:55.269600  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.271231  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1162) took 2433 ms. Trace:
I20260709 13:01:55.232905  3999 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:55.269197  4095 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.289006  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:55.289368  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:55.266618  4250 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:55.290956  3999 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:55.287705  1720 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:01:55.297384  1720 pending_rounds.cc:85] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 3
I20260709 13:01:55.298893  1720 pending_rounds.cc:107] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:01:55.251982  4204 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:55.299396  1720 raft_consensus.cc:2889] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.270681  4130 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.289443  4257 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:55.497262  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.501561  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 401673us
I20260709 13:01:55.283975  1646 rpcz_store.cc:276] 0709 13:01:52.837928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.891881 (+1053953us) service_pool.cc:224] Handling call
0709 13:01:53.892267 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.892744 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.892774 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.954210 (+ 61436us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.299085 (+344875us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.305672 (+  6587us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.340835 (+ 35163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.340939 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.270913 (+929974us) raft_consensus.cc:1685] finished
0709 13:01:55.270946 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.271147 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34862,"prepare.queue_time_us":6008,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":35337,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":23795,"thread_start_us":245771,"threads_started":1,"wal-append.queue_time_us":1164651}
I20260709 13:01:55.512959  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1799) took 1097 ms. Trace:
I20260709 13:01:55.291509  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:55.514940  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.356994  4030 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:55.578408  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:55.377848  4041 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:01:55.523223  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 316920us
I20260709 13:01:55.524142  3837 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:55.543412  1643 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:55.310655  4095 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:55.309170  4133 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.588914  4133 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.592110  3837 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.548558  4159 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.312274  4204 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.577188  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1811) took 1030 ms. Trace:
I20260709 13:01:55.556836  4252 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.582512  4030 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.588068  4041 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:55.573488  1644 rpcz_store.cc:276] 0709 13:01:54.414975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.255417 (+840442us) service_pool.cc:224] Handling call
0709 13:01:55.512493 (+257076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.617581  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1827) took 2532 ms. Trace:
W20260709 13:01:55.604115  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 218) took 3308 ms (client timeout 3186 ms). Trace:
I20260709 13:01:55.608669  1645 rpcz_store.cc:276] 0709 13:01:54.546906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.291099 (+744193us) service_pool.cc:224] Handling call
0709 13:01:55.576845 (+285746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.604044  3837 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:55.618655  1567 rpcz_store.cc:269] 0709 13:01:52.295118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.497870 (+202752us) service_pool.cc:224] Handling call
0709 13:01:55.603351 (+3105481us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b540027d154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:55.603682 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2286223616}
I20260709 13:01:55.635015  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.638011  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1805) took 1181 ms. Trace:
I20260709 13:01:55.617589  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.642113  1643 rpcz_store.cc:276] 0709 13:01:54.448959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.089053 (+640094us) service_pool.cc:224] Handling call
0709 13:01:55.622120 (+533067us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.602193  4267 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:55.650573  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:55.658658  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1172) took 2243 ms (client timeout 1878 ms). Trace:
W20260709 13:01:55.659382  1647 rpcz_store.cc:269] 0709 13:01:53.415096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.468294 (+ 53198us) service_pool.cc:224] Handling call
0709 13:01:55.657931 (+2189637us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5400307354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:55.658319 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717398272}
I20260709 13:01:55.660138  3837 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:55.661216  4252 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:55.661864  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1167) took 2256 ms (client timeout 1875 ms). Trace:
W20260709 13:01:55.664105  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.658701  4130 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:01:55.668596  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.669683  1646 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:55.670965  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1806) took 1213 ms. Trace:
I20260709 13:01:55.671401  1646 rpcz_store.cc:276] 0709 13:01:54.457397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.632419 (+1175022us) service_pool.cc:224] Handling call
0709 13:01:55.670895 (+ 38476us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.673285  1572 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:55.673686  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:55.676530  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.676885  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.677399  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:55.678038  1720 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:01:55.662984  1648 rpcz_store.cc:269] 0709 13:01:53.405174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.443053 (+ 37879us) service_pool.cc:224] Handling call
0709 13:01:55.661365 (+2218312us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b5400133b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:55.661634 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215885568}
I20260709 13:01:55.690469  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.692719  1648 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:55.693812  3688 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:55.688027  1717 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:55.645476  4267 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:01:55.699285  4252 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:55.697678  4267 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:55.693737  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1807) took 1227 ms. Trace:
I20260709 13:01:55.704891  1648 rpcz_store.cc:276] 0709 13:01:54.466601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.689613 (+1223012us) service_pool.cc:224] Handling call
0709 13:01:55.693627 (+  4014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.618446  1574 rpcz_store.cc:276] 0709 13:01:53.084738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.869691 (+1784953us) service_pool.cc:224] Handling call
0709 13:01:54.888291 (+ 18600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.888636 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:55.145805 (+257169us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.616515 (+470710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.616682 (+   167us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:55.616852 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.616979 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.617300 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":326794,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":58191,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":381,"thread_start_us":601046,"threads_started":3}
I20260709 13:01:55.712687  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:55.716600  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.718163  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:55.718433  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.719242  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1810) took 1178 ms. Trace:
I20260709 13:01:55.720567  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:55.720860  4206 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:55.722014  4126 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.722587  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:55.723183  1644 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:55.723783  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1808) took 1241 ms. Trace:
I20260709 13:01:55.724291  1644 rpcz_store.cc:276] 0709 13:01:54.482790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.616747 (+1133957us) service_pool.cc:224] Handling call
0709 13:01:55.723688 (+106941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.724678  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:55.726230  1644 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:55.726938  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1804) took 1308 ms. Trace:
I20260709 13:01:55.727319  1644 rpcz_store.cc:276] 0709 13:01:54.418147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.725077 (+1306930us) service_pool.cc:224] Handling call
0709 13:01:55.725669 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.726620 (+   951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.726838 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.729256  1572 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:55.730854  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.732860  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.734500  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1801) took 1318 ms. Trace:
I20260709 13:01:55.734254  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.735805  1647 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:55.716679  1643 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:55.739356  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1803) took 1321 ms. Trace:
I20260709 13:01:55.737040  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:55.719655  1645 rpcz_store.cc:276] 0709 13:01:54.541069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.718045 (+1176976us) service_pool.cc:224] Handling call
0709 13:01:55.719134 (+  1089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.739857  1647 rpcz_store.cc:276] 0709 13:01:54.417836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.680920 (+1263084us) service_pool.cc:224] Handling call
0709 13:01:55.739285 (+ 58365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.750403  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.753333  1717 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:55.704723  4133 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.756726  1568 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:01:55.735268  1646 rpcz_store.cc:276] 0709 13:01:54.417117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.672276 (+1255159us) service_pool.cc:224] Handling call
0709 13:01:55.734402 (+ 62126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.757398  1568 pending_rounds.cc:85] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 2
I20260709 13:01:55.762092  1568 pending_rounds.cc:107] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:01:55.762507  1568 raft_consensus.cc:2889] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.732208  1722 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:55.739698  1643 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:55.737080  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.774425  4135 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:55.777499  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1828) took 2364 ms. Trace:
I20260709 13:01:55.770704  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.781209  1648 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:55.781910  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1812) took 1215 ms. Trace:
I20260709 13:01:55.737033  3999 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:55.782330  1648 rpcz_store.cc:276] 0709 13:01:54.567140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.705822 (+1138682us) service_pool.cc:224] Handling call
0709 13:01:55.781835 (+ 76013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.783335  1643 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:55.784623  3999 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:55.785147  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.787393  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.787680  4158 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:55.784587  1644 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:55.750810  4134 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.736263  4132 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.790786  1719 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:55.792011  1717 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:55.792150  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:55.784538  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:55.784109  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:55.793191  1719 raft_consensus.cc:2376] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8. Re-sending same reply.
I20260709 13:01:55.778368  1573 rpcz_store.cc:276] 0709 13:01:53.413142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.886538 (+1473396us) service_pool.cc:224] Handling call
0709 13:01:54.888871 (+  2333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.889117 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:55.079199 (+190082us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.641318 (+562119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.667038 (+ 25720us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:55.757092 (+ 90054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.767289 (+ 10197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.777268 (+  9979us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84371,"prepare.queue_time_us":326565,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":39119,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":203,"thread_start_us":608772,"threads_started":3,"wal-append.queue_time_us":628289}
I20260709 13:01:55.794106  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:55.795116  1572 raft_consensus.cc:2376] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8. Re-sending same reply.
I20260709 13:01:55.795233  4141 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.815488  2371 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:55.800104  1643 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:55.796420  1717 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:55.796792  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:55.807062  4158 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:55.821319  2371 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:55.796133  4141 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:55.787943  4232 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.987s	user 0.017s	sys 0.034s
I20260709 13:01:55.822232  4158 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:55.796232  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:55.823256  4232 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.811275  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:55.792268  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:55.835218  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:55.841945  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49921.3us
I20260709 13:01:55.835381  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.851238  1722 raft_consensus.cc:2368] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6.
I20260709 13:01:55.853111  4141 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.855166  2371 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:01:55.859175  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.102s (SENT)
W20260709 13:01:55.861773  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 58371.2us
I20260709 13:01:55.866684  4141 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:55.871416  1572 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:55.872905  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:55.874742  4141 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:55.876621  4141 consensus_queue.cc:1243] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:55.877310  4158 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:55.882975  4130 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:55.813355  4133 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.888746  4265 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:55.889453  4265 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:01:55.890013  4265 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:55.894312  4206 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.895462  4206 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:55.884248  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:55.909611  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:55.912752  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:55.915143  4214 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:55.917059  4199 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.154s	user 0.020s	sys 0.021s
W20260709 13:01:55.917408  4211 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 2.353s	user 0.007s	sys 0.000s
I20260709 13:01:55.917703  4199 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:55.879871  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:55.921427  4199 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:55.924302  1572 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:55.925050  1723 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:01:55.926358  4258 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.367s	user 0.001s	sys 0.001s
I20260709 13:01:55.927389  1723 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:55.897423  1643 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:55.922287  4214 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:55.928892  4174 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:01:55.930568  4185 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.932199  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 242) took 3759 ms. Trace:
I20260709 13:01:55.932765  1718 rpcz_store.cc:276] 0709 13:01:52.172189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.771591 (+599402us) service_pool.cc:224] Handling call
0709 13:01:52.772002 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.028637 (+256635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.028682 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.028738 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:53.028775 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.931589 (+2902814us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54003dc7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:55.931690 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.931926 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589620480}
I20260709 13:01:55.933879  4206 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:55.937141  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:01:55.939582  4182 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:55.940457  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 274) took 1017 ms. Trace:
W20260709 13:01:55.940892  4275 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.229s	user 0.000s	sys 0.002s
W20260709 13:01:55.942471  2373 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 376) took 1668 ms (client timeout 2120 ms). Trace:
I20260709 13:01:55.940019  3837 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:01:55.939366  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:55.944784  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:55.945706  4278 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 13:01:55.946722  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:55.953787  1572 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:55.963466  4161 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:55.897960  4267 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:55.967301  4200 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:55.970212   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:55.972026  1643 rpcz_store.cc:276] 0709 13:01:54.922901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.652326 (+729425us) service_pool.cc:224] Handling call
0709 13:01:55.940368 (+288042us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:55.981267  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.972431  4182 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:01:56.001046  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 56475.6us
I20260709 13:01:56.001811  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1177) took 2102 ms. Trace:
I20260709 13:01:56.002699  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:56.003338  1570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1807) took 3938 ms (client timeout 3142 ms). Trace:
W20260709 13:01:55.974550  2373 rpcz_store.cc:269] 0709 13:01:54.273733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.278364 (+  4631us) service_pool.cc:224] Handling call
0709 13:01:55.941870 (+1663506us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54005e01d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:55.942191 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1318671232}
I20260709 13:01:56.005440  4182 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.006883  4204 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:56.009450  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63050.4us
I20260709 13:01:56.009955  4100 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.010870  4261 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:56.011502  4204 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:01:56.012846  4282 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.175s	user 0.003s	sys 0.000s
I20260709 13:01:56.015123  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1813) took 2085 ms. Trace:
I20260709 13:01:56.018226  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1819) took 1243 ms. Trace:
I20260709 13:01:56.018839  1725 rpcz_store.cc:276] 0709 13:01:54.774549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.774721 (+   172us) service_pool.cc:224] Handling call
0709 13:01:54.775247 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.912795 (+137548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.912826 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.912867 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:54.912905 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.017646 (+1104741us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001a6dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:56.017717 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.017955 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315369728}
I20260709 13:01:56.021684  4060 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:56.021490  4159 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:56.007970  4191 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:56.023108  4159 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:56.023686  4159 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:56.025013  4159 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.003928  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:56.015671  1716 rpcz_store.cc:276] 0709 13:01:53.930016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.051802 (+121786us) service_pool.cc:224] Handling call
0709 13:01:54.068312 (+ 16510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.141804 (+ 73492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.141887 (+    83us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.298786 (+156899us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.654358 (+1355572us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b180015a600. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:01:55.710566 (+ 56208us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:55.711540 (+   974us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.712020 (+   480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.712154 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.014489 (+302335us) raft_consensus.cc:1685] finished
0709 13:01:56.014656 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.014842 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96096,"prepare.queue_time_us":542,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":211736,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":4285,"spinlock_wait_cycles":120482688,"thread_start_us":55621,"threads_started":1,"wal-append.queue_time_us":1405700}
I20260709 13:01:56.028461  3861 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:56.001322  1567 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:01:56.030093  1567 pending_rounds.cc:85] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 2
I20260709 13:01:56.030510  1567 pending_rounds.cc:107] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:01:56.027185  4159 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.030829  1567 raft_consensus.cc:2889] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:56.002293  1651 rpcz_store.cc:276] 0709 13:01:53.872788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.325800 (+453012us) service_pool.cc:224] Handling call
0709 13:01:54.326160 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.783004 (+456844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.973117 (+1190113us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54003fd754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:55.975277 (+  2160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013367680}
I20260709 13:01:56.004014  4245 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:56.003973  1570 rpcz_store.cc:269] 0709 13:01:52.068396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.068632 (+   236us) service_pool.cc:224] Handling call
0709 13:01:55.974421 (+3905789us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540027db54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:56.003080 (+ 28659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574599936}
I20260709 13:01:56.042254  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:56.044739  1573 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:56.045106  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 249) took 1116 ms. Trace:
I20260709 13:01:56.029372  3861 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:56.046141  4261 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:56.047684  4064 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.047555  3861 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.048460  4174 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:56.049134  3861 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:56.049261  4174 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:56.049569  3782 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:56.045738  1724 rpcz_store.cc:276] 0709 13:01:54.924827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.984614 (+ 59787us) service_pool.cc:224] Handling call
0709 13:01:55.938995 (+954381us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:01:56.040744 (+101749us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6884480}
I20260709 13:01:55.988111  4041 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:01:56.038679  4245 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.056995  3782 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.079082  4191 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.079334  4060 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:56.080447  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:56.080668  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.081321  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:56.082826  1725 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:56.065451  4245 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.064224  2371 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:56.089291  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.089200  4060 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.095119  1568 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:56.095999  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.090646  1725 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:56.101653  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:56.104933  4268 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:56.110805  4174 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.126860  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:56.113569  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.114298  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:56.090901  3782 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.134068  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.129940  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:56.134066  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.139183  2372 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:01:56.144604  4060 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.072222  4158 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:01:56.125976  4268 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:56.148586  1716 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:56.152513  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:56.097249  4182 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.148028  1725 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.158012  1568 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:56.158478  1725 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:56.161276  4279 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.246s	user 0.005s	sys 0.000s
W20260709 13:01:56.164664  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 1.85152e+06us
W20260709 13:01:56.165457  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.145522  4265 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.157788  4178 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:56.183832  1725 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:56.183439  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58597.4us
W20260709 13:01:56.193542  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79403.3us
I20260709 13:01:56.189620  4064 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:56.201112  4064 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:56.202667  4064 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.205408  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:56.131628  4191 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.177358  4141 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:56.178726  4232 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:56.200953  1649 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:01:56.216740  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.224382  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.226367  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:56.214464  4267 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:56.245422  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.240796  4178 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:56.274825  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:56.206995  4268 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:56.287256  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73886.8us
I20260709 13:01:56.274606  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:56.271320  4178 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:01:56.303979  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 153556us
I20260709 13:01:56.298717  4178 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:56.315740  3972 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:01:56.297699  4214 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.327219  4268 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.343052  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:56.354831  4068 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:01:56.340593  4262 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.023s	user 0.014s	sys 0.000s
I20260709 13:01:56.359531  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1820) took 3670 ms. Trace:
I20260709 13:01:56.362179  4265 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:56.363161  4284 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 13:01:56.367672  4281 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.209s	user 0.003s	sys 0.000s
I20260709 13:01:56.370188  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1181) took 1425 ms. Trace:
I20260709 13:01:56.370714  1648 rpcz_store.cc:276] 0709 13:01:54.945099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.784141 (+839042us) service_pool.cc:224] Handling call
0709 13:01:55.784676 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.791732 (+  7056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.791766 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.792550 (+   784us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.035507 (+242957us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:56.044141 (+  8634us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.055757 (+ 11616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.055859 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.369855 (+313996us) raft_consensus.cc:1685] finished
0709 13:01:56.369880 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.370052 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11257,"prepare.queue_time_us":8000,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":17919,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":3333,"thread_start_us":242465,"threads_started":1,"wal-append.queue_time_us":303798}
I20260709 13:01:56.346463  4182 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:56.338409  4293 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:56.378295  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.379588  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.380589  4232 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.381353  4232 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.383864  4232 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:56.384054  4285 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:01:56.406641  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:01:56.327723  4267 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:01:56.394110  4158 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.426092  1572 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:56.426327  4079 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.427091  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.440140  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1163) took 3594 ms. Trace:
I20260709 13:01:56.434486  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.435734  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.449131  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.452873  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.454070  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.456138  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:56.359963  1569 rpcz_store.cc:276] 0709 13:01:52.688896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.655332 (+1966436us) service_pool.cc:224] Handling call
0709 13:01:54.655735 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:54.771221 (+115486us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:55.089189 (+317968us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.241422 (+152233us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:55.242140 (+   718us) log.cc:844] Serialized 52 byte log entry
0709 13:01:55.242300 (+   160us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:55.242333 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.242418 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.014855 (+772437us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b1800364120. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:01:56.285392 (+270537us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:01:56.359039 (+ 73647us) raft_consensus.cc:1685] finished
0709 13:01:56.359063 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.359297 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":425128,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":9564,"raft.queue_time_us":72244,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":13631,"spinlock_wait_cycles":102817920,"thread_start_us":454946,"threads_started":3,"wal-append.queue_time_us":188481}
I20260709 13:01:56.457975  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:56.468055  1570 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:56.348775  4141 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
W20260709 13:01:56.457657  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1721594909999453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
W20260709 13:01:56.476889  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1577594909999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:01:56.477345  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1577594909999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:01:56.477705  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1505594909999672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
W20260709 13:01:56.489996  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 215294us
I20260709 13:01:56.467917  1642 rpcz_store.cc:276] 0709 13:01:52.845097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.892029 (+1046932us) service_pool.cc:224] Handling call
0709 13:01:53.892502 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.563989 (+671487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.564039 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.616229 (+ 52190us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.889489 (+273260us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:55.181843 (+292354us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.185643 (+  3800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.438988 (+1253345us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b540038aed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:56.439066 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.439119 (+    53us) raft_consensus.cc:1685] finished
0709 13:01:56.439218 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.439630 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29722,"prepare.queue_time_us":384222,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":67901,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":36411,"spinlock_wait_cycles":1527115008,"thread_start_us":372995,"threads_started":2,"wal-append.queue_time_us":362019}
I20260709 13:01:56.498610  1568 raft_consensus.cc:1217] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-6.5]   Dedup: 3.3->[4.4-6.5]
W20260709 13:01:56.508922  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1190594909999731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 13:01:56.509712  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1190594909999731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
I20260709 13:01:56.505021  1568 pending_rounds.cc:85] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 3
I20260709 13:01:56.515590  1568 pending_rounds.cc:107] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:01:56.516095  1568 raft_consensus.cc:2889] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:56.521499  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.496867  4191 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.506589  4261 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:56.552950  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.554217  1725 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.426160  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet b837a5ec9a0d4f2a8635dedf45d51aa2 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
W20260709 13:01:56.562419  1539 connection.cc:256] RPC call timeout handler was delayed by -2.11095949099996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 13:01:56.562980  1539 connection.cc:256] RPC call timeout handler was delayed by -2.11095949099996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:01:56.492084  4079 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.565176  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.577135  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:56.570596  1460 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 195597us
I20260709 13:01:56.579576  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:56.580536  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.580866  1718 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:56.581332  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.582067  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.583808  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.584039  4079 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:56.584697  1614 leader_election.cc:400] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6.
W20260709 13:01:56.584874  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46822.5us
I20260709 13:01:56.585913  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.586902  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:56.585084  1614 leader_election.cc:403] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:56.584723  1718 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:01:56.590517  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.586894  1724 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:01:56.527078  2372 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:01:56.597366  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.605021  4098 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:56.605862  4098 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.606278  4098 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.609370  4079 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:56.596853  4295 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:56.611080  4295 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.581262  4261 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:56.609472  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 226) took 2504 ms (client timeout 1538 ms). Trace:
I20260709 13:01:56.610176  4079 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.589025  4064 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 13:01:56.654249  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45910.8us
W20260709 13:01:56.665385  1571 rpcz_store.cc:269] 0709 13:01:54.067911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.302528 (+234617us) service_pool.cc:224] Handling call
0709 13:01:56.533225 (+2230697us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b540030a054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:56.572367 (+ 39142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1139522432}
I20260709 13:01:56.572919  3782 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.674878  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.578012  4245 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:56.682612  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0803594909999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
W20260709 13:01:56.683835  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0587594909999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:01:56.684202  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0587594909999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:01:56.684564  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0587594909999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
I20260709 13:01:56.669400  4079 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:56.685359  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 139232us
I20260709 13:01:56.689669  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.669811  4181 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.691388  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.683414  4214 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:56.708787  1539 connection.cc:256] RPC call timeout handler was delayed by -1.937592412000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
I20260709 13:01:56.713829  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:56.706622  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:56.717833  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.725265  1719 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:56.733135  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.743022  4245 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.747032  4261 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.750576  1571 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:56.717164  4232 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:01:56.754786  1474 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 114) took 14169 ms (client timeout 15000 ms). Trace:
I20260709 13:01:56.756124  1725 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:56.748394  4178 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.759919  4158 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:56.761252  1649 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:56.761400  3782 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.719980  4214 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:56.762514  1642 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:01:56.763363  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5392732839999894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:01:56.762089  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:56.764371  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:56.779543  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5392732839999894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:01:56.780530  1648 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:01:56.780766  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4474732840000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
I20260709 13:01:56.771024  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:56.781293  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4474732840000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 13:01:56.781746  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3583732839999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 13:01:56.782152  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3583732839999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 13:01:56.782564  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3295732839999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:01:56.782981  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3295732839999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:01:56.784031  1539 connection.cc:256] RPC call timeout handler was delayed by -1.262073283999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
W20260709 13:01:56.784869  1539 connection.cc:256] RPC call timeout handler was delayed by -1.262073283999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
I20260709 13:01:56.790668  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:56.791352  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0883732839999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
I20260709 13:01:56.791723  1644 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
W20260709 13:01:56.792186  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0883732839999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
W20260709 13:01:56.792898  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0316732840000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 13:01:56.793449  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0316732840000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
I20260709 13:01:56.808802  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.810817  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.777225  4191 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.813756  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 131066us
I20260709 13:01:56.818084  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.818866  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.819000  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.799702  4060 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.826265  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.826452  1725 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:56.827327  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.828576  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.822803  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:56.832525  4298 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 13:01:56.834699  4178 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.836524  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.837276  4120 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.837641  1650 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:56.838035  4120 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.838917  4191 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.840632  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.841085  4120 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:56.841563  1575 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:56.842680  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.844441  1717 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:56.845969  4060 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:56.820899  3782 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:01:56.757473  1474 rpcz_store.cc:269] 0709 13:01:42.585299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.593671 (+  8372us) service_pool.cc:224] Handling call
0709 13:01:44.495942 (+1902271us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:01:56.754467 (+12258525us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:49.948693 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:49.949379 (+   686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:49.949417 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:01:49.949437 (+    20us) tablet.cc:662] Decoding operations
0709 13:01:49.950589 (+  1152us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:49.950673 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:49.950707 (+    34us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:01:50.062303 (+111596us) tablet.cc:701] Row locks acquired
0709 13:01:50.062330 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:01:50.062572 (+   242us) write_op.cc:270] Start()
0709 13:01:50.062750 (+   178us) write_op.cc:276] Timestamp: P: 1783602110062496 usec, L: 0
0709 13:01:50.062777 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:01:50.063857 (+  1080us) log.cc:844] Serialized 4396 byte log entry
0709 13:01:51.750702 (+1686845us) op_driver.cc:464] REPLICATION: finished
0709 13:01:52.178898 (+428196us) write_op.cc:301] APPLY: starting
0709 13:01:52.179002 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:52.179475 (+   473us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:52.179618 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:52.341390 (+161772us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:52.341728 (+   338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:01:52.341793 (+    65us) write_op.cc:312] APPLY: finished
0709 13:01:52.342210 (+   417us) log.cc:844] Serialized 83 byte log entry
0709 13:01:52.342415 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:52.342585 (+   170us) write_op.cc:454] Released schema lock
0709 13:01:52.342649 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:52.343266 (+   617us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:52.343307 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":821396,"reported_tablets":21,"spinlock_wait_cycles":50678912,"child_traces":[["op",{"apply.queue_time_us":421323,"apply.run_cpu_time_us":5855,"apply.run_wall_time_us":164352,"num_ops":11,"prepare.queue_time_us":357750,"prepare.run_cpu_time_us":15765,"prepare.run_wall_time_us":656810,"raft.queue_time_us":944000,"raft.run_cpu_time_us":38831,"raft.run_wall_time_us":329758,"replication_time_us":1643528,"thread_start_us":2080555,"threads_started":4,"wal-append.queue_time_us":561304,"wal-append.run_cpu_time_us":33361,"wal-append.run_wall_time_us":2709763}]]}
I20260709 13:01:56.850775  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.851969  1716 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.852911  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.853642  1724 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:56.855027  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.858361  4098 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.861686  1644 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:01:56.859510  4181 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6.
I20260709 13:01:56.860939  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:56.865592  1645 raft_consensus.cc:2368] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:01:56.828904  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:56.863267  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.867596  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:56.868417  1569 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:56.868903  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:56.873164  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48657.1us
I20260709 13:01:56.868506  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.883435  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:56.885836  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.893226  1721 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:56.894315  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.893452  4060 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:56.924489  4232 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:56.927098  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.748s (SENT)
I20260709 13:01:56.927472  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:56.934661  1571 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:56.936578  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:56.941061  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:56.945127  1716 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:56.953531  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.955729  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.961126  1570 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.971204  1718 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:56.971730  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65335us
I20260709 13:01:56.954645  1571 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:01:56.979596  1724 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:01:56.973028  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:56.983661  1722 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:56.989591  4305 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:56.990072  1645 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:56.990352  4305 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:56.991207  4295 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:56.975004  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 69296.4us
I20260709 13:01:56.964432  1575 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:56.972141  1572 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
W20260709 13:01:56.995672  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 165547us
I20260709 13:01:56.973582  4290 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:57.003285  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.940802  4268 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:01:57.010074  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.903486  4178 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:01:57.014248  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:56.970429  3688 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:56.975721  3972 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.007843  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:57.027218  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1836) took 2415 ms. Trace:
I20260709 13:01:57.027822  1576 rpcz_store.cc:276] 0709 13:01:54.512839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.096240 (+583401us) service_pool.cc:224] Handling call
0709 13:01:56.587846 (+1491606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.608024 (+ 20178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.608065 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:56.608139 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:56.608190 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.608255 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.928209 (+319954us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540030a050. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:01:56.928370 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3271436672}
I20260709 13:01:57.038882  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:57.045213  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:57.049074  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 56726.1us
I20260709 13:01:57.051908  1724 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:57.048867  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:56.991900  4305 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:56.975591  4079 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:57.059558  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.062561  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:01:57.062742  1574 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:57.063521  1575 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:57.028754  4290 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:57.056780  1572 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:57.065389  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.065389  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.056689  1651 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
W20260709 13:01:57.073395  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 146332us
I20260709 13:01:57.003062  4238 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:01:57.081163  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:57.082435  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.085354  4197 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:57.090550  4312 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:57.073586  4301 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 13:01:57.099051  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.100792  4313 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:57.100705  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.102768  1567 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:01:57.060555  3972 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:57.103449  4306 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.248s	user 0.000s	sys 0.003s
W20260709 13:01:57.103637  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74667.8us
I20260709 13:01:57.105139  4079 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:57.106426  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.106508  1646 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:57.107280  1567 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:57.113231   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:57.113540  4305 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:57.117120  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.116669  4290 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.081555  1576 raft_consensus.cc:2376] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 5. Re-sending same reply.
I20260709 13:01:57.118844  4098 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:57.119657  1576 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:57.121523  1572 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:57.072257  1575 raft_consensus.cc:2376] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 6. Re-sending same reply.
I20260709 13:01:57.101881  4313 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:57.139168  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58153.9us
I20260709 13:01:57.167256  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.076694  3837 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.168418  4306 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.064s	user 0.004s	sys 0.001s
I20260709 13:01:57.199641  4232 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.204392  4189 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:01:57.153998  4304 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:57.153446  4191 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.210184  1575 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:57.186055  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.182518  4261 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:01:57.234586  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.220419  3837 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:57.259322  1571 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:57.261323  3837 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.236562  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.264695  1571 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:57.218194  4304 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:57.239096  1651 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:57.268126  4304 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:57.268612  1571 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:01:57.215600  4232 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:57.273255  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.275779  1570 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:57.256692  4313 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.269042  1571 pending_rounds.cc:85] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 3
I20260709 13:01:57.279727  4315 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:57.282099  4315 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:57.282414  4064 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:57.283123  4064 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:57.279650  1571 pending_rounds.cc:107] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:01:57.283877  1571 raft_consensus.cc:2889] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:57.283978  4313 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.284556  4120 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:57.285900  4120 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:57.285029  4191 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.288923  4302 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.443s	user 0.003s	sys 0.003s
I20260709 13:01:57.289261  4191 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.293152  1576 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:57.246511  1722 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:57.299163  1574 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:01:57.301246  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 220643us
I20260709 13:01:57.301010  1569 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:57.271201  4232 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:57.300868  4306 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:01:57.342294  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:57.287400  4120 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.355376  4120 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:57.360781  1473 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:01:57.363329  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.363253  1473 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 4 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:57.369041  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 361396us
I20260709 13:01:57.370689  3972 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.371470  4215 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:01:57.372895  1568 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:57.280089  4159 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:01:57.371500  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:57.385841  4238 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.397156  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54958.7us
I20260709 13:01:57.398335  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:57.399457  4191 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:57.400594  4197 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:57.400851  4245 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:57.401237  4197 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:57.395632  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.323770  4204 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:01:57.420964  4313 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:57.426033  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46317.6us
I20260709 13:01:57.426862  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:57.284308  4064 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.400625  4295 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:57.402012  4159 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.401505  4245 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:57.401849  4141 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 13:01:57.499291  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.438998  4268 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.463378  1473 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:57.443958  4295 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:57.402915  4238 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.511966  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:57.492441  4178 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:57.518828  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.538s (SENT)
I20260709 13:01:57.521042  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:57.518384  1723 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:57.522725  1473 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:57.532485  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.533210  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.533861  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.533568  4304 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:57.534786  1690 connection.cc:256] RPC call timeout handler was delayed by -1.9364142910000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
W20260709 13:01:57.535246  1690 connection.cc:256] RPC call timeout handler was delayed by -1.6957612220000442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 13:01:57.535579  1690 connection.cc:256] RPC call timeout handler was delayed by -1.57782652100002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2666s for a response
I20260709 13:01:57.517917  4268 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.461172  4159 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.524927  1473 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 4 to 5, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:57.539810  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:57.553352  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 181982us
W20260709 13:01:57.536051  1690 connection.cc:256] RPC call timeout handler was delayed by -1.536461222000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 13:01:57.559918  1690 connection.cc:256] RPC call timeout handler was delayed by -1.2628612220000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
I20260709 13:01:57.536782  4178 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:57.566319  4178 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:57.536281  4304 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:57.549954  4319 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:01:57.573642  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134104us
I20260709 13:01:57.545862  1473 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:57.554679  4064 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.549260  4315 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:57.581907  1718 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:57.595820  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:57.596953  1651 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:57.601804  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:57.601800  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:57.541858  4159 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.543788  4120 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:57.603427  1651 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:57.537150  4215 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:01:57.624914  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49181.9us
I20260709 13:01:57.540304  4268 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.657032  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:57.659600  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:01:57.662868  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49609.5us
W20260709 13:01:57.666512  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.667304  1647 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:57.659791  1473 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 5 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:01:57.658257  4296 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:01:57.677317  1719 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:01:57.537514  4245 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:57.681439  1723 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:57.695050  1473 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:57.537047  4295 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:57.673084  4297 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:01:57.708405  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 208283us
W20260709 13:01:57.689393  4305 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.006s
I20260709 13:01:57.700677  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:57.562918  4197 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:57.741822  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 167638us
W20260709 13:01:57.721863  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.682605  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:57.660147  4120 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:57.716149  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:57.673442  4321 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:57.753540  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74351.9us
I20260709 13:01:57.759097  1473 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:57.752998  4295 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:57.751137  4317 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:57.779696  4331 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:01:57.781996  4327 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:01:57.756569  4311 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:57.783015  4332 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:01:57.758410  4159 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:57.800823  1473 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:01:57.813012  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 131180us
W20260709 13:01:57.848973  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 168369us
I20260709 13:01:57.856431  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:01:57.850991  4329 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:01:57.849700  4159 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.784538  4335 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Committing config change with OpId 3.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:57.874245  4341 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.850405  4305 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:01:57.877637  3972 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:57.921196  3972 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:57.918375  1646 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:57.923875  3972 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.903712  4321 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:01:57.874945  4191 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:01:57.932101  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:57.933393  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:57.933807  1722 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:01:57.874544  4060 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:01:57.935509  1721 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:57.934213  4341 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:57.946887  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:57.892594  4333 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:01:57.956318  4341 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.957836  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:57.966394  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.967314  1575 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:57.972968  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:57.973768  1570 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:01:57.893318  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:57.877653  1473 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 3, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:57.975026  1567 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:57.888962  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.913053  4120 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:57.902359  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:01:57.948403  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89664.6us
I20260709 13:01:57.976292  1651 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:01:57.978226  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:57.942152  4305 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:01:57.981738  4204 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:57.992195  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:57.878968  4159 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:57.940249  4311 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:58.012086  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.062s (SENT)
I20260709 13:01:58.021318  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:57.981719  4305 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:57.911816  4238 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:58.028080  4060 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:01:58.003505  4120 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:58.089002  4120 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.090086  4120 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.004401  4064 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:01:58.029304  4336 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:58.093438  4336 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.094466  4336 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.097949  4290 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:58.035729  4305 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:58.100801  4305 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.131459  1473 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 3, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.148792  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:58.161087  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.165871  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 153838us
I20260709 13:01:58.162976  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:58.120577  4268 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.143049  4305 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.179648  4120 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:58.211990  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.216454  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67809.1us
I20260709 13:01:58.237442  4120 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:58.238801  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 335927us
I20260709 13:01:58.246937  1473 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 2, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.229127  3861 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:58.234552  3837 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:58.268693  1567 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:01:58.266986  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 390871us
I20260709 13:01:58.285485  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:58.286542  1646 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.290652  1473 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 3, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:01:58.294854  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.266247  4182 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.234174  4268 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:58.276794  3861 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.303647  1473 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 5, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.277292  3837 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:58.323261  1473 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 4, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:01:58.332978  1646 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:58.325939  1473 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 4, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.336390  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:01:58.335429  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 431304us
I20260709 13:01:58.317734  3861 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.322729  3837 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.368762  1473 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 4, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:58.398217  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 149483us
I20260709 13:01:58.346937  4295 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:58.381110  4333 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:01:58.405179  1473 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 4, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:58.411954  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 178429us
W20260709 13:01:58.419082  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.227s (SENT)
I20260709 13:01:58.363384  4182 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.423671  1473 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.431901  4361 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:58.432835  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:58.433300  4361 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:58.433559  1716 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.433765  4361 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:58.434865  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.436864  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.438012  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.439518  4352 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:58.441483  4191 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:58.441992  4191 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:58.442436  4191 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.431439  1473 catalog_manager.cc:5697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 4, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.445753  4268 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:58.445622  1473 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.409590  4295 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:58.447726  1473 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.454851  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:58.457181  4295 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:58.462617  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:58.462584  1722 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:58.470486  1651 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:01:58.467511  1473 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 4 to 3, leader changed from <none> to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 3 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:01:58.471946  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:58.462899  4159 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:58.460204  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 115129us
I20260709 13:01:58.472510  1473 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:01:58.485026  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66610.2us
I20260709 13:01:58.435832  4336 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:01:58.486430  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 161086us
I20260709 13:01:58.498051  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:58.477195  4060 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:58.504734  1473 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 4, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:58.525408  1473 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 1 to 5, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.504719  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:58.506810  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0195240310000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 13:01:58.486842  4268 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:58.530762  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:58.531109  4268 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.531690  4268 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:58.533595  4361 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.535285  4361 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.536952  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:58.524694  4060 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:58.539520  4060 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.509258  4336 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:58.548233  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.764s (SENT)
I20260709 13:01:58.548895  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:58.549949  1473 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 3, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.487155  4357 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:58.533654  4346 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
W20260709 13:01:58.576436  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71825.1us
W20260709 13:01:58.575551  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 83747.2us
I20260709 13:01:58.543869  3837 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.549069  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:01:58.586874  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 148986us
I20260709 13:01:58.565725  4357 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:58.571985  4315 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.614945  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:58.615967  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:01:58.595517  4335 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.005s
I20260709 13:01:58.562486  4350 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:01:58.588598  4141 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:58.562999  4352 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:01:58.609032  1569 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:58.634378  1569 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:58.640456  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:58.641645  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.632920  1573 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:58.643270  1573 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:01:58.646389  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 114110us
I20260709 13:01:58.647787  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:01:58.609851  4313 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:58.609236  4290 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:58.617024  4357 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.669332  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:58.673795  1567 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:01:58.677258  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60466.6us
I20260709 13:01:58.678668  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:58.669229  1569 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.685189  4371 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:58.692803  1575 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:58.658133  4313 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:58.681269  4189 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.680099  4182 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:58.687423  1573 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.671165  4174 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.680315  4159 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:58.715488  1569 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.689531  4141 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:58.727104  1573 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:58.677958  4290 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:01:58.717295  4358 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:01:58.708097  4371 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:01:58.716923  4316 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:58.725796  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 101308us
W20260709 13:01:58.769305  4347 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:01:58.771144  4360 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:01:58.764672  4368 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:58.777926  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.777747  4361 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:58.780213  4361 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:58.780715  4361 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:58.780910  4290 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:58.781188  4361 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.782079  4313 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:58.782999  3861 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.784997  4371 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.785959  3861 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:58.725248  3782 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.778119  4369 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:58.780495  4352 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:58.810144  4348 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.352s	user 0.005s	sys 0.000s
I20260709 13:01:58.815482  4321 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:01:58.800096  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.186s (SENT)
I20260709 13:01:58.816972  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:58.781219  3837 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:58.744318  4189 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.804560  3782 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.806671  4268 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:58.814033   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:58.793838  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.827570  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:58.866608  1719 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.835140  1567 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:01:58.873083  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79461.9us
I20260709 13:01:58.765954  4316 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:58.875737  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75678.8us
I20260709 13:01:58.721692  4174 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.875522  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:58.800779  4369 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:01:58.954867  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.037029  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 150042us
I20260709 13:01:58.887249  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:58.886751  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:59.038331  1646 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:01:59.030521  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 155304us
W20260709 13:01:59.005702  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.816018  4321 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:58.865535  4315 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.898089  4361 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:58.828567  4290 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:59.027004  4191 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.000s
I20260709 13:01:59.085829  1646 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:59.050709  4268 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.059273  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:59.060694  4189 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.060941  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.088603  1723 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:58.892879  4120 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.063411  1722 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:01:59.064580  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 416951us
W20260709 13:01:59.079620  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 205646us
I20260709 13:01:59.047901  3782 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:59.071854  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 144889us
I20260709 13:01:59.083491  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:59.103709  1720 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.065442  4305 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:01:59.109073  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.109365  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:59.113063  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.110226  1647 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:58.955583  4316 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.062247  1724 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:58.975907  4364 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.900558  4354 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:58.900872  4182 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.865724  4060 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.102092  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:59.110625  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:59.087713  4366 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.083911  1719 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:59.132102  4064 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:59.126957  4354 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:59.102859  4120 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.042097  4141 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.117501  4305 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.147316  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:59.148365  1651 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
W20260709 13:01:59.145680  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.140523  1568 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:59.150045  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:59.150857  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.179546  1720 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:59.156585  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.153368  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:59.109598  4356 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:59.132223  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:59.181413  1719 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:59.132452  4182 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.183068  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:01:59.161463  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.185905  1724 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:59.132675  4364 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.132583  4376 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:59.133154  4354 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.163830  4361 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.134117  4357 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.132869  4064 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:59.156802  4321 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.144585  4366 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.086594  4315 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.138648  4120 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.207121  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:59.167441  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.168253  4351 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:59.091001  4191 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.170058  4365 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.171211  4174 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.173219  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.139961  4060 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.198338  1716 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:59.237046  1718 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:01:59.238945  2372 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:59.247617  4064 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.248706  4364 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.249754  4382 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:59.250664  4382 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:01:59.264366  4380 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.099s	user 0.001s	sys 0.001s
I20260709 13:01:59.208225  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:59.269121  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60983.9us
I20260709 13:01:59.269218  1646 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:59.183032  3861 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:59.185025  4356 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.276247  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.225507  4316 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.198184  4159 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:59.197788  4376 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.197790  3782 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.178234  4369 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.200088  1647 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:01:59.270949  1576 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.274402  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.296003  1720 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:59.275163  1721 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:59.194434  4204 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:01:59.223556  4361 raft_consensus.cc:2764] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:59.276665  4189 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:59.205034  4336 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:01:59.291776  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:01:59.292824  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 108313us
W20260709 13:01:59.301152  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 169045us
I20260709 13:01:59.260028  4366 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.282555  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.276156  3861 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.277160  4304 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.306406  1723 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:59.308746  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:01:59.306238  4189 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:59.300617  4311 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 7 is higher.
I20260709 13:01:59.283344  4159 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.281370  4356 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.300701  3782 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.324223  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.282341  4376 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:59.367794  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:59.324184  3861 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.409502  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:59.409617  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:59.419024  1642 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:01:59.408080  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:59.314239  4351 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.334441  4189 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.426910  1649 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:59.369043  4159 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:59.425891  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 117241us
W20260709 13:01:59.422982  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 255662us
I20260709 13:01:59.251968  4382 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.426986  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:59.432965  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.439332  1724 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:59.437817  1644 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:01:59.438727  1645 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.409693  3782 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:59.408063  4357 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.448999  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.334656  4304 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.465009  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:59.453547  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.480774  1721 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:01:59.463733  1646 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.451979  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:59.448114  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:59.456537  1649 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:59.461539  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:59.481894  1567 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:59.487226  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.481945  1570 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:59.478907  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:59.505666  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:59.507076  1572 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:59.509346  1575 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:59.439957  4381 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:59.462630  4174 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.477442  4350 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.517136  4365 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:59.532608  4174 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.533301  4365 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.533753  4365 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:59.535233  4174 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:59.464488  4060 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:59.519855  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.101s (SENT)
I20260709 13:01:59.537366  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:59.527829  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.538651  1570 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:01:59.449954  4182 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.538278  4064 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.459420  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:01:59.544304  1724 raft_consensus.cc:606] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:01:59.545150  1724 raft_consensus.cc:614] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:59.530586  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 81754.9us
I20260709 13:01:59.551422  1647 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:59.553635  1567 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:01:59.442065  4390 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.482905  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:59.524128  4336 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:59.575136  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:01:59.576023  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.211s (SENT)
I20260709 13:01:59.576699  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:59.584004  4064 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:59.584924  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:01:59.585027  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 260922us
I20260709 13:01:59.586009  2376 raft_consensus.cc:2368] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:01:59.572255  4316 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:59.604589  1688 leader_election.cc:400] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
W20260709 13:01:59.607082  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.615675  1688 leader_election.cc:403] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:01:59.617153  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 118019us
I20260709 13:01:59.624286  1575 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:59.627534  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:01:59.627346  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.628512  1649 raft_consensus.cc:2368] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:01:59.630944  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:59.588714  4191 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.588982  4336 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:59.632867  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:01:59.633713  1724 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:01:59.636286  4394 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.636698  1648 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.595307  4315 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:01:59.638366  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77432.5us
I20260709 13:01:59.641280  1642 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:59.633780  1567 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:59.646526  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:59.621981  4381 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:59.651219  1574 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:01:59.650449  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:01:59.654631  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 50418.1us
I20260709 13:01:59.570853  4390 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.657886  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.659260  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:59.670190  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:59.679237  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:59.681406  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:01:59.679795  1688 leader_election.cc:400] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:01:59.667872  4393 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.684964  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.687614  4351 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.692312  4381 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:59.693035  1724 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:59.707487  1575 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:59.690007  4376 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.694648  4382 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.677783  4316 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.698405  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:01:59.701217  4389 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:01:59.701768  4370 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:59.663987  4401 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.664866  4350 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:01:59.679280  4400 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.728010  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:01:59.681852  2372 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:01:59.731191  1716 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:01:59.728327  4401 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.725805  4304 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.740828  4204 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:59.746028  4397 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:59.746690  1723 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:01:59.723318  4141 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:59.747634  4398 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:01:59.752816  1645 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:01:59.730748  4354 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.001s
I20260709 13:01:59.761468  4204 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:01:59.727695  4381 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:59.775424  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.018s (SENT)
I20260709 13:01:59.720188  4351 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:01:59.732343  4370 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:59.716495  4393 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.779573  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:59.778553  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:59.790915  4204 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.791553  4387 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:59.792320  4387 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:01:59.793648  4369 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.793378  4387 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:01:59.794937  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:01:59.795645  1645 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:01:59.798306  4197 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:59.798832  4197 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:01:59.799291  4197 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:59.790254  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 109626us
I20260709 13:01:59.720883  4336 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.806068  1646 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:59.812629  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:01:59.810662  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:01:59.746057  4400 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.805303  4351 consensus_queue.cc:1243] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:59.827065  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.788416  4401 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:59.849601  4401 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:59.849671  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.850191  4401 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:59.850631  4401 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:59.854875  4400 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:01:59.855638  4400 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:59.789461  4356 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.858740  4356 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.859747  4317 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:01:59.862954  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52413.1us
I20260709 13:01:59.865574  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:01:59.866149  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.044s (SENT) [suppressed 23 similar messages]
I20260709 13:01:59.775656  4354 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.830811  4393 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:01:59.786244  4382 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:01:59.785712  4315 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:01:59.831851  4370 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.833353  4409 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:59.833866  4386 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:59.834707  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:01:59.835067  4390 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:01:59.836382  4376 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:01:59.839107  4407 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:01:59.844899  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:01:59.870373  1567 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:01:59.858781  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1910) took 1264 ms. Trace:
I20260709 13:01:59.852115  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:01:59.856047  4400 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:01:59.902182  1570 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:59.865607  4313 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.911139  1576 rpcz_store.cc:276] 0709 13:01:58.594079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.594256 (+   177us) service_pool.cc:224] Handling call
0709 13:01:58.594605 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.294912 (+700307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.858238 (+563326us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54002a71d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:01:59.858567 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554647296}
I20260709 13:01:59.882755  4182 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:59.888789  4351 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:59.894690  4409 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:01:59.895306  4386 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:59.952242  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 117676us
W20260709 13:01:59.952955  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100923us
I20260709 13:01:59.860276  4365 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.941826  2376 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:01:59.964890  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:01:59.887039  4189 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:01:59.914561  4311 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:59.946702  3861 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.513s	user 0.014s	sys 0.000s
I20260709 13:01:59.943463  4182 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.915208  4400 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:59.973634  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 161377us
I20260709 13:01:59.981096  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:00.001200  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:00.002140  1569 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:59.961730  4384 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:02:00.007704  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 141583us
W20260709 13:02:00.008898  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47870.8us
I20260709 13:02:00.015050  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1912) took 1382 ms. Trace:
I20260709 13:01:59.961818  4386 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.023890  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:00.028921  1576 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:02:00.030267  1651 raft_consensus.cc:2368] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7.
I20260709 13:02:00.024611  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:00.031971  1572 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:01:59.962024  4393 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:00.015753  1568 rpcz_store.cc:276] 0709 13:01:58.632619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.633029 (+   410us) service_pool.cc:224] Handling call
0709 13:01:58.633406 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.151067 (+517661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.014642 (+863575us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b5400420ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:00.014951 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165810048}
I20260709 13:01:59.962289  4351 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:00.033776  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:00.039330  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:00.062880  1650 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.012960  4369 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.006487  4182 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.070930  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:00.001204  4390 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.068815  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:00.053809  4159 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.003s
I20260709 13:02:00.055805  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:00.072295  1721 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:00.061360  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:00.085238  1568 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:00.083204  1645 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.086936  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:00.087771  1648 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:00.084444  1644 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:00.063179  4393 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:00.096745  1568 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:00.092322  1569 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:00.003265  3861 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.059113  4350 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.101547  1576 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 6. Ops: []
I20260709 13:02:00.104676  1569 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.113560  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:00.038481  4238 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:02:00.112473  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:00.109525  4341 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.129338  1574 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:00.130262  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:00.082612  4369 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.104153  4215 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:02:00.118702  4315 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:00.094863  4159 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:00.126071  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 66077.5us
I20260709 13:02:00.179126  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:00.180231  1570 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:00.131641  4416 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:02:00.095112  4390 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.093881  4393 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:00.184839  4417 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 13:02:00.168491  4204 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:02:00.168234  4315 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:00.190963  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60837.3us
I20260709 13:02:00.130561  4365 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.114457  4350 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.134220  4189 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:00.220521  1570 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:00.132951  2376 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:02:00.180629  4381 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:00.180289  4354 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:00.247238  1650 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:00.187527  4387 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:00.130733  4419 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:02:00.197208  4315 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.272009  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:00.166568  4341 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.193511  4356 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.211633  4295 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:00.275063  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96702.4us
I20260709 13:02:00.256603  4423 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:00.289351  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:00.302678  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.194348  4351 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.320101  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:00.264161  4381 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:00.318311  4412 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.224s	user 0.000s	sys 0.002s
I20260709 13:02:00.278569  4387 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.313930  4354 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:00.334878  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 117120us
I20260709 13:02:00.292426  4395 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:02:00.310497  4423 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.364904  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:00.329165  4410 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:00.332816  4356 raft_consensus.cc:2764] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:00.292730  4295 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.390161  4159 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:00.337760  3861 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:00.409344  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66576.5us
I20260709 13:02:00.387676  4354 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:00.395216  4395 consensus_queue.cc:1243] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:00.394731  4321 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.392275  4295 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.393759  4410 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:00.586017  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 161691us
W20260709 13:02:00.585882  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 242893us
I20260709 13:02:00.589375  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:00.590694  4356 raft_consensus.cc:2764] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:00.591223  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:00.559322  4311 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:00.470505  4410 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:00.466621  4415 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:02:00.421509  4369 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:00.581357  4399 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.158s	user 0.004s	sys 0.005s
I20260709 13:02:00.584296  4382 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:00.580535  4376 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:00.477157  4390 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:00.465799  4350 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.436100  4395 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:00.477646  4428 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:02:00.637755  4341 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:00.655900  4422 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.306s	user 0.000s	sys 0.002s
I20260709 13:02:00.612118  4295 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:02:00.647346  4369 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.424257  4435 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.226s
I20260709 13:02:00.650043  4382 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:00.680910  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.697754  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:00.699967  1719 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:00.704777  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:00.676465  4368 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:00.658802  4376 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.701169  4305 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.653848  4354 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.660744  4447 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:02:00.739648  1719 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:00.736421  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:00.708088  4423 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.700248  4369 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.665540  4395 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.758081  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:00.762928  4434 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
I20260709 13:02:00.768646  4215 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:00.763602  4419 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:02:00.769331  4215 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.769742  4215 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.772589  4376 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.764855  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:00.775840  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.768667  4438 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:00.714805  4368 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.790658  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:00.776381  4356 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:00.793639  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:00.809798  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51816.3us
I20260709 13:02:00.809890  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1198) took 1380 ms. Trace:
I20260709 13:02:00.749835  4382 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.808524  4451 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:00.814215  1647 rpcz_store.cc:276] 0709 13:01:59.429935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.430131 (+   196us) service_pool.cc:224] Handling call
0709 13:01:59.430670 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.552462 (+121792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.809513 (+1257051us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54000ba254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:00.809792 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277272832}
I20260709 13:02:00.786810  4387 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:00.819682  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:00.807015  4356 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:00.809355  4435 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:00.815097  4352 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:00.809468  4448 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:00.856815  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92099.3us
W20260709 13:02:00.842916  4400 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.004s
I20260709 13:02:00.860548  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:00.840030  4379 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:00.842440  4447 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:00.846562  4356 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.853766  4435 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:00.859879  4448 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.856572  4352 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:00.880126  4429 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:00.880288  4448 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:00.881038  4429 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.881125  4352 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:00.867453  4401 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.016s	user 0.016s	sys 0.003s
I20260709 13:02:00.882650  4401 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.883633  4429 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.885110  4401 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.865382  4382 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:00.875530  4313 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.944s	user 0.015s	sys 0.003s
W20260709 13:02:00.891090  4197 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.091s	user 0.015s	sys 0.002s
I20260709 13:02:00.866060  4400 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.901196  1721 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:00.902201  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:00.887127  4215 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.906695  4457 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.907891  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:00.909286  4457 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:00.909885  4457 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:00.894946  4447 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.841262  4451 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.919221  4435 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.918951  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:00.926618  4381 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.927268  1539 leader_election.cc:400] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:00.927657  1539 leader_election.cc:403] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:00.904945  4393 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.928983  4393 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.919346  4215 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:00.910863  4457 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.903996  4197 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:00.981973  4400 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:00.978802  4197 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:00.983248  4321 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.547s	user 0.014s	sys 0.002s
I20260709 13:02:00.995756  4356 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:00.935318  4369 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:01.017732  4466 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:01.018020  4451 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:00.970070  4182 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.007s
I20260709 13:02:01.019639  3972 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:00.972026  4381 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:01.021168  4386 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.967s	user 0.015s	sys 0.002s
W20260709 13:02:01.036356  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 109237us
I20260709 13:02:00.966835  4447 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.038506  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:01.041018  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:01.003985  4443 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:02:01.042038  1719 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
W20260709 13:02:01.030208  4459 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:02:01.035949  4444 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.330s	user 0.000s	sys 0.003s
W20260709 13:02:01.044940  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 281) took 1873 ms (client timeout 2429 ms). Trace:
I20260709 13:02:01.045684  1718 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:01.014704  4395 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:01.044746  4369 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.064682  4369 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.067494  4369 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:01.069849  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1944) took 1012 ms. Trace:
I20260709 13:02:01.045713  1722 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:01.070407  1569 rpcz_store.cc:276] 0709 13:02:00.057516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.111682 (+ 54166us) service_pool.cc:224] Handling call
0709 13:02:00.111995 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.112229 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:02:00.112253 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.122570 (+ 10317us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.487573 (+365003us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:00.487639 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.487750 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.035107 (+547357us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:01.069401 (+ 34294us) raft_consensus.cc:1685] finished
0709 13:02:01.069431 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.069763 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5783808,"thread_start_us":277727,"threads_started":1,"wal-append.queue_time_us":468372}
I20260709 13:02:01.052553  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:01.022135  4393 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:01.072680  1719 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:01.080919  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:01.081820  1642 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.083005  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 1935) took 1437 ms. Trace:
I20260709 13:02:01.083496  1567 rpcz_store.cc:276] 0709 13:01:59.645969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.646303 (+   334us) service_pool.cc:224] Handling call
0709 13:02:01.082788 (+1436485us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54001aaed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:01.082915 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458018048}
W20260709 13:02:01.052698  1723 rpcz_store.cc:269] 0709 13:01:59.171619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.171794 (+   175us) service_pool.cc:224] Handling call
0709 13:02:01.044765 (+1872971us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54001a6dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:01.044880 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279732608}
I20260709 13:02:01.053751  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:01.053455  4447 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:01.056420  4315 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.768s	user 0.016s	sys 0.001s
I20260709 13:02:01.038605  3972 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.040132  4182 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.099099  4182 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:01.090854  4393 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.100271  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.031025  4420 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:01.057756  4413 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.974s	user 0.003s	sys 0.001s
I20260709 13:02:01.101804  4420 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.100811  4393 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.102963  4420 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.104045  4420 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.040915  4386 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.113761  4447 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:01.115234  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.112687  4315 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.135196  4447 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.114481  4462 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:01.118633  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:01.136291  4386 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:01.136866  4315 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:01.119881  4395 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:01.139293  2372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 433) took 1727 ms (client timeout 1599 ms). Trace:
I20260709 13:02:01.140713  4395 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:01.140801  2372 rpcz_store.cc:269] 0709 13:01:59.412135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.418810 (+  6675us) service_pool.cc:224] Handling call
0709 13:02:01.139054 (+1720244us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54005d25d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:01.139200 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253006592}
I20260709 13:02:01.143536  4395 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:01.147233  1642 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
W20260709 13:02:01.149817  4433 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.714s	user 0.003s	sys 0.000s
I20260709 13:02:01.150619  4141 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:01.151288  4141 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.152081  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1256) took 1170 ms. Trace:
I20260709 13:02:01.152791  1576 rpcz_store.cc:276] 0709 13:01:59.981951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.104841 (+122890us) service_pool.cc:224] Handling call
0709 13:02:00.105154 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.125829 (+ 20675us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:00.125862 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.145703 (+ 19841us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.289037 (+143334us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:00.289087 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.289170 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.148791 (+859621us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:01.151739 (+  2948us) raft_consensus.cc:1685] finished
0709 13:02:01.151764 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.152011 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":943,"raft.queue_time_us":6444,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":17985,"spinlock_wait_cycles":16685440,"thread_start_us":78840,"threads_started":1,"wal-append.queue_time_us":200378}
I20260709 13:02:01.121065  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.040180  4381 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.137567  1650 raft_consensus.cc:606] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:01.159219  4381 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:01.159155  4141 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.160964  4141 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:01.161139  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1913) took 1887 ms (client timeout 1874 ms). Trace:
W20260709 13:02:01.161836  1716 rpcz_store.cc:269] 0709 13:01:59.273320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.273445 (+   125us) service_pool.cc:224] Handling call
0709 13:02:01.160903 (+1887458us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54001a77d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:01.161042 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":960776960}
I20260709 13:02:01.164783  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:01.165223  4410 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.007s
I20260709 13:02:01.165577  1649 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.165833  4410 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.093613  4369 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.105261  4393 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:01.196839  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.105026  3972 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.136623  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:01.173384  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:01.192859  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.201022  1572 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:01.024159  4448 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.197212  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:01.137246  4462 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:01.198046  1576 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.203501  1567 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:01.202453  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:01.204905  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 75775.4us
I20260709 13:02:01.203727  1642 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:01.162005  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:01.207746  1572 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:02:01.209012  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47104.3us
I20260709 13:02:01.209129  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.212713  1572 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:01.214941  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:01.216274  1649 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:01.218312  1573 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:01.219877  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:01.220043  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.220711  1716 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.220775  1570 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:01.208031  4356 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:01.233827  4356 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.236478  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:01.239349  4410 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:01.240260  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:01.207692  4393 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:01.241080  1718 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:01.241487  1720 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:01.244717  1720 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:01.204730  4462 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.210112  4442 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.244251  1724 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:01.210076  4466 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:01.245379  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:01.243129  1570 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:01.204492  4457 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.217712  3972 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.265580  1649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:01.201532  4369 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.266762  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:01.264737  1716 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:01.246604  1720 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:01.270732  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 55865.8us
I20260709 13:02:01.269450  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:01.267215  1649 raft_consensus.cc:606] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:01.278632  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:01.279474  1717 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.286258  1720 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.288204  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:01.294708  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.297266  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:01.302444  4409 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.304557  4442 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:01.307062  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:01.310302  1717 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:01.270741  4386 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:01.264082  4393 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:01.325362  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:01.327029  1720 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:01.274788  4466 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:01.304167  4409 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.307999  4215 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:01.307682  4197 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:01.287689  4437 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:02:01.321928  4442 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:01.337985  4469 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:01.337016  4386 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:01.372838  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:01.342468  4381 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:01.374058  2377 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:01.382221  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:01.357322  4366 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:02:01.389931  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:01.390749  4379 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.391513  4464 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:02:01.391323  4379 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.392490  4466 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.395054  4379 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:01.338320  4182 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:02:01.401997  4182 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:01.402637  4182 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.404009  4182 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:01.408710  4316 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:02:01.422302  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:01.358193  4215 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.423609  2378 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.415542  4368 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:01.425840  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43714.7us
I20260709 13:02:01.356842  4469 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:01.431821  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:01.439639  1718 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:01.442807  1576 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:01.446074  1575 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:01.465394  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:01.471685  1573 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:01.472700  1646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:01.473664  1646 raft_consensus.cc:606] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:01.474465  1646 raft_consensus.cc:614] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:01.476727  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 45001.8us
I20260709 13:02:01.436022  3861 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:01.480883  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.482149  1650 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:01.482553  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.488371  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:01.440716  4469 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.446332  4182 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.506614  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.509541  1645 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:01.511693  1643 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:01.424999  4381 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.515856  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.507409  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:01.517015  1649 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:01.517153  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.471268  4356 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.526647  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.527611  1646 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:01.424454  4215 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.534988  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.535068  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:01.508091  4448 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:01.551287  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 73438.8us
W20260709 13:02:01.541673  1651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1189) took 2250 ms (client timeout 1649 ms). Trace:
I20260709 13:02:01.556988  1649 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:01.557479  1651 rpcz_store.cc:269] 0709 13:01:59.291232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.291387 (+   155us) service_pool.cc:224] Handling call
0709 13:02:01.541340 (+2249953us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540005d2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:01.541564 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661890304}
I20260709 13:02:01.512950  3861 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.550175  1646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:01.561084  1646 raft_consensus.cc:606] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:01.533685  4460 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.055s	user 0.004s	sys 0.000s
I20260709 13:02:01.562373  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.562639  1650 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:02:01.530643  4447 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:01.287529  4395 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:02:01.565321  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.573590  1614 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.018s	sys 0.025s
I20260709 13:02:01.476881  4393 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:01.574165  1614 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.018s	sys 0.026s
W20260709 13:02:01.592743  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 557069us
I20260709 13:02:01.594198  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.597110  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:01.598096  1718 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:01.599057  1721 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.602134  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:01.594231   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:01.609916  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.612099  1722 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:01.516038  4386 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.572614  4369 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:01.623090  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 70330.2us
I20260709 13:02:01.620668  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:01.624274  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.628173  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:01.573906  4457 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:01.634321  4475 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:02:01.637230  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.520s (SENT)
I20260709 13:02:01.639919  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:01.638227  1721 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:01.630120  1575 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:01.641587  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.640321  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:01.648262  2344 connection.cc:256] RPC call timeout handler was delayed by -1.209353048999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 13:02:01.644013  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:01.652781  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.653306  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.651712  1718 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:01.657022  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.652421  3837 raft_consensus.cc:2764] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:02:01.670708  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.673182  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.677057  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.678054  1575 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 3.
I20260709 13:02:01.645524  4442 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.650951  4238 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:01.682512  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.651722  4457 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.687609  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.703718  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.705753  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.708108  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.709771  1569 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:01.710453  1575 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:01.674088  4356 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.711287  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.711562  1646 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:01.712106  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.714037  1569 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:01.712880  1567 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:01.717413  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.719986  1567 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:01.724918  1567 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:01.718356  1571 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:01.734894  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:01.736773  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:01.737914  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.705614  4363 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:01.730731  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:01.741888  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.743448  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:01.681560  3861 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.691661  4245 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:01.695868  4238 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:01.757920  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.746250  1571 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:01.744925  4388 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.781863  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.741250  4405 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:02:01.771101  2378 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
W20260709 13:02:01.793111  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 153285us
I20260709 13:02:01.797111  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:01.802475  1571 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:01.802718  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.804219  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:01.806679  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:01.768788  4424 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:01.754238  4381 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:01.746642  4197 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:01.845093  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:01.846170  2378 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:01.857689  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.836019  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.750510  4363 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:01.746701  4204 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:02:01.864346  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:01.838773  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.805651  4420 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:02:01.869006  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:01.807703  4379 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:02:01.869805  1642 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:01.869899  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.856063  3972 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.856163  3782 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:01.873443  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:01.895416  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:01.874976  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:01.880213  4484 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:02:01.881170  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:01.883652  4159 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.898798  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.909678  1576 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:02:01.899734  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:01.905537  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:01.909025  1567 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:01.862782  4424 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:01.924153  1642 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:01.869266  4197 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:01.920852  4487 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:02:01.922616  4486 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:02:01.944481  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:01.949460  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:01.951215  2380 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:01.953352  4238 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:01.919296  3782 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.962673  1647 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:01.970196  4363 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:01.908865  4311 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:01.973263  4311 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.922150  4159 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:01.918380  4369 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:01.920555  4441 raft_consensus.cc:2764] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:02.012225  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:02.014955  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 66595.2us
W20260709 13:02:02.021989  4493 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:02:02.022239  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:02.022943  1721 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:02:02.024221  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:02.026510  1642 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:02.029486  1645 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:02.036022  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:02.050061  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:02.052069  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:02.054924  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:02.057766  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:02.006325  4462 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.006s
I20260709 13:02:02.063377  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:02.068864  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 271843us
I20260709 13:02:02.070279  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.073487  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:02.073890  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:02.020220  4480 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.338s	user 0.005s	sys 0.006s
I20260709 13:02:02.065546  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:02.080765  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:01.986501  4159 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:02.125696  2380 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:02.132458  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53818.1us
I20260709 13:02:02.132738  1651 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:02.056484  4381 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.065708  4369 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:02.068156  4442 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.067557  4441 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:02.062109  4410 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:01.991838  4469 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:02.160027  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 89884.3us
I20260709 13:02:02.141625  4260 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:02.134063  4466 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:02.171631  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 109084us
I20260709 13:02:02.146360  4393 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:02.136615  3782 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:02.182372  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:02.143203  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:02.145150  4381 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:02.186774  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 240786us
I20260709 13:02:02.082557  4462 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.180940  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:02.175437  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:02.160732  4457 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:02.160851  4442 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.160331  4441 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.155933  4369 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.177646  4393 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:02.192847  4409 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.855s	user 0.013s	sys 0.004s
I20260709 13:02:02.204857  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:02.205626  3782 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:02.205984  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:02.206555  4204 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:02.199717  4238 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:02.213757  4442 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.190863  4381 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.224903  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:02.226295  1575 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:02.149570  4424 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.226455  1724 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:02.190541  4359 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:02.171257  4410 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.229504  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:02.195032  4457 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:02.195372  2376 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:02:02.216245  4441 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:02.187935  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.191s (SENT)
I20260709 13:02:02.220000  4369 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:02.284132  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.222605  4393 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.263043  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:02.271215  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.271508  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.271844  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.177739  4197 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:02.281246  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93351.9us
I20260709 13:02:02.283239  1570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1acf73e529394004842219637cff7cfc"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:44738
I20260709 13:02:02.292488  1570 raft_consensus.cc:606] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Received request to transfer leadership
W20260709 13:02:02.292673  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.876s (SENT)
I20260709 13:02:02.293207  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.295229  1567 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:02.295907  1567 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:02.296587  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 233278us
I20260709 13:02:02.276285  4457 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.283479  2375 raft_consensus.cc:2368] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:02:02.281950  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:02.269363  4359 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:02.308826  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.284554  4439 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:02.315594  1568 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:02.315872  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:02.309526  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:02.310189  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 229580us
I20260709 13:02:02.317934  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.318058  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:02.320619  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.312601  1567 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.324258  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.326583  1567 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:02.327252  1568 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:02.330149  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:02.332468  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44048.6us
I20260709 13:02:02.338083  2380 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:02.337958  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:02.339419  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 143933us
I20260709 13:02:02.315824  4359 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.323104  4505 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:02.344321  4498 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:02.309265  4503 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:02.344025  4409 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:02.326823  4440 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:02.352885  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:02.345283  4506 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:02:02.356714  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.342203  2373 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:02.361212  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.345659  4504 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:02.366777  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.362138  4448 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:02.369119  1568 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:02.380503  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.366650  4503 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:02.397738  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:02.402813  4502 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:02:02.403748  4496 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:02.404990  4469 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:02.416208  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.405778  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.369656  4370 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:02.417925  1570 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:02.408361  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.408742  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.419780  1576 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:02:02.409250  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:02.412866  4499 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:02:02.382824  4390 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:02:02.413637  1539 leader_election.cc:400] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:02:02.414079  2374 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:02.396811  4462 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:02.424283  4462 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:02.425951  4469 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:02.426608  4469 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:02.427362  4245 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:02.427878  4245 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:02.428439  4494 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.234s	user 0.000s	sys 0.002s
I20260709 13:02:02.400547  1567 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:02.427621  4469 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.427982  4311 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.446012  4469 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.445844  4507 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:02:02.447197  4507 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:02.447599  4507 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:02.447819  4311 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:02.417414  1573 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:02.428951  4448 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.450879  1573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1acf73e529394004842219637cff7cfc"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:44738
W20260709 13:02:02.467760  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 106634us
W20260709 13:02:02.466884  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 142581us
I20260709 13:02:02.388810  4510 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
W20260709 13:02:02.476696  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 115954us
I20260709 13:02:02.459429  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:02.526351  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.551545  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.469125  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:02.476969  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:02.598631  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 257929us
I20260709 13:02:02.469556  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:02.605023  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 136068us
I20260709 13:02:02.460323  4369 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.608014  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:02.629688  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 152843us
I20260709 13:02:02.630640  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:02.631517  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:02.472076  4410 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:02.605602  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54182.3us
I20260709 13:02:02.632583  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.600154  4204 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:02.639297  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:02.475387  4245 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.664208  4245 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:02.668579  4410 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.669543  4311 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.670521  4311 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.672696  4204 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.673198  4518 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.674371  4518 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:02.674836  4311 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:02.675786  4204 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:02.677160  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44949.8us
I20260709 13:02:02.726831  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.677729  1574 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:02.674968  4518 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.677820  4410 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:02.627988  4505 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:02.688534  4245 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.475543  4503 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:02:02.700798  4369 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:02.702811  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:02.654179  4510 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:02:02.466023  4507 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.677772  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:02.637491  4363 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:02.746676  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 434) took 1083 ms. Trace:
I20260709 13:02:02.736649  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1949) took 1137 ms. Trace:
W20260709 13:02:02.756808  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.757014  1724 rpcz_store.cc:276] 0709 13:02:01.599767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.599998 (+   231us) service_pool.cc:224] Handling call
0709 13:02:01.600428 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.259361 (+658933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.736180 (+476819us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54001702d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:02.736541 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438775680}
I20260709 13:02:02.749301  1576 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:02.769577  1569 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:02.756876  2377 rpcz_store.cc:276] 0709 13:02:01.649538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.649699 (+   161us) service_pool.cc:224] Handling call
0709 13:02:01.650070 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.802594 (+152524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.802632 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.802676 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:01.802717 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.732045 (+929328us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54006fed54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:02.732137 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.732487 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350231808}
I20260709 13:02:02.627873  4440 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 13:02:02.731645  4518 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.659121  4442 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.734659  4505 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:02.787812  1576 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:02.804850  4505 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.739712  4457 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.789209  4369 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.751468  4245 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.822649  4245 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:02.825685  4440 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.826576  4440 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:02.777294  4421 raft_consensus.cc:993] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:02.828044  4421 raft_consensus.cc:1081] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:02.708118  4381 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.834607  4521 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:02.835656  4521 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:02.838382  4512 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.432s	user 0.002s	sys 0.000s
I20260709 13:02:02.843124  2378 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:02.829950  4511 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.421s	user 0.001s	sys 0.000s
I20260709 13:02:02.789299  4363 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:02.849653  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.852424  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:02.852808  4366 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:02.853530  4366 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:02.854879  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:02.852447  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:02.861640  4370 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:02.862368  4439 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.866098  4439 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.868081  4370 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:02.873315  4492 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.874296  4492 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.875357  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:02.880268  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.884743  4204 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.885458  4204 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.885946  4395 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:02.886914  4395 consensus_queue.cc:1243] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:02:02.890987  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 149432us
I20260709 13:02:02.854801  4366 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.893731  4366 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:02.901014  4141 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.902410  4495 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.667s	user 0.000s	sys 0.012s
I20260709 13:02:02.902148  4141 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:02.905462  4141 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:02.965540  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:02.921324  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:02.907895  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1264) took 1329 ms. Trace:
I20260709 13:02:02.928278  1650 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.010510  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:03.012037  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:03.016443  1569 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.912022  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:02.965615  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.020514  1567 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.009464  1575 rpcz_store.cc:276] 0709 13:02:01.579021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.714159 (+135138us) service_pool.cc:224] Handling call
0709 13:02:01.739300 (+ 25141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.260555 (+521255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.907469 (+646914us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54000c3854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:02.907789 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109552512}
W20260709 13:02:03.016887  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 290184us
W20260709 13:02:03.029848  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64401.4us
I20260709 13:02:03.030504  1574 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:02:03.028429  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:03.046365  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.047664  1650 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:03.059724  1647 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.064049  1646 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.063287  1642 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
W20260709 13:02:03.052207  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 420767us
I20260709 13:02:03.069934  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.070154  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:02.981220  4507 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.927824  4528 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:03.099815  1722 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:03.099783  1651 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.102782  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.104559  1642 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:03.106209  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.113517  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:03.117152  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.018893  4311 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.118149  1721 raft_consensus.cc:2368] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:03.121322  1642 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 13:02:03.122421  4376 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.129932  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:03.135970  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.049270  4509 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:03.122952  4316 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:03.123811  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95596.7us
I20260709 13:02:03.153245  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.153245  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:03.124687  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.125523  4525 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:03.126034  4363 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.126500  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.136394  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:03.163247  1721 raft_consensus.cc:1217] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 5.7->[6.8-6.8]   Dedup: 6.8->[]
I20260709 13:02:03.163146  1716 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:03.163940  1721 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:03.138814  1722 raft_consensus.cc:1217] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:02:03.148034  1569 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:03.043602  4381 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:03.147682  4393 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.846s	user 0.018s	sys 0.010s
I20260709 13:02:03.057370  4440 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:03.155449  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:03.155843  1724 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:02:03.172708  1646 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:03.085608  4517 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:03.079021  4159 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.077322  4462 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.122721  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.149844  4519 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.167121  1722 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:03.148280  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:03.136606  4506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.007s	sys 0.022s
I20260709 13:02:03.126550  4442 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.132866  4365 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.194659  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.195657  1724 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.173841  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:03.156042  4409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.029s	sys 0.020s
I20260709 13:02:03.133213  4457 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.130956  4524 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:03.151064  4523 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:03.160521  4424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.011s	sys 0.034s
I20260709 13:02:03.164811  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.184795  1642 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.186622  1722 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.187659  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.158972  4369 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:03.186648  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:03.197571   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:02:03.181000  4359 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.017s
I20260709 13:02:03.198767  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.155403  4514 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:03.146584  4311 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.175693  4525 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.129024  4528 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.162031  4509 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.155887  4376 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.260788  1646 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.182260  4503 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:03.202211  4409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.030s	sys 0.021s
I20260709 13:02:03.193475  4204 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.265852  1721 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.263633  4457 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:03.187049  4440 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.274474  4525 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.179725  4393 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.204496  4381 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:03.203308  4365 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:03.277521  4441 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.980s	user 0.017s	sys 0.000s
I20260709 13:02:03.278676  4311 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.281672  4492 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.284211  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:03.256983  4510 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:02:03.264792  4359 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:03.256493  4424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.011s	sys 0.034s
W20260709 13:02:03.186331  4506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.007s	sys 0.023s
I20260709 13:02:03.295085  1724 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.287774  4532 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:02:03.345000  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.283840  4352 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.302477  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:03.307359  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:03.308259  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.309749  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.363204  1722 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.162492  4316 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.310616  4409 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.258504  4390 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:03.319401  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.319666  1573 raft_consensus.cc:606] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:03.321616  1642 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.375444  1573 raft_consensus.cc:614] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:03.177623  4517 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.376657  1719 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:03.326817  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.203286  4462 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.329421  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:03.392534  1647 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.392822  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.391889  1642 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:03.393746  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:03.394651  1569 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:03.307610  4503 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.405102  1642 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:03.331183  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 320898us
I20260709 13:02:03.194097  4519 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.318015  4370 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:03.334050  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.339129  1720 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:03.329353  4469 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.883s	user 0.016s	sys 0.001s
I20260709 13:02:03.415339  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:03.369072  4532 consensus_queue.cc:1243] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:03.327199  4507 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.381243  1646 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:03.383595  1722 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:03.408010  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:03.332918  4441 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.416106  1574 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:03.356109  1721 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:03.433702  1690 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:03.408530  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.429066  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:03.419097  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.434007  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.437224  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.437774  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.436513  1567 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:03.440562  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.442559  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:03.443296  1567 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:03.444667  1567 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.444550  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:03.447121  4528 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.447877  4529 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:03.448601  4526 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:03.449209  4526 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.449363  4529 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:03.449775  4529 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.429666  4536 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.448058  4506 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:02:03.456693  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:03.458128  4536 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.458549  4245 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:03.460201  4245 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.431232  4370 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:02:03.450330  4526 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:03.463086  4534 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:02:03.457463  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.464768  1574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44746
I20260709 13:02:03.465581  1574 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:03.466028  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.466753  1721 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:03.466496  4409 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.467792  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.466030  1574 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:03.454504  4518 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.638s	user 0.003s	sys 0.009s
I20260709 13:02:03.473306  4518 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.421806  4311 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:03.464818  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:03.480520  1647 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:03.494246  4370 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:03.494743  4370 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:03.495183  4370 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:03.499050  4530 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:03.393929  4440 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.396054  4390 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.414219  4492 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.426822  4448 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:03.495872  4519 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.488657  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:03.490134  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.520663  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.413270  4503 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.511629  1539 leader_election.cc:400] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:03.428035  4537 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.520849  1567 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:03.546047  1539 leader_election.cc:403] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:03.506008  4462 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.523698  1721 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:02:03.555116  2378 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.507411  1574 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.545076  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:03.506494  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 333508us
I20260709 13:02:03.526475  4509 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.528925  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:03.431012  4507 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.582666  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.551570  4537 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.516363  4311 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.583184  4536 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.557442  4538 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:03.557433  4366 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.663s	user 0.013s	sys 0.007s
I20260709 13:02:03.529512  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:03.517225  4457 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.518677  4448 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:03.543900  4492 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.592093  2379 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:03.602336  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:03.606259  1574 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.589491  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.557695  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.601003  1568 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:02:03.631810  1724 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:03.616575  2378 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:02:03.633224  1724 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.634910  1724 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:03.594920  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:03.596138  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:03.599828  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54801.4us
W20260709 13:02:03.584461  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73071.9us
I20260709 13:02:03.638746  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:03.638747  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:03.631173  1721 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:03.645393  1576 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:03.604220  4311 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.602849  4507 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.653007  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:03.657949  1647 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:03.598053  4455 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.588549  4514 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:03.557530  4462 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.655711  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:03.663668  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:03.664710  1725 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:03.667459  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:03.600503  4539 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:03.671731  1574 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.667492  1721 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:03.643571  4528 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.603065  4537 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.590124  4518 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:03.603960  4536 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:03.668968  1647 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.710083  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:03.707824  1642 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.629971  4457 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.712090  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:03.655454  4141 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:03.604545  4538 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.726517  4530 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.651094  4525 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.722110  1574 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:02:03.654527  4245 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.773128  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:03.803246  1645 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:03.669148  4547 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.779218  1721 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:03.783766  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 116385us
I20260709 13:02:03.788542  1723 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:02:03.727418  4311 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.789439  4518 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.790019  4492 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.764704  4528 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.731081  4462 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:03.793004  4540 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:02:03.793799  4390 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:03.794905  4439 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.928s	user 0.018s	sys 0.000s
I20260709 13:02:03.707868  4514 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.765722  4448 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.807943  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:03.820719  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 108703us
I20260709 13:02:03.798936  4457 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.727511  4507 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.707612  4509 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:03.839649  1642 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:03.841104  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.672538  4409 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:03.760000  4546 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.846082  4518 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.862485  4546 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.844995  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:03.778322  4141 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:03.864773  4141 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.865792  4419 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:03.866318  4419 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.866693  4419 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.866573  4370 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.834836  4530 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.868695  4421 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:03.855893  4381 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.870105  4381 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.870577  4530 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:03.842264  4311 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.873989  4381 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:03.865845  4538 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.875641  4304 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:03.835570  4525 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.877434  3837 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:03.877573  4366 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:02:03.856503  4505 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.051s	user 0.008s	sys 0.007s
I20260709 13:02:03.877912  3837 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:03.880812  4514 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:03.880868  4505 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.881556  3837 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.882396  1716 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:03.882929  3837 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.841842  4245 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:03.851997  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.861156  4528 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:03.886272  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.886276  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 78461.2us
I20260709 13:02:03.820109  4369 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.820567  4537 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.558166  4529 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:03.900036  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.839509  4204 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.904685  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:03.897989  4492 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:03.965378  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 269990us
I20260709 13:02:03.727906  4539 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.896932  4455 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.849182  4536 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:02:03.984951  1721 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:02:03.981755  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 1937) took 2747 ms (client timeout 2350 ms). Trace:
W20260709 13:02:03.985980  1723 rpcz_store.cc:269] 0709 13:02:01.234594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.236115 (+  1521us) service_pool.cc:224] Handling call
0709 13:02:03.981314 (+2745199us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b5400223b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:03.981578 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5866567936}
W20260709 13:02:03.980504  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 75980.4us
I20260709 13:02:03.910509  4448 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.910857  4525 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:03.891582  4507 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:03.909863  4469 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:03.995740  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1289) took 1066 ms. Trace:
I20260709 13:02:04.037304  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:04.038656  2379 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:03.848568  4439 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.901381  4369 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:04.054840  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.979161  4538 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:04.126065  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.019069  1571 rpcz_store.cc:276] 0709 13:02:02.929877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.933471 (+  3594us) service_pool.cc:224] Handling call
0709 13:02:02.933884 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.982271 (+ 48387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.982324 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.025010 (+ 42686us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.413441 (+388431us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:03.413496 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.413574 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.995514 (+581940us) raft_consensus.cc:1685] finished
0709 13:02:03.995536 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.995672 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":851,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":22058,"thread_start_us":296082,"threads_started":1,"wal-append.queue_time_us":488278}
I20260709 13:02:03.906656  4409 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:03.992859  4492 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:03.898183  4509 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.172458  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:04.001210  4311 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.170953  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:04.186446  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.245671  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:04.194933  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.170840  4536 consensus_queue.cc:1243] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:04.247145  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.195154  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:04.246143  1650 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:04.258430  1647 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:04.220366  1724 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:04.195622  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.249068  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:04.196147  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.250765  4538 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:04.252767  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:04.172276  4518 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.254313  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.254371  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:04.258862  1647 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:04.282495  4536 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:04.283443  4536 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:04.283812  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.282058  1647 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.284782  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.288712  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.299351  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.300419  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:04.301435  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.302230  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.302836  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.255080  4546 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.303743  4546 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:04.302980  4553 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
W20260709 13:02:04.305435  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 133134us
I20260709 13:02:04.306931  4547 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:04.311520  4558 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.322680  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1882) took 1189 ms. Trace:
I20260709 13:02:04.323088  1725 rpcz_store.cc:276] 0709 13:02:03.132663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.137410 (+  4747us) service_pool.cc:224] Handling call
0709 13:02:03.137933 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.731318 (+593385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.321355 (+590037us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54001a86d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:04.321615 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210733952}
I20260709 13:02:04.169333  4469 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.247768  4370 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:04.301819  1571 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:04.325309  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.261507  4564 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.284780  4311 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:04.335705  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:04.305437  4536 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:04.344270  4245 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.345940  4462 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:04.346819  4503 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.782s	user 0.011s	sys 0.000s
I20260709 13:02:04.347084  4462 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:04.347359  4503 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.320498  4554 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:02:04.348734  4439 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.349498  4554 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:04.349958  4554 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.349987  4505 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.351310  4554 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.312592  4542 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:04.358136  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:04.328619  4547 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.359721  4311 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.360461  4547 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.361042  4311 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.363034  4393 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.364933  1570 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:04.365366  1570 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.367046  1569 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:04.359762  4536 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.369614  4507 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.369823  4536 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:04.361526  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1883) took 1214 ms (client timeout 1556 ms). Trace:
I20260709 13:02:04.372236  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1970) took 1002 ms. Trace:
I20260709 13:02:04.374384  1571 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:04.318362  4539 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:04.372963  1720 rpcz_store.cc:269] 0709 13:02:03.146993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.154696 (+  7703us) service_pool.cc:224] Handling call
0709 13:02:04.361264 (+1206568us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:04.361454 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323789696}
W20260709 13:02:04.383066  4419 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.000s
I20260709 13:02:04.357090  4492 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.383694  4419 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:04.389858  4141 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:02:04.321455  4509 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:04.393110  3837 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
I20260709 13:02:04.333860  4365 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:04.339870  4409 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:04.377295  1570 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.390635  4419 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.403577  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.409016  1570 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.409934  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.333130  4538 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:02:04.412874  1720 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:04.409551  1571 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:04.412254  4551 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:04.427312  1721 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:02:04.427868  1721 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 2
I20260709 13:02:04.428143  1721 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:02:04.428215  4509 raft_consensus.cc:476] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Not starting pre-election -- already a leader
I20260709 13:02:04.428568  1721 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:04.395447  4542 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:04.395063  4369 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.312311  4558 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.407519  4559 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:04.446621  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.378280  4564 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.392184  4141 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:04.450524  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 114917us
I20260709 13:02:04.418632  4554 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:04.456881  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:04.410321  4546 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.458814  4141 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:04.462837  1725 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:04.464284  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.465155  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.421344  4551 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.468214  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:04.449776  4448 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:04.554616  1570 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:04.555495  1570 pending_rounds.cc:85] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:04.555989  1570 pending_rounds.cc:107] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:04.558363  1570 raft_consensus.cc:2889] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:04.560453  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.564020  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:04.565546  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97446.9us
I20260709 13:02:04.560451  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1297) took 1055 ms. Trace:
I20260709 13:02:04.581404  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.582337  1570 rpcz_store.cc:276] 0709 13:02:03.505342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.364472 (+859130us) service_pool.cc:224] Handling call
0709 13:02:04.364795 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.559793 (+194998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.560308 (+   515us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.468209  4507 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:04.517459  4462 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:04.519757  4448 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.600225  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.600617  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.603459  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.449327  4542 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.601567  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.605396  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.608326  1642 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:04.601274  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.518200  4568 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:04.606560  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.610798  1648 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:04.584903  4410 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:04.582824  4503 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.615309  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.616317  1644 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:04.618347  1720 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:04.620757  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.621794  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.624069  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:04.620965  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.596781  3837 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.628979  3837 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.629768  1648 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:04.632846  4546 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.596944  4141 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.636798  4141 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.594625  4492 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:04.617982  4410 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:04.647253  4141 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.590993  4559 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:02:04.648249  4410 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.613266  4507 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:04.606299  4448 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:04.655028  3837 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.656548  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.661028  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.663969  1644 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:04.612582  4568 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:04.668423  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.700390  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:04.678423  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77558.9us
I20260709 13:02:04.688374  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1897) took 1350 ms. Trace:
I20260709 13:02:04.654680  4492 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:04.665007  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:04.705215  1720 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.705098  1724 rpcz_store.cc:276] 0709 13:02:03.338431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.648022 (+309591us) service_pool.cc:224] Handling call
0709 13:02:03.981585 (+333563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.281193 (+299608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.687110 (+405917us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400223b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:04.688093 (+   983us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400223b50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:04.688265 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313687680}
I20260709 13:02:04.374333  1723 rpcz_store.cc:276] 0709 13:02:03.372669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.991133 (+618464us) service_pool.cc:224] Handling call
0709 13:02:03.992140 (+  1007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.992482 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:04.069375 (+ 76893us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.268466 (+199091us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.358686 (+ 90220us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.371868 (+ 13182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.371921 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:04.371974 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.372042 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.372166 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113148,"prepare.queue_time_us":179169,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":137,"raft.queue_time_us":176399,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":7577,"spinlock_wait_cycles":16154368,"thread_start_us":164038,"threads_started":1,"wal-append.queue_time_us":179508}
I20260709 13:02:04.707672  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:04.690338  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1886) took 1515 ms (client timeout 1975 ms). Trace:
I20260709 13:02:04.709280  1645 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:04.693241  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2007) took 1240 ms. Trace:
W20260709 13:02:04.711541  1716 rpcz_store.cc:269] 0709 13:02:03.175071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.175360 (+   289us) service_pool.cc:224] Handling call
0709 13:02:04.690056 (+1514696us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400214154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:04.690253 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426859904}
I20260709 13:02:04.698268  3837 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:02:04.713848  1539 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.873s (SENT)
I20260709 13:02:04.714324  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:04.714383  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:04.714291  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.716284  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:04.717828  1723 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:04.712255  1576 rpcz_store.cc:276] 0709 13:02:03.453126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.647530 (+194404us) service_pool.cc:224] Handling call
0709 13:02:03.647708 (+   178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.848097 (+200389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.848187 (+    90us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.848254 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:03.848311 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.692818 (+844507us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000c1a54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:04.692891 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.693142 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460160384}
W20260709 13:02:04.682511  4557 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 13:02:04.722887  4528 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:04.672375  4568 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.695446  4316 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.728127  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.731906  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.731108  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.732692  1716 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:04.732841  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.733173  1723 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:04.733572  1724 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:02:04.734140  1720 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:04.734409  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.711179  4492 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.713325  4559 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:04.727277  4554 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:04.653792  4507 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.734619  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.738550  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.744529  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1973) took 1283 ms. Trace:
I20260709 13:02:04.745175  1723 rpcz_store.cc:276] 0709 13:02:03.460690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.743588 (+1282898us) service_pool.cc:224] Handling call
0709 13:02:04.743926 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.744115 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.744140 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.744187 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:04.744237 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.744292 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.744454 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.746519  1724 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:04.746615  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1974) took 1281 ms. Trace:
I20260709 13:02:04.727883  4369 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:02:04.756985  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:04.757712  1570 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.742569  4558 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.750272  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:04.759248  1645 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:04.751752  4572 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:04.755836  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:04.756827  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.763935  1571 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:04.713701  4503 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:04.747681  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1976) took 1278 ms. Trace:
I20260709 13:02:04.747542  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:04.758497  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:04.742951  4559 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.721657  4370 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:02:04.696429  4517 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.766664  1644 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:04.853472  1716 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:04.793856  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:04.854349  1576 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:04.820832  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44738 (request call id 75) took 2370 ms. Trace:
I20260709 13:02:04.851686  1648 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:04.854734  1573 rpcz_store.cc:276] 0709 13:02:02.450560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.450717 (+   157us) service_pool.cc:224] Handling call
0709 13:02:04.820586 (+2369869us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540027cc54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d68853 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:04.820726 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996284160}
I20260709 13:02:04.859998  4559 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.873987  1645 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:04.860900  4551 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:04.744546  4316 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:04.771325  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:04.890331  1575 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:04.889962  4316 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:04.853509  1723 rpcz_store.cc:276] 0709 13:02:03.468520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.746343 (+1277823us) service_pool.cc:224] Handling call
0709 13:02:04.746665 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.746862 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.746890 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.746936 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:04.746973 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.747127 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.747209 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.851267  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 448) took 1661 ms. Trace:
I20260709 13:02:04.758723  4570 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.883882  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.892782  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.894187  1723 raft_consensus.cc:2368] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:02:04.758754  4462 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.792184  4564 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:04.894187  1716 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:04.775890  1570 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:04.897485  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1977) took 1428 ms. Trace:
I20260709 13:02:04.903558  1724 rpcz_store.cc:276] 0709 13:02:03.468955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.749263 (+1280308us) service_pool.cc:224] Handling call
0709 13:02:04.749556 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.749784 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:04.789279 (+ 39495us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.896917 (+107638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.897007 (+    90us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:04.897096 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.897208 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.897392 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30598,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":1299,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":161,"thread_start_us":107058,"threads_started":1}
I20260709 13:02:04.894068  4517 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.924752  4316 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:04.935005  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.936513  1716 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 4: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:04.938273  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.912618  4538 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:02:04.911141  4570 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.968849  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1971) took 1588 ms. Trace:
I20260709 13:02:04.977310  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.985424  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:04.982564  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 446) took 1804 ms. Trace:
I20260709 13:02:04.963765  4539 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.986299  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 228352us
I20260709 13:02:04.945257  4564 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:04.985441  1721 rpcz_store.cc:276] 0709 13:02:03.380957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.995712 (+614755us) service_pool.cc:224] Handling call
0709 13:02:03.998051 (+  2339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.429428 (+431377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.968505 (+539077us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:04.968764 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985976192}
I20260709 13:02:05.001024  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:05.007766  1576 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:05.007993  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:04.913682  4462 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.000255  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:04.960276  4141 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:05.018558  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:05.020228  1569 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:05.024865  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:05.026945  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.030460  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:05.032889  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:05.034407  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:05.043270  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.049834  1576 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 4.
I20260709 13:02:05.050696  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:05.052081  1569 raft_consensus.cc:2410] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:05.051656  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:05.053217  1576 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:04.893458  2373 rpcz_store.cc:276] 0709 13:02:03.190338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.193951 (+  3613us) service_pool.cc:224] Handling call
0709 13:02:04.850132 (+1656181us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:02:04.850508 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.850770 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.850812 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.850881 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:04.850934 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.851022 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.851185 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111150464}
I20260709 13:02:05.056808  1573 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:04.959568  4448 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.081701  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:04.998359  2371 rpcz_store.cc:276] 0709 13:02:03.178634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.182042 (+  3408us) service_pool.cc:224] Handling call
0709 13:02:03.262802 (+ 80760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.282183 (+ 19381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.282357 (+   174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.282452 (+    95us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:03.282514 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.982159 (+1699645us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b5400573ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:04.982233 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.982469 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5499264}
I20260709 13:02:05.091820  4197 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.088369  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:05.011477  4539 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:05.105482  1539 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 2.865s (SENT) [suppressed 13 similar messages]
I20260709 13:02:05.105955  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:05.107453  4583 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.134s	user 0.000s	sys 0.002s
I20260709 13:02:05.107968  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:05.109483  4538 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:05.092608  4197 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.110145  4538 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:05.110899  4538 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:05.112546  4197 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.113317  4197 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:05.113871  4197 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.039587  4141 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.119129  4141 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.017217  4564 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.120834  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1978) took 1651 ms. Trace:
I20260709 13:02:05.121455  1723 rpcz_store.cc:276] 0709 13:02:03.469282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.904837 (+1435555us) service_pool.cc:224] Handling call
0709 13:02:04.905157 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.912849 (+  7692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.912884 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.918770 (+  5886us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.068256 (+149486us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:05.068326 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.069056 (+   730us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.120439 (+ 51383us) raft_consensus.cc:1685] finished
0709 13:02:05.120461 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.120755 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":210,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5745,"thread_start_us":114063,"threads_started":1,"wal-append.queue_time_us":180371}
I20260709 13:02:05.122603  4564 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.128044  4492 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.130497  4462 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.132853  1569 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:05.134392  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:05.173705  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.177508  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.061019  4568 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.201136  4568 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.204735  4537 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.210861  4539 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.224995  4197 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.227205  1716 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:05.228035  1717 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:05.228596  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1873) took 2126 ms. Trace:
I20260709 13:02:05.186038  4448 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.229038  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:05.235196  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57397.7us
I20260709 13:02:05.256104  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.245617  4517 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:05.252233  1719 rpcz_store.cc:276] 0709 13:02:03.102164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.102412 (+   248us) service_pool.cc:224] Handling call
0709 13:02:05.227590 (+2125178us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400223654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:05.228516 (+   926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626558848}
I20260709 13:02:05.264426  1719 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:05.234432  4316 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:05.248976  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1979) took 1779 ms. Trace:
W20260709 13:02:05.237011  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 149572us
I20260709 13:02:05.266001  4316 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.267060  4316 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:05.270824  4527 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.959s	user 0.028s	sys 0.010s
I20260709 13:02:05.265348  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1980) took 1791 ms. Trace:
I20260709 13:02:05.259521  1723 raft_consensus.cc:2376] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6. Re-sending same reply.
I20260709 13:02:05.271431  1721 rpcz_store.cc:276] 0709 13:02:03.469620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.021936 (+1552316us) service_pool.cc:224] Handling call
0709 13:02:05.022388 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.022733 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:05.059042 (+ 36309us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.248320 (+189278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.248445 (+   125us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:05.248525 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.248636 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.248876 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37386,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":6060,"raft.queue_time_us":143,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":4143,"thread_start_us":188646,"threads_started":1}
I20260709 13:02:05.278488  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.294551  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:05.282845  4457 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:02:05.259924  4197 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.290596  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 1998) took 1866 ms. Trace:
I20260709 13:02:05.291460  1719 rpcz_store.cc:276] 0709 13:02:03.474479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.263323 (+1788844us) service_pool.cc:224] Handling call
0709 13:02:05.263842 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.264945 (+  1103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.265258 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:05.296425  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.305697  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1983) took 1266 ms. Trace:
I20260709 13:02:05.305825  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.306310  1719 rpcz_store.cc:276] 0709 13:02:04.039559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.304516 (+1264957us) service_pool.cc:224] Handling call
0709 13:02:05.304937 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.305246 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.305289 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.305344 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:05.305399 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.305478 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.305624 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:05.308475  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.308866  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.254736  4492 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.316427  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:05.316459  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1981) took 1812 ms. Trace:
I20260709 13:02:05.317653  1648 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:02:05.317461  1723 rpcz_store.cc:276] 0709 13:02:03.503598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.292815 (+1789217us) service_pool.cc:224] Handling call
0709 13:02:05.315269 (+ 22454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.315847 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.315876 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.315925 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:05.315985 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.316053 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.316331 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:05.320453  4526 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:05.321460  4526 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.322057  4526 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:05.297231  4527 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.989s	user 0.029s	sys 0.011s
I20260709 13:02:05.249606  4537 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.291925  4528 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:05.301997  1572 rpcz_store.cc:276] 0709 13:02:03.423793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.424775 (+   982us) service_pool.cc:224] Handling call
0709 13:02:03.425069 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.460696 (+ 35627us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:03.461078 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.904143 (+443065us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.936651 (+ 32508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.936713 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:03.936768 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.290191 (+1353423us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b54000cbf54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:05.290272 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.290496 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40075,"prepare.queue_time_us":122484,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":124,"raft.queue_time_us":38393,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":21317,"spinlock_wait_cycles":81337088,"thread_start_us":442341,"threads_started":2,"wal-append.queue_time_us":388372,"wal-append.run_cpu_time_us":3095,"wal-append.run_wall_time_us":1315872}
I20260709 13:02:05.327816  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:05.301347  4551 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:05.314013  4517 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:05.339159  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:05.317512  4448 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:05.352658  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:05.354305  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.357121  1572 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:05.363991  1572 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:05.390501  1572 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:05.317380  4197 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.334414  4559 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.367844  4551 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:05.392915  4603 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.462363  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1982) took 1877 ms. Trace:
I20260709 13:02:05.463891  1721 rpcz_store.cc:276] 0709 13:02:03.585019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.298210 (+1713191us) service_pool.cc:224] Handling call
0709 13:02:05.298746 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.299082 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:05.324323 (+ 25241us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.461542 (+137219us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.461851 (+   309us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.461926 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:05.461985 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.462065 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.462301 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7620,"prepare.queue_time_us":25094,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":101,"raft.queue_time_us":11157,"spinlock_wait_cycles":8267648,"thread_start_us":120806,"threads_started":1}
I20260709 13:02:05.473750  2376 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:05.475399  4504 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:05.476119  4504 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:05.457068  4517 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:05.462266  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:05.491581  4588 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.257s	user 0.001s	sys 0.001s
W20260709 13:02:05.499292  4597 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:02:05.505983  4376 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:05.508306  4559 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.508665  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.510010  4197 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.510336  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:05.479007  4531 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 2.168s	user 0.016s	sys 0.027s
W20260709 13:02:05.477327  4595 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:02:05.505784  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.518682  1572 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:05.475759  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:05.486205  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.477303  4504 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:05.523949  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.536224  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 242149us
I20260709 13:02:05.518929  2377 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:02:05.536778  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.539243  1572 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:05.540014  1572 pending_rounds.cc:85] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:05.540531  1572 pending_rounds.cc:107] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:05.541033  1572 raft_consensus.cc:2889] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:05.544117  1572 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:02:05.544896  1572 pending_rounds.cc:85] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:05.545303  1572 pending_rounds.cc:107] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:05.545732  1572 raft_consensus.cc:2889] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:05.549713  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.468662  4603 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.524808  4197 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:05.570626  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.520565  4602 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:02:05.557490  4492 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.525033  4527 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:02:05.534672  4559 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:02:05.587474  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:05.548388  4604 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:02:05.575305  4564 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:05.594326  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 55183.5us
I20260709 13:02:05.596714  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:05.559944  4528 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:02:05.569497  4538 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.594712  4537 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:05.622256  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 514412us
I20260709 13:02:05.697237  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:05.698123  4563 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.386s	user 0.006s	sys 0.006s
I20260709 13:02:05.699303  4593 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:05.699492  4457 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.700248  4593 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.700192  4457 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.700973  4593 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.703284  4457 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.703792  4568 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.704367  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1239) took 2037 ms. Trace:
W20260709 13:02:05.623178  4605 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:02:05.575817  4197 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.704934  1646 rpcz_store.cc:276] 0709 13:02:03.666682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.666861 (+   179us) service_pool.cc:224] Handling call
0709 13:02:03.726667 (+ 59806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.729184 (+  2517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.729236 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.776124 (+ 46888us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.980075 (+203951us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:03.980145 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.980252 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.697061 (+1716809us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:05.703937 (+  6876us) raft_consensus.cc:1685] finished
0709 13:02:05.703961 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.704261 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1961,"raft.queue_time_us":2768,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":38629,"spinlock_wait_cycles":23940352,"thread_start_us":179193,"threads_started":1,"wal-append.queue_time_us":428368}
I20260709 13:02:05.708485  4197 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:05.646055  4608 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:02:05.704896  4568 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.714993  4568 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.715760  4593 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.717919  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2004) took 2265 ms. Trace:
I20260709 13:02:05.718557  1568 rpcz_store.cc:276] 0709 13:02:03.452384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.632573 (+180189us) service_pool.cc:224] Handling call
0709 13:02:03.633046 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.633531 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:03.634047 (+   516us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.650670 (+2016623us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:05.716801 (+ 66131us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b18003014e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a
0709 13:02:05.717332 (+   531us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.717401 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:05.717476 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.717598 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.717818 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8356,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":399,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":72530048,"thread_start_us":75184,"threads_started":1,"wal-append.queue_time_us":256840}
I20260709 13:02:05.599625  4564 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:05.676494  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 79868.4us
I20260709 13:02:05.731114  4564 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.589457  4376 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:02:05.685269  4589 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:05.734062  4529 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:05.734400  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1875) took 2629 ms (client timeout 1873 ms). Trace:
W20260709 13:02:05.739074  1718 rpcz_store.cc:269] 0709 13:02:03.104999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.105161 (+   162us) service_pool.cc:224] Handling call
0709 13:02:05.734103 (+2628942us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400224554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:05.734292 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839492864}
I20260709 13:02:05.741688  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:05.686167  4609 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
W20260709 13:02:05.686158  4550 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.702s	user 0.021s	sys 0.012s
I20260709 13:02:05.587689  4614 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:02:05.606593  4611 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:05.686474  4316 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.581911  4517 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:04.748616  1720 rpcz_store.cc:276] 0709 13:02:03.464862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.745265 (+1280403us) service_pool.cc:224] Handling call
0709 13:02:04.745884 (+   619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.746261 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.746293 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.746335 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:04.746368 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.746427 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.746551 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2362}
I20260709 13:02:05.688738  4439 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:05.689680  4612 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.596212  4442 raft_consensus.cc:2764] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:05.692839  4538 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.693989  4603 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:05.588950  4492 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.648597  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 1960) took 2292 ms. Trace:
W20260709 13:02:05.787338  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.736408  4602 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:05.759977  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:05.761081  4573 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.110s	user 0.004s	sys 0.001s
I20260709 13:02:05.773085  4439 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:05.760830  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.796435  4439 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.796939  4526 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.767942  4611 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:05.798600  4611 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.799505  4526 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:05.761741  4469 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.731915  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:05.800714  2371 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:05.703862  4457 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.801762  4457 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.802901  4376 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.803486  4376 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.765295  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.763584  4424 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.805270  4424 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.805727  4376 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:05.746557  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1900) took 1438 ms. Trace:
I20260709 13:02:05.806963  1725 rpcz_store.cc:276] 0709 13:02:04.308098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.324224 (+ 16126us) service_pool.cc:224] Handling call
0709 13:02:05.738507 (+1414283us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540010e5d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:05.746478 (+  7971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284571392}
I20260709 13:02:05.767544  4559 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:02:05.789629  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:05.791556  4492 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.787885  1717 rpcz_store.cc:276] 0709 13:02:03.356687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.752802 (+396115us) service_pool.cc:224] Handling call
0709 13:02:05.227781 (+1474979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.251849 (+ 24068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.648231 (+396382us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400223650. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:05.648489 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3329465600}
W20260709 13:02:05.763468  4174 consensus_queue.cc:1175] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3b57321031416cba9db75450ef302a" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260709 13:02:05.792253  2375 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:05.785303  4612 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.813372  4612 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.812146  4551 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.816524  4551 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.819360  1722 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:05.819687  4316 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:05.820608  4424 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:05.800053  4611 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:05.804071  4457 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.819763  4551 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:05.686547  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2002) took 2234 ms. Trace:
I20260709 13:02:05.881711  1567 rpcz_store.cc:276] 0709 13:02:03.451783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.547616 (+ 95833us) service_pool.cc:224] Handling call
0709 13:02:03.581936 (+ 34320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.657625 (+ 75689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.657679 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.657745 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:03.657809 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.656988 (+1999179us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b54001257d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:05.686272 (+ 29284us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.686447 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173312384}
I20260709 13:02:05.754079  1568 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:05.690465  1569 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:05.686764  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2013) took 2232 ms. Trace:
I20260709 13:02:05.891641  1574 rpcz_store.cc:276] 0709 13:02:03.454807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.836314 (+381507us) service_pool.cc:224] Handling call
0709 13:02:03.836919 (+   605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.178203 (+341284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.178260 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.178345 (+    85us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:04.178395 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.631694 (+1453299us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400310bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:05.686563 (+ 54869us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.686700 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784241536}
W20260709 13:02:05.896332  1522 auto_rebalancer.cc:893] Removing replace marker failed: Replica 4f737473248e4c059fa74af3b8e1f77e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:05.896117  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:05.897624  1522 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:02:05.898234  1522 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:05.898092  1573 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.258931   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:05.914023  1645 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:05.756229  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:05.921388  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 165060us
I20260709 13:02:05.934894  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:05.946386  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.948832  4538 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.950276  4538 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:05.953142  4517 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:05.954097  4517 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.960387  4366 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:05.967585  4469 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:05.968775  4469 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:05.901448  1575 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:05.954736  4517 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:05.980651  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:05.986150  1645 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:05.982998  1576 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:05.988755  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:05.997714  1570 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:06.003513  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 69794.6us
I20260709 13:02:06.003795  4614 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.004558  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.005007  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:06.005795  1574 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:06.006672  4442 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:06.007519  4442 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:06.008069  4602 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.009150  4602 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.009367  4442 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.013211  1646 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:06.014680  4619 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.015983  4619 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.019747  4619 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:05.993090  4469 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.023303  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.024525  2377 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:06.026822  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:06.029367  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46484.4us
I20260709 13:02:06.031113  4615 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:02:06.027998  2375 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 3.
I20260709 13:02:06.034833  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.036761  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.037328  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.038102  1651 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.040803  1650 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:06.014680  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.048925  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.049124  4551 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:06.050199  1567 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:06.049026  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:06.052946  4614 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:06.054774  4538 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:06.059880  4538 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:06.060967  4538 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.062119  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1915) took 1141 ms. Trace:
I20260709 13:02:06.062667  1716 rpcz_store.cc:276] 0709 13:02:04.920331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.937302 (+ 16971us) service_pool.cc:224] Handling call
0709 13:02:06.061551 (+1124249us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:06.061810 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425517440}
I20260709 13:02:06.064389  4538 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:06.067878  4625 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.112s	user 0.000s	sys 0.003s
I20260709 13:02:06.071264  4359 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 4 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.076915  4602 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:06.081444  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.082178  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.082465  4504 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.082728  1568 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:06.088243  4141 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:06.088974  4141 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:06.089027  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:06.091183  4141 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.096896  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:06.100495  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53344.7us
W20260709 13:02:06.103415  4575 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.457s	user 0.000s	sys 0.008s
I20260709 13:02:06.103900  4359 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:06.107335  2378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 396) took 2699 ms (client timeout 1936 ms). Trace:
W20260709 13:02:06.108004  2378 rpcz_store.cc:269] 0709 13:02:03.407502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.407753 (+   251us) service_pool.cc:224] Handling call
0709 13:02:06.106805 (+2699052us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540064f354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:06.107038 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300611968}
I20260709 13:02:06.110486  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1916) took 1157 ms. Trace:
I20260709 13:02:06.111173  1722 rpcz_store.cc:276] 0709 13:02:04.953275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.976919 (+ 23644us) service_pool.cc:224] Handling call
0709 13:02:06.109898 (+1132979us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540004ef54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:06.110203 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880980352}
I20260709 13:02:06.131798  1570 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:06.133370  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.136169  4364 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:06.137691  2380 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:06.137446  4364 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:06.142892  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.147176  4364 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.148936  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.153383  1643 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:06.157972  4570 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:06.165436  1573 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:06.166239  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1302) took 1172 ms. Trace:
I20260709 13:02:06.166587  1573 rpcz_store.cc:276] 0709 13:02:04.994155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.082307 (+ 88152us) service_pool.cc:224] Handling call
0709 13:02:05.895634 (+813327us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:02:06.166139 (+270505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12893952}
I20260709 13:02:06.169754  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:06.186215  1651 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:06.186630  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.205598  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:06.206658  1574 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.211069  1647 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:06.212579  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 237103us
I20260709 13:02:06.217031  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.223958  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.237808  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.239274  1643 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:06.240639  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.242091  4568 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.242929  4568 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.245604  4547 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.246578  4547 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.249747  4568 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.251276  4547 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:06.254863  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.255788  4419 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:06.256704  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.257531  1571 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:06.257932  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.257097  4419 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.183647  4570 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:06.257109  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.260304  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.260010  4570 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.261307  1573 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:06.263418  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:06.263775  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.261307  1570 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:06.278757  1574 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:06.252467  4547 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.287828  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.292165  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.297940  4611 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:06.298732  4419 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.298988  4611 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.300663  4547 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.303929  4547 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.304786  4570 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.307483  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:06.308441  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 174538us
I20260709 13:02:06.299937  4611 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.315910  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 450) took 2854 ms. Trace:
I20260709 13:02:06.312008  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.323945  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.370954  1573 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:06.324860  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.324898  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:06.325871  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 177269us
I20260709 13:02:06.342109  1647 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:02:06.345908  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.349421  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.351428  1648 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:06.351610  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.352491  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:06.352823  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.354099  1643 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:06.376757  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:06.377753  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.369722  2380 rpcz_store.cc:276] 0709 13:02:03.462025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.464920 (+  2895us) service_pool.cc:224] Handling call
0709 13:02:06.137104 (+2672184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.138336 (+  1232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.315598 (+177262us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b540064f350. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:06.315803 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6087405952}
W20260709 13:02:06.383539  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1849) took 2179 ms (client timeout 1615 ms). Trace:
W20260709 13:02:06.384366  1647 rpcz_store.cc:269] 0709 13:02:04.194728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.194876 (+   148us) service_pool.cc:224] Handling call
0709 13:02:05.714944 (+1520068us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b180027a0e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833da9646 00007fc833ce358e 00007fc833d64f9e 00007fc833d667de 00007fc833d7227c 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c
0709 13:02:06.182384 (+467440us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b58004202d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833cca0aa 00007fc833d4c447 00007fc833d64fe7 00007fc833d667de 00007fc833d7227c 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c
0709 13:02:06.373645 (+191261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771202688}
I20260709 13:02:06.390777  1576 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:06.382449  1650 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:06.396210  1642 raft_consensus.cc:2368] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6.
I20260709 13:02:06.397991  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:06.403908  1567 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:02:06.405615  4440 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:06.406462  4440 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.407068  4440 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.423632  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.426275  4519 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:06.426654  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:06.427027  4519 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.399410  3782 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.427577  4519 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:06.436997  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:06.396857  4570 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.443758  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.444036  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.446255  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.434294  4504 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:06.451058  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.454663  1576 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:06.458109  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.469645  4370 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:06.469317  4570 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.441682  3782 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.474483  1570 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:06.463697  1688 connection.cc:256] RPC call timeout handler was delayed by -1.17788367899999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 13:02:06.477067  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 129885us
I20260709 13:02:06.463163  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:06.485409  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.485937  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0230836790000239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:02:06.486193  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0149836790000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
I20260709 13:02:06.467594  4504 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:06.488198  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:06.497042  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1319) took 1436 ms. Trace:
I20260709 13:02:06.497043  4441 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:06.488426  4542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.746s	user 0.050s	sys 0.064s
I20260709 13:02:06.486614  4517 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:06.513497  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.508767  4542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.857s	user 0.051s	sys 0.065s
I20260709 13:02:06.523104  1570 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:06.536410  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.519382  4607 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:06.520364  4370 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:06.539402  4441 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:06.548138  4634 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:06.568193  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.539565  4359 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:02:06.576274  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.578060  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.579367  1569 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:02:06.579545  4636 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:06.554190  4643 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.580919  4643 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.582058  4614 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:06.584270  4504 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.572491  4607 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.595105  4607 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.598021  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.599084  1568 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:06.599793  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.497737  1572 rpcz_store.cc:276] 0709 13:02:05.060380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.580989 (+520609us) service_pool.cc:224] Handling call
0709 13:02:05.581435 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.889188 (+307753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.889297 (+   109us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.890643 (+  1346us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.977397 (+ 86754us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:05.977469 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.495775 (+518306us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400001c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:06.495990 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.496080 (+    90us) raft_consensus.cc:1685] finished
0709 13:02:06.496095 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.496658 (+   563us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74720,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":258,"spinlock_wait_cycles":704498432,"thread_start_us":80848,"threads_started":1,"wal-append.queue_time_us":180281}
I20260709 13:02:06.600278  4642 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.598138  1570 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:06.599273  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.603850  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:06.618968  1643 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:06.620276  1574 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 4: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:06.628116  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:06.633311  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.635573  4607 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.614917  4517 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.655030  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.662876  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.664664  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.665231  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.666138  2375 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 6: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:06.666604  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.667248  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.644711  4440 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.668656  4642 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.669857  4440 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.667302  4370 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.681984  3782 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.676486  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.634608  4441 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.658640  1646 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:06.684296  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.683281  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:06.648214  4359 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.650655  4570 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.707882  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:06.712527  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 166958us
I20260709 13:02:06.685353  4642 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.719780  1643 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:06.726724  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.728247  4592 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.728466  4457 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:06.729308  4457 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.729185  4592 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.730131  4457 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.736567  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.738266  4547 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:06.699483  4607 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.750648  1643 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:06.761889  4641 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.002s
I20260709 13:02:06.739573  4359 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.791404  4648 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:06.792268  4648 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:06.793606  4648 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:06.795130  4648 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.797122  4607 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:06.795166  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 138042us
I20260709 13:02:06.783746  4547 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:02:06.809054  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.812232  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.812527  4311 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:06.813298  4570 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:06.813769  4570 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.814482  4570 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.815465  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.808974  4607 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.816598  4607 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:06.813457  4611 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:06.817811  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.818849  4526 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:06.813290  4311 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.827297  1539 leader_election.cc:400] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6.
I20260709 13:02:06.873689  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.898880  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:06.847298  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:06.873121  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.851410  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:06.840269  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.920735  2380 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:06.883872  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.921670  1725 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:06.923254  1576 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:06.913178  1649 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:02:06.894990  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.850106  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:06.851186  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.889101  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:06.819820  4526 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:06.869707  4592 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.868098  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.924602  1716 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:06.885869  4441 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:06.895949  4410 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.948819  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:06.962059  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 135053us
I20260709 13:02:06.962389  1718 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:06.894757  4141 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.820139  4507 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:06.819269  4607 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.820283  4311 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.902361  4643 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.947367  1573 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:06.948553  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.963852  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.968103  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:06.964217  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:06.976449  1648 raft_consensus.cc:2376] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4. Re-sending same reply.
W20260709 13:02:06.969156  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.977818  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:06.890368  4359 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.847242  4611 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:06.898880  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:06.972434  1717 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:02:06.973349  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.822621  4642 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:06.906262  4369 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
W20260709 13:02:06.985611  2379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 340) took 1533 ms (client timeout 1754 ms). Trace:
I20260709 13:02:06.981781  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:06.836949  4440 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.141s
I20260709 13:02:06.997746  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:06.998698  1725 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:06.985235  2375 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 3.
I20260709 13:02:07.000720  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:07.005914  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 155987us
I20260709 13:02:07.003628  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:06.991792  2379 rpcz_store.cc:269] 0709 13:02:05.444819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.445206 (+   387us) service_pool.cc:224] Handling call
0709 13:02:06.888675 (+1443469us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:02:06.978444 (+ 89769us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10551552}
I20260709 13:02:06.961128  4570 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:07.011343  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 284190us
I20260709 13:02:07.008483  1716 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:07.013674  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48959.1us
I20260709 13:02:06.986155  4602 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:07.016698  1576 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:06.908057  4612 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 13:02:07.019529  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:07.007053  4519 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.568s	user 0.016s	sys 0.000s
I20260709 13:02:06.997150  4547 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:07.029000  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.031656  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.034232  1725 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:07.030236  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:07.006243  4607 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:07.037981  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.041927  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.053617  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.057561  1718 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.057711  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.058440  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.058902  1719 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:07.037760  4592 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:07.061098  4592 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.061548  4592 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:07.062340  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:07.051683  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:07.064643  4547 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:07.065353  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46974.8us
I20260709 13:02:07.066535  4604 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.067324  4604 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:07.065583  4647 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.269s	user 0.006s	sys 0.000s
I20260709 13:02:07.066524  4612 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:07.065395  4547 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:07.057572  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.072880  1721 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:07.049621  4526 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.074033  4526 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.077708  1720 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:07.049414  4602 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.081439  4424 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:07.082018  4424 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:07.057801  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.083160  4424 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.057997  1717 raft_consensus.cc:2376] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6. Re-sending same reply.
I20260709 13:02:07.088505  1724 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:02:07.091717  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:07.097723  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58341.8us
W20260709 13:02:07.101122  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50658.7us
I20260709 13:02:07.104317  1573 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:07.106462  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:07.108541  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.109927  4352 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.110818  4352 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.114967  4643 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:07.115005  4359 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:07.117897  4359 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.072253  4547 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.116074  4643 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.125257  4440 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:07.127942  4643 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.128548  4369 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.129338  4369 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.130028  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.131799  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.133103  1567 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.134594  4424 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:07.034235  4519 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.154968  4440 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.157680  4440 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.158355  4440 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:07.159394  1650 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:07.160276  4359 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:07.155035  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 135840us
I20260709 13:02:07.047391  4642 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:07.173804  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 109586us
I20260709 13:02:07.188593  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.190503  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:07.191802  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.198457  1644 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.158744  4457 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.128865  4643 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:07.203136  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:07.202220  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50511.8us
I20260709 13:02:07.186740  4507 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:07.187342  4604 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.236550  4182 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:07.236899  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.237247  4182 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.237712  4182 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:07.225445  4441 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.239202  4441 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.240331  1722 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:07.241154  4570 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.240826  4643 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.242239  4570 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:07.242676  4570 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.243038  4570 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.243666  4643 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.245990  4457 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.217749  4369 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.252111  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.257980  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.248725  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:07.230772  4507 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.280692  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.281772  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.278589  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.291257  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:07.319798  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:07.319808  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:07.287328  4507 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.283129  4602 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:07.342562  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93926.4us
I20260709 13:02:07.341398  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.272704  4141 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:07.345252  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:07.345645  1718 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.345319  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.354202  1717 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:07.319538  4539 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:07.358160  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 167746us
I20260709 13:02:07.361199  1724 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:07.356848  4352 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:07.338469  4636 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:07.361654  1719 raft_consensus.cc:2368] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:07.325855  4369 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:07.342485  4646 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:02:07.362823  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:07.371157  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 122597us
W20260709 13:02:07.372179  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.338318  4659 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:07.374256  1720 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:07.374683  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.383296  4640 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:07.385062  4662 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:02:07.385871  4637 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.846s	user 0.006s	sys 0.003s
I20260709 13:02:07.387584  4642 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.390161  4439 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.394191  1725 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:07.399281  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59063.8us
I20260709 13:02:07.400520  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:07.400596  4539 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:07.402117  4539 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.402155  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:07.414232  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.423457  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:07.423825  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.425128  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51693.3us
I20260709 13:02:07.422511  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.427464  1718 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:07.428020  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:07.410352  4661 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.050s	user 0.002s	sys 0.000s
I20260709 13:02:07.427057  1721 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:02:07.430377  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.433271  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.436239  1720 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:07.436496  1723 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:07.438807  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.350816  4311 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.445406  1724 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:07.379777  4659 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:07.422628  4612 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:07.466202  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 59247.6us
I20260709 13:02:07.446597  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:07.454022  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.459718  4304 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:07.460259  4547 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:07.463120  4592 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.424903  4640 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:07.464243  4672 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:07.465351  4663 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:02:07.481977  1718 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:07.484740  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.489846  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.449604  4519 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.429395  4457 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:02:07.466655  4526 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:07.451912  4659 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:07.502024  4672 raft_consensus.cc:2764] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 13:02:07.512267  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 141775us
W20260709 13:02:07.519076  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72586.8us
I20260709 13:02:07.513430  1716 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
W20260709 13:02:07.521248  4666 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:02:07.522464  4640 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.524976  4311 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.528301  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.530547  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.531613  4441 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.529757  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.532295  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.533525  1717 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:07.534196  1720 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.534719  4612 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:07.535288  4612 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:07.535422  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.535571  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:07.532507  4421 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:07.536659  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.532780  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.538208  1724 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:07.539388  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.533008  1718 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:07.541996  4671 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:07.542397  1718 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:07.542464  4671 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:07.537570  1721 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:07.544732  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 61605.6us
I20260709 13:02:07.547362  1718 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:07.547842  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.556810  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.558084  1716 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:07.562909  1720 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.563359  4182 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.566051  4182 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.567721  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.568567  1570 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:07.568636  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.569509  1572 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.570187  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.577422  4612 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.577950  4592 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.581727  1717 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:07.589535  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 143243us
I20260709 13:02:07.586889  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:07.608162  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47783.3us
I20260709 13:02:07.611155  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:07.613288  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:07.614300  1571 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.635632  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.543834  4671 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.591637  4570 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.637611  1575 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.638821  4570 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:07.639591  4648 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.640198  4648 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.644411  4441 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:07.645892  4612 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.647145  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.653016  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:07.663671  1724 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:07.626015  4526 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:07.676838  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 162717us
W20260709 13:02:07.679122  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.654953  4457 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:07.679839  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.688009  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.672760  4526 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.672842  4642 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:07.697691  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.680806  4592 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:07.707427  4675 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:07.707471  4604 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:07.701766  4519 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:02:07.723753  1573 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:07.724233  4652 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:02:07.725675  4650 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:07.726136  4650 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:07.727169  4650 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.728698  4650 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:07.729920  4648 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.701152  4457 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.742926  4457 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:07.703776  4141 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:02:07.700023  4526 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:07.757534  4539 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:07.758356  4539 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:07.767556  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.701323  4592 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:07.772074  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69348.9us
I20260709 13:02:07.782439  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.759639  4539 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:07.787494  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 89925.3us
I20260709 13:02:07.784080  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:07.790540  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:07.791355  1651 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:07.801268  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.806744  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.778272  4592 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:07.770426  4675 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:07.694613  4421 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 13:02:07.813994  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:07.814477  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:07.792222  3782 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.815693  3782 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.818693  3782 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:07.809244  4424 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:02:07.825687  4509 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.826436  4509 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.810887  4600 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:07.847003  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.840482  4611 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:07.877826  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 89305.2us
I20260709 13:02:07.881640  4600 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.881834  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81649.3us
I20260709 13:02:07.882287  2379 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:07.882748  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.883800  4526 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.886351  4526 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.887451  4509 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.887974  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:07.876653  4643 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:02:07.888713  1573 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:07.889143  4643 consensus_queue.cc:1243] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:07.889770  4643 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:07.890149  4643 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:07.892302  4469 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:07.893137  4469 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:07.893635  4469 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.894068  2375 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:07.805105  4369 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:02:07.891268  4643 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.902500  4643 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.772173  4507 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.882299  4600 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:07.880869  4659 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.905881  4600 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.892210  4457 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:07.907142  4659 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:07.908202  1719 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:07.808683  4672 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:07.916098  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:07.916814  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.905354  4675 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:07.784741  4650 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:02:07.917219  1574 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:07.919296  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 209874us
I20260709 13:02:07.884122  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:07.918294  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 208251us
I20260709 13:02:07.918751  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:07.926273  1648 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:07.932701  1571 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:07.879988  4614 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.936597  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.937935  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53897.3us
I20260709 13:02:07.939178  4683 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:07.941757  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.942404  1573 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:07.943902  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:07.941782  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.916471  4672 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:07.949538  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.949158  1651 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
W20260709 13:02:07.957494  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.960109  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.962092  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:07.944746  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:07.965674  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:07.970656  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.953481  4672 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:07.943809  4457 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:07.966615  1567 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.962637  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:07.979777  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 100799us
I20260709 13:02:07.980201  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:07.968834  1717 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.971639  1725 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.933894  4671 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.983011  1570 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:07.978444  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.984445  1718 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:07.985796  1573 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:02:07.986934  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:07.968362  4311 raft_consensus.cc:2764] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:07.992144  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:07.988523  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:07.997515  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:07.973543  4517 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:07.995260  4642 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:07.969211  4547 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:08.004788  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:08.019004  4539 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:07.949308  4614 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:08.021647  4676 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:02:08.031304  1571 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:08.033955  1688 leader_election.cc:400] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6.
I20260709 13:02:08.034355  1688 leader_election.cc:403] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:08.035205  1573 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:08.047827  1569 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.996256  4369 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:08.056008  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 57495.7us
I20260709 13:02:08.058672  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:07.993135  4683 raft_consensus.cc:2764] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:08.053663  1574 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:08.064556  1576 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:08.066461  1568 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:08.077538  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:08.047010  4642 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:08.107134  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.088145  4526 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:08.087666  4369 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:08.092175  4614 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:08.073155  4469 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.101966  4659 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:08.133023  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 167431us
I20260709 13:02:08.137022  1567 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:08.136801  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:08.094233  4519 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 6.
W20260709 13:02:08.151077  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.150871  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 186668us
I20260709 13:02:08.152266  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:08.156111  4698 raft_consensus.cc:2764] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:02:08.154273  4659 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:08.173035  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.173944  1648 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:08.176084  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.176961  1644 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.180896  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.181474  4469 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:08.182395  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.182654  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.184000  4509 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:08.184504  4509 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.184916  4509 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.184947  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.185973  1642 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:08.187444  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.173122  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:08.158218  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:08.195350  4698 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:08.195840  4698 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.196254  4698 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.196833  4648 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:08.197383  4648 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:08.198344  4648 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.198486  4369 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.199837  4648 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:08.182731  4659 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:08.209450  4689 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:02:08.201273  4652 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:08.212641  4652 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.213191  4652 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.202392  4643 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.216959  4643 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.218017  4642 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.203075  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.196060  1574 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:02:08.205718  4507 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:08.225136  1719 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:08.238194  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.206415  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.234133  1649 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:08.240568  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:08.242554  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.244831  4672 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.246799  4643 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:08.247172  4672 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:08.247972  1569 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:08.249181  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.249269  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.252530  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:08.232915  1567 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:08.259390  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.258939  4441 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:08.266425  4640 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.269290  4611 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:08.271092   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:08.272699  4611 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:08.276595  4542 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:08.277125  4542 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:08.278198  4542 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.279337  4542 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.293519  1648 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:08.306066  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:08.322885  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.329438  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.241314  4519 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.335223  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.336723  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:08.337494  1725 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:08.341725  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:08.342594  2380 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.344020  1649 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:08.345177  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.345911  1722 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.346952  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:08.349879  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.354830  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:08.344224  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:08.357077  1720 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:08.355980  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:08.280220  4652 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.359961  2379 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:08.360455  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.363288  1648 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:08.368741  1576 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:08.368963  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.369875  1723 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:08.372023  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.373353  1716 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.367604  4507 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.388985  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.389127  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.389793  1567 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:08.397241  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.404054  1644 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:08.346328  4643 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.407528  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.407528  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.408459  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.381974  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:08.408686  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.409322  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.413683  1648 raft_consensus.cc:2368] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10.
W20260709 13:02:08.411058  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 427207us
I20260709 13:02:08.328616  4517 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:02:08.417146  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.423173  1651 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.418308  1568 raft_consensus.cc:2368] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 4. Current term is 5.
I20260709 13:02:08.419274  1644 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:08.419852  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:08.427044  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:08.417146  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.427785  1725 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:08.427940  1569 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 4.
I20260709 13:02:08.382212  4671 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:08.361573  4592 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.441891  1716 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:08.414160  4675 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.383767  4519 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.452914  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.455394  1720 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:08.458441  2375 raft_consensus.cc:2368] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:02:08.464826  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.449723  2372 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:08.469343  4704 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:02:08.395622  4507 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.472427  1719 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:08.479487  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:08.506450  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 86787.2us
I20260709 13:02:08.507623  1721 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:08.456528  4693 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:08.438378  4643 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:08.521445  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63205.5us
I20260709 13:02:08.487779  4675 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:08.533428  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:08.509881  1472 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.532199  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:08.531605  1651 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
W20260709 13:02:08.538169  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 90600.7us
I20260709 13:02:08.545115  1539 leader_election.cc:400] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:08.551458  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:08.555950  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.478021  4311 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:02:08.558076  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.584054  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.507494  4671 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.560587  4469 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:02:08.618000  4698 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.597668  1472 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.619657  1567 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
W20260709 13:02:08.618119  1473 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 206) took 23380 ms (client timeout 15000 ms). Trace:
I20260709 13:02:08.618790  4469 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:08.624327  4469 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.624993  4469 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.626400  4643 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:08.558899  4693 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:02:08.629406  4519 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:08.558781  4683 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:02:08.636799  4592 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:08.623406  1472 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.645836  4652 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:08.650683  4507 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:08.652195  4705 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:08.594404  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:08.664009  1472 catalog_manager.cc:5697] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 3 to 4, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 4 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.665951  4671 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:08.662138  4706 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:02:08.669313  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.669627  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.670599  1648 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:08.671505  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.672212  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.672740  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.673938  1650 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:08.675724  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.680460  1648 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:08.681303  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.628082  4693 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:08.688905  4693 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:08.688979  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.690248  4693 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.692919  4693 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:08.698678  4469 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:08.703838  4469 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:08.707396  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.759534  1651 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:08.760664  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:08.708935  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.763957  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:08.764859  1721 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:08.765955  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:08.767230  1576 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:08.714548  1646 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:02:08.714881  4698 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:08.550189  4650 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 13:02:08.769335  1575 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:08.770502  1646 pending_rounds.cc:85] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 4
I20260709 13:02:08.771164  1646 pending_rounds.cc:107] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:02:08.772001  1646 raft_consensus.cc:2889] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:08.726130  1650 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:02:08.773593  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 250453us
I20260709 13:02:08.775843  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:08.734155  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.812888  4182 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:08.731869  1472 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 3 to 4, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 4 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:08.814982  4182 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:08.815068  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:08.815653  4182 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:08.743197  4710 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.125s	user 0.004s	sys 0.005s
I20260709 13:02:08.816253  4604 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:08.816408  2372 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:08.817342  4707 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:08.817554  4650 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:08.818521  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 260351us
I20260709 13:02:08.819234  4707 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.819528  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.821350  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.821619  1572 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:08.823177  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.823823  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.826095  4714 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:08.825007  1722 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:08.826645  4714 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.827097  4714 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.746852  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:08.828593  4604 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.829155  1645 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:08.753129  1574 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:08.708562  1723 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
W20260709 13:02:08.621349  1473 rpcz_store.cc:269] 0709 13:01:45.093279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.093777 (+   498us) service_pool.cc:224] Handling call
0709 13:02:08.450508 (+23356731us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:59.460475 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:59.461435 (+   960us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:59.461495 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:01:59.461515 (+    20us) tablet.cc:662] Decoding operations
0709 13:01:59.681671 (+220156us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:59.704501 (+ 22830us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:59.704558 (+    57us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:01:59.762456 (+ 57898us) tablet.cc:701] Row locks acquired
0709 13:01:59.762516 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:01:59.762766 (+   250us) write_op.cc:270] Start()
0709 13:01:59.762939 (+   173us) write_op.cc:276] Timestamp: P: 1783602119762677 usec, L: 0
0709 13:01:59.762976 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:01:59.860821 (+ 97845us) log.cc:844] Serialized 11038 byte log entry
0709 13:02:00.483574 (+622753us) op_driver.cc:464] REPLICATION: finished
0709 13:02:00.755845 (+272271us) write_op.cc:301] APPLY: starting
0709 13:02:00.775757 (+ 19912us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:00.844841 (+ 69084us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:00.845004 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:00.957295 (+112291us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:01.018446 (+ 61151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=28
0709 13:02:01.018504 (+    58us) write_op.cc:312] APPLY: finished
0709 13:02:01.019188 (+   684us) log.cc:844] Serialized 188 byte log entry
0709 13:02:01.019477 (+   289us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:01.019765 (+   288us) write_op.cc:454] Released schema lock
0709 13:02:01.019886 (+   121us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:01.092531 (+ 72645us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:01.095718 (+  3187us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":370216,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":226593,"apply.run_cpu_time_us":11238,"apply.run_wall_time_us":314424,"num_ops":28,"prepare.queue_time_us":263394,"prepare.run_cpu_time_us":35120,"prepare.run_wall_time_us":563154,"raft.queue_time_us":202437,"raft.run_cpu_time_us":29060,"raft.run_wall_time_us":298647,"replication_time_us":649552,"thread_start_us":441891,"threads_started":4,"wal-append.queue_time_us":178000,"wal-append.run_cpu_time_us":23793,"wal-append.run_wall_time_us":1969949}]]}
I20260709 13:02:08.778204  1570 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:08.832052  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:08.791529  4640 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.798177  4680 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:08.804970  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.805948  1575 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:08.841531  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:08.731181  4509 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.000s
I20260709 13:02:08.818254  4650 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.773707  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:08.744649  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 150414us
I20260709 13:02:08.838308  1570 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:08.843241  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.836867  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:08.847373  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.850420  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:08.851586  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.851897  4687 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:02:08.848647  4441 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:02:08.854982  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79251.3us
I20260709 13:02:08.857525  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:08.858384  1573 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.858501  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:08.860737  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.863590  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:08.867424  4680 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:08.868121  4680 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.848192  4421 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:08.868865  4421 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.869271  4421 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.854907  4526 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.849177  4604 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:08.885730  4699 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:08.886070  4441 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:08.886415  4699 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:08.886862  4699 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:08.887388  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:08.888554  4713 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.889124  4713 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.852283  4469 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:08.852573  4698 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.853238  4519 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.851897  4410 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:08.861699  4642 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.893045  4469 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.861635  4715 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:08.934674  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.937479  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.861714  4141 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:02:08.942109  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.856866  4671 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:08.944056  1573 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:08.946595  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 92968.9us
I20260709 13:02:08.854447  4509 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:08.894570  4604 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:08.965556  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.966179  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.894153  4716 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:08.968355  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64964.5us
I20260709 13:02:08.978811  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:08.974453  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 122639us
I20260709 13:02:08.976074  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:08.942903  4469 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.932613  4683 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:08.907697  4707 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:08.906492  4650 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:08.982426  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:08.941167  4715 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.981329  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:08.905110  4592 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:02:08.904843  4526 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:02:08.991458  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:08.996762  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:08.967809  4517 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:08.967566  4614 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.987255  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.981786  4604 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:08.943534  4643 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.980772  4693 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.986605  4683 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:08.980994  4716 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.981223  4675 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.943234  4640 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:02:08.985677  4644 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:08.986646  4519 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:08.988683  4659 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.003443  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:08.988840  4713 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:09.026324  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50315.7us
I20260709 13:02:08.997867  4507 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:08.993518  4715 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.009218  4542 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:09.066532  4182 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.011457  4311 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.059849  4141 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:09.011269  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.011734  4672 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:09.019264  4600 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.057039  4687 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:02:09.028693  4683 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.028915  4716 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.013512  4614 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:09.125146  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63678.7us
I20260709 13:02:09.081563  4311 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.126044  4141 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:09.137253  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 154875us
I20260709 13:02:09.028474  4693 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.145427  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:09.108850  4726 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:09.141911  4728 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:09.149940  4141 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.152741  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:09.153847  1723 raft_consensus.cc:606] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.154232  1723 raft_consensus.cc:614] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:09.119366  2372 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:09.085131  4517 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:02:09.127324  4671 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.046923  4675 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.130935  4672 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:09.139828  4311 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:09.160750  4721 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:02:09.151041  4728 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.212001  4421 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.125224  4600 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.213092  4517 consensus_queue.cc:1243] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:09.247984  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:09.251122  1723 raft_consensus.cc:606] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.252040  1723 raft_consensus.cc:614] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:09.248714  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 262800us
I20260709 13:02:09.262732  1645 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:09.270208  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 107260us
W20260709 13:02:09.276115  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 272785us
I20260709 13:02:09.245604  4182 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.279532  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.243672  4723 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:02:09.301242  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:09.269662  4644 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:09.304874  4644 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.307413  4644 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.292069  4592 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.309139  4725 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:02:09.308971  4592 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:09.311410  4722 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:02:09.275031  4640 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.278829  4687 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.277348  4519 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.284269  4611 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:09.326045  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 316111us
W20260709 13:02:09.367027  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.366916  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:09.368429  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47266.7us
I20260709 13:02:09.371170  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:09.371212  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.296380  4642 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.312039  4592 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.292038  4509 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.385157  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:09.386090  1720 raft_consensus.cc:606] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.386507  1720 raft_consensus.cc:614] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:09.330374  4410 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.325228  4687 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:09.402807  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 264621us
I20260709 13:02:09.403052  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:09.457270  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84952.3us
I20260709 13:02:09.379309  4519 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:09.514652  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 213539us
I20260709 13:02:09.517481  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.393887  4675 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.521409  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:09.539270  1648 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:09.454049  4680 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.004s
I20260709 13:02:09.374758  4713 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:09.331194  4640 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.552269  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:09.553128  4469 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.005s
I20260709 13:02:09.553529  4469 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.555008  4640 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.518419  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:09.459879  4687 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.454270  4507 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.564092  4507 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:09.564641  4507 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.566911  4507 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.532296  4182 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:09.461396  4410 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.373162  4735 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.219s
I20260709 13:02:09.568774  4640 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:09.569854  4410 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.570431  4410 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.571066  4410 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.550712  4519 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.406417  4728 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.572965  4539 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:09.573585  4539 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.574610  4410 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.575040  4728 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.394027  4642 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.579222  4642 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.580570  4650 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.583664  1648 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
W20260709 13:02:09.584839  4740 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:02:09.588557  4643 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.589248  4643 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.593847  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:09.595743  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:09.596426  1724 raft_consensus.cc:606] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.596786  1724 raft_consensus.cc:614] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:09.597086  4600 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:09.597393  4698 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.003s
I20260709 13:02:09.581845  4650 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.546036  4614 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.598515  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.392915  4644 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.600953  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:09.622402  1724 raft_consensus.cc:606] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.607970  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:09.622885  1724 raft_consensus.cc:614] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:09.601907  1644 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:09.561837  4713 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.373324  4730 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.156s
I20260709 13:02:09.393358  4611 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.629365  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.539146  4542 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.637760  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 105513us
I20260709 13:02:09.631824  1720 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:09.632408  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:09.629460  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 258372us
I20260709 13:02:09.636673  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:09.643193  1723 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:09.647789  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.648593  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:09.649181  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:09.649583  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:09.650126  4748 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.650336  1575 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.650568  4519 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:09.650640  4642 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:09.656657  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.598021  4698 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:09.621250  4675 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.523216  4715 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.666520  4640 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:09.599421  4643 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.618446  4671 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.666301  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 149497us
I20260709 13:02:09.373169  4672 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.625499  4693 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.674889  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.597784  4687 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.574899  4539 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.683837  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.688087  2379 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:09.667475  4421 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.666399  4182 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.602088  4680 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.690557  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:09.695516  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.693603  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.666301  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.669386  4675 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:09.685429  4714 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.858s	user 0.009s	sys 0.008s
I20260709 13:02:09.676859  4469 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.706290  4675 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.712531  4733 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:09.729045  4733 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.730147  4723 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:09.730659  4723 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.731606  4611 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.722818  1575 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:02:09.660118  4542 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.697574  2373 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:09.705191  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:09.692086  4652 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:09.739518  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.701021  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.683499  4749 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:09.774147  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 69073.3us
W20260709 13:02:09.772667  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 493237us
I20260709 13:02:09.775615  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:09.738170  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.679787  4671 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:09.718990  4736 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:02:09.769719  4539 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:09.723963  4604 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.004s
I20260709 13:02:09.722446  4592 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:09.730213  4182 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.750150  4714 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.732040  4723 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.788285  4715 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.759924  4739 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.803683  4739 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.804052  4739 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.775336  4439 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:09.809702  4714 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.810153  4439 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:09.810757  4439 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:09.813405  4713 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.730175  4733 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.782642  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:09.818209  4733 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:09.818454  4732 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.818722  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:09.819047  4732 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.819469  4732 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.719772  4644 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.820796  4644 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.791728  4604 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.822482  4650 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.823841  4604 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.827479  4749 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:09.829233  4749 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.833938  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:09.836278  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:09.837437  1722 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:09.752033  4652 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.838161  4652 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:09.839555  4683 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.000s
I20260709 13:02:09.840114  4683 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.842751  4683 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.844734  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.845551  1642 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:09.846884  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.848057  4680 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:09.800468  4643 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:09.849097  4643 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.821331  4644 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.852609  4643 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:09.855304  4699 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.968s	user 0.014s	sys 0.000s
I20260709 13:02:09.785903  4687 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:09.779582  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:09.857048  1644 raft_consensus.cc:606] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:09.861367  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 186594us
I20260709 13:02:09.793373  4592 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.796422  4671 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:09.865257  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.872936  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:09.876084  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 100670us
W20260709 13:02:09.877131  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.877601  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.879346  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.717588  4741 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:09.802232  4723 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.801288  2375 raft_consensus.cc:2410] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:09.884961  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:09.802260  4659 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.001s
W20260709 13:02:09.821319  4311 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 13:02:09.791142  4542 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.890439   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:09.864506  4745 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.915673  1645 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:09.917938  4750 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.918721  4750 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.919884  4742 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:09.926481  4742 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:09.926802  4439 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.921600  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:09.928722  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51247.9us
W20260709 13:02:09.923765  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.929764  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:09.929919  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.012s (SENT)
I20260709 13:02:09.894721  4707 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:09.930799  4714 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:09.855988  4699 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.931635  4730 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.931994  4614 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:09.925011  4592 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.925591  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.930426  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:09.931088  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46281.3us
I20260709 13:02:09.909217   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:09.933559  1647 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:09.932035  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:09.941056  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:09.887027  4741 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:09.942549  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:09.959380  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:09.942761  1645 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:09.953362  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.972968  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:09.905694  4659 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:09.906209  4680 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.981606  4683 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:09.923786  4752 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:09.987282  4750 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:09.987622  4699 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:09.987792  4725 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 13:02:09.988480  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58594us
I20260709 13:02:10.061573  1651 pending_rounds.cc:85] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 3
I20260709 13:02:09.927927  4742 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.990445  4744 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:09.997134  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:09.998970  4713 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:09.933764  4671 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.971590  4741 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.000598  2372 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:10.001067  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.078080  1646 raft_consensus.cc:2410] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:10.001658  4680 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:09.981014  4714 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:10.003743  4732 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:10.008236  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 48939.9us
I20260709 13:02:09.933923  4745 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:10.013422  2380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.006s	sys 0.009s
I20260709 13:02:10.018584  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.008944  4592 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.039710  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 380) took 1150 ms. Trace:
I20260709 13:02:10.041975  1649 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:09.983518  4715 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:09.961448  4311 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:09.989225  4469 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:10.062011  1651 pending_rounds.cc:107] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:10.102164  1651 raft_consensus.cc:2889] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:10.069180  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72203.5us
I20260709 13:02:10.103170  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:10.106328  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:10.107995  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:10.109620  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.109381  1647 rpcz_store.cc:276] 0709 13:02:08.883043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.883592 (+   549us) service_pool.cc:224] Handling call
0709 13:02:08.890730 (+  7138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.939991 (+1049261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.033000 (+ 93009us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001bb7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:10.033325 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397932416}
W20260709 13:02:10.072487  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.006s	sys 0.025s
I20260709 13:02:10.046231  4754 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:10.094138  2380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.006s	sys 0.009s
W20260709 13:02:10.115315  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.006s	sys 0.025s
I20260709 13:02:10.115283  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:10.119517  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.120263  1721 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:10.048720  4683 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:10.068763  4744 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:10.128365  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:10.117957  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.124454  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.126159  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 357) took 1147 ms. Trace:
I20260709 13:02:10.047637  4675 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.071959  4671 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:10.080746  4742 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.157701  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:10.108701  4745 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:10.146363  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.150629  1721 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.152149  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:10.150544  2380 rpcz_store.cc:276] 0709 13:02:08.970020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.970220 (+   200us) service_pool.cc:224] Handling call
0709 13:02:08.983499 (+ 13279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.986863 (+  3364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.986900 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.000412 (+ 13512us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.077244 (+ 76832us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:09.322668 (+245424us) log.cc:844] Serialized 14 byte log entry
0709 13:02:10.116580 (+793912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.116722 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.116799 (+    77us) raft_consensus.cc:1685] finished
0709 13:02:10.116819 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.117023 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7326,"prepare.queue_time_us":375954,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":3493,"raft.queue_time_us":49721,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":23436,"thread_start_us":1068551,"threads_started":3,"wal-append.queue_time_us":220238}
I20260709 13:02:10.164608  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.078608  4752 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:10.153245  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:10.152228  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:10.131822  1720 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:02:10.151505  4716 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.014s	user 0.018s	sys 0.000s
I20260709 13:02:10.126372  4699 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:10.125490  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.162153  1569 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:10.163638  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.166247  1723 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:10.158632  4671 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.156214  2373 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.129232  4683 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.107585  4469 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:10.149688  4744 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:10.123281  4698 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:10.122998  4747 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:10.179147  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.108526  4714 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:10.210922  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.221321  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.242774  1647 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:10.121063  4707 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:02:10.239895  1572 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:10.250231  1646 raft_consensus.cc:2368] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7.
I20260709 13:02:10.197260  4735 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.200991  4752 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:10.127941  4754 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.231935  4469 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.229660  4699 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.290630  4714 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.292682  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.286235  4542 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.302330  4614 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:10.303822  4762 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:02:10.305685  4758 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:02:10.306800  4764 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:02:10.266232  4698 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:10.313503  4760 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.314625  4751 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.317013  4763 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:02:10.331174  1721 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
W20260709 13:02:10.314049  4765 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:02:10.361608  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:10.337356  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.753s (SENT) [suppressed 24 similar messages]
I20260709 13:02:10.401731  4698 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.200265  4716 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:10.364296  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 127762us
I20260709 13:02:10.404994  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:10.405468  4672 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.405916  4716 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.406186  4672 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.370075  4699 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:10.368542  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.366035  4542 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.371582  4675 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.408814  4675 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:10.408978  4699 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.409451  4675 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.378121  4730 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:10.356922  4750 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.413733  4750 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.285429  4756 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:10.418686  4652 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.001s
I20260709 13:02:10.419189  4311 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:10.419574  4652 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:10.402302  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65007us
I20260709 13:02:10.421697  4542 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:10.406819  2377 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:10.369598  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47895.5us
I20260709 13:02:10.423617  4526 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:10.399267  4738 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:10.425143  4768 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:02:10.424957  4738 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.392987  4751 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.365660  4614 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:10.427858  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.428444  4751 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:10.429165  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.430313  4680 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.401789  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.431962  1646 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:10.414331  4754 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.433138  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.433300  4750 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:10.399075  4752 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.375396  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:10.441702  1645 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:10.448185  1646 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:10.280483  4735 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.448995  2373 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:10.366664  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:10.475921  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 71054.4us
I20260709 13:02:10.412726  4675 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:10.478984  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 112395us
I20260709 13:02:10.487546  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:10.493729  4730 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:10.494297  4730 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:10.495070  4730 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.496286  4675 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.496675  4732 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:10.497519  4675 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.447165  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:10.503680  2375 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:10.506134  1649 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:10.492805  4687 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:10.509048  4687 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:10.510223  4687 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:10.511065  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.511814  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.515331  4735 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.474903  4699 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:10.516781  4675 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.516984  4699 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:10.518193  4738 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.518779  4699 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.453380  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:10.521299  2372 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:10.524856  4739 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.720s	user 0.014s	sys 0.003s
I20260709 13:02:10.525444  4739 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.528667  1648 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:10.529834  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:10.530700  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:10.531032  1721 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:10.533097  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.516620  4716 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:10.534538  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.535995  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:10.545186  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 187105us
I20260709 13:02:10.534650  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.548700  1720 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:10.550185  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.562754  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.563865  1717 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.563673  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.566365  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.569095  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.570086  1719 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:10.574913  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87534.5us
I20260709 13:02:10.567072  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.582638  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.398315  4760 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.490758  4713 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.529871  4652 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.536545  4732 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:10.606022  4604 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:10.606766  4604 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.607717  4756 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:10.608577  1572 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:10.609483  4771 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:02:10.611409  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.612715  1576 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:10.613821  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:10.607177  4675 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.620669  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:10.622735  1539 leader_election.cc:400] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:02:10.623189  1539 leader_election.cc:403] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:02:10.618327  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:10.628499  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:10.637558  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.643815  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.718936  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:10.627056  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:10.721943  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94988.3us
W20260709 13:02:10.727644  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.728894  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.785046  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 62213.2us
I20260709 13:02:10.803613  4659 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:10.804016  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 179776us
I20260709 13:02:10.620193  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:10.804895  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:10.807284  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:10.624922  4733 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.803s	user 0.019s	sys 0.000s
I20260709 13:02:10.809953  4469 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.811558  4775 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:02:10.812951  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.818410  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:10.721196  4542 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:10.826539  1571 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:10.827052  4542 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:10.827350  4745 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.004s
I20260709 13:02:10.828048  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:10.827873  4745 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.828864  1571 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.829394  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.830168  1725 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.831653  4745 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:10.833076  1644 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:10.834127  4439 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:10.835387  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.835570  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:10.616973  4644 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.763s	user 0.015s	sys 0.000s
I20260709 13:02:10.622402  4732 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.811995  4604 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:10.838660  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.881532  1575 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:10.838342  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.883841  1569 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:10.840518  1614 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:10.805048  2375 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.623893  4756 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:10.903183  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 280654us
I20260709 13:02:10.904460  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:10.907600  1569 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:10.844871  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.845199  1571 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:10.909050  1721 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.910151  1571 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:10.876392  1567 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:10.603179  4761 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:10.571383  4683 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.880137  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.853945  1719 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:10.925338  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:10.851106  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:10.926379  1724 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:10.843003  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.889683  1573 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:10.804406  4671 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.004s
W20260709 13:02:10.922843  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.758s (SENT)
I20260709 13:02:10.928623  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:10.941527  1568 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:10.946655  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.804406  4659 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.964954  1725 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
W20260709 13:02:10.902364  4742 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.003s
I20260709 13:02:10.878973  4733 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.845778  4754 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5.
I20260709 13:02:10.984213  1576 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:10.887467  4728 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.920955  4741 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.827s	user 0.003s	sys 0.009s
I20260709 13:02:10.833276  2372 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:10.998236  1651 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.051s	sys 0.104s
W20260709 13:02:10.906374  4592 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.001s
W20260709 13:02:10.979650  4744 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.004s
I20260709 13:02:10.923713  4745 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:10.621438  4713 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:10.996644  1570 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:10.723086  4675 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:10.901885  4756 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:11.001428  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 97167.4us
W20260709 13:02:10.999575  1651 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.051s	sys 0.104s
I20260709 13:02:10.833168  4614 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:10.978619  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.011s	sys 0.043s
I20260709 13:02:10.912528  4644 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:11.024020  1721 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
W20260709 13:02:11.076282  1722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.006s	sys 0.023s
W20260709 13:02:11.076956  1722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.006s	sys 0.023s
I20260709 13:02:11.027881  1717 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:10.979466  4659 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:11.040577  1650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.012s	sys 0.014s
W20260709 13:02:11.041136  1642 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.016s
W20260709 13:02:11.088811  1642 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.009s	sys 0.016s
I20260709 13:02:10.961651  4761 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:11.064982  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.012s	sys 0.043s
W20260709 13:02:11.098554  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 674361us
I20260709 13:02:11.010537  4752 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:11.066612  4756 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:11.120918  4672 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:11.090431  4643 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.127411  4643 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:11.130627  4643 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:10.998061  4728 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:11.040896  4680 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.015s	sys 0.005s
I20260709 13:02:10.975636  4776 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:10.956943  4671 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:11.090667  1650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.012s	sys 0.014s
I20260709 13:02:11.036669  4745 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.046819  4687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.019s
I20260709 13:02:11.034432  4744 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:11.090472  4640 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.414s	user 0.016s	sys 0.002s
I20260709 13:02:11.111073   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:11.089213  4766 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:11.113480  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:11.184116  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70731.2us
W20260709 13:02:11.163889  4770 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:02:11.117877  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1308) took 1204 ms. Trace:
I20260709 13:02:11.120414  4469 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.064276  4715 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:11.041009  4441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.017s	sys 0.005s
I20260709 13:02:11.138233  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1318) took 1088 ms. Trace:
I20260709 13:02:10.997661  4733 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:11.067876  4739 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.169073  4783 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:11.150334  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.218216  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.218664  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.149744  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 342540us
I20260709 13:02:11.017395  4741 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:11.118738  4761 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:11.169613  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 289370us
W20260709 13:02:11.212193  4441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.017s	sys 0.005s
I20260709 13:02:11.204649  4751 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:11.163848  4750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.025s	sys 0.043s
W20260709 13:02:11.093255  4723 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.189s	user 0.012s	sys 0.002s
I20260709 13:02:11.096274  4698 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:02:11.143751  4509 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:11.174369  4687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.008s	sys 0.020s
W20260709 13:02:11.201804  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 361533us
I20260709 13:02:11.202625  1644 rpcz_store.cc:276] 0709 13:02:09.913519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.913733 (+   214us) service_pool.cc:224] Handling call
0709 13:02:09.914256 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.833845 (+919589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.117427 (+283582us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400411ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:11.117808 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2112138496}
I20260709 13:02:11.174669  4745 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:11.212275  1645 rpcz_store.cc:276] 0709 13:02:10.018058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.018224 (+   166us) service_pool.cc:224] Handling call
0709 13:02:11.105656 (+1087432us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b5400306954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:11.105863 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798669696}
I20260709 13:02:11.303575  4643 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.313436  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:11.314328  1645 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:11.314392  4806 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:11.215904  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 291270us
W20260709 13:02:10.965332  4714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.008s	sys 0.015s
I20260709 13:02:11.215907  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1322) took 1136 ms. Trace:
I20260709 13:02:11.220556  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:11.227680  1720 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.003s
I20260709 13:02:11.043924  4735 raft_consensus.cc:2764] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:02:11.174350  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 246145us
I20260709 13:02:11.304054  4797 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:02:11.183980  4732 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:11.160679  4680 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.016s	sys 0.005s
I20260709 13:02:11.247987  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:11.201541  4640 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:11.246625  1724 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
W20260709 13:02:11.250864  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 203355us
I20260709 13:02:11.200174  4793 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:11.104089  4752 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:11.280880  1643 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:11.285975  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1320) took 1226 ms. Trace:
I20260709 13:02:11.255740  4592 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.233727  4715 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:11.505213  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 284819us
W20260709 13:02:11.305047  4788 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:11.306088  1723 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:11.306421  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:11.513664  1644 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:11.306732  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:11.252200  4675 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.307510  1725 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:11.307416  4699 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:11.309471  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:11.181856  4776 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:11.234337  4783 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:11.222103  4648 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.289361  4760 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.318629  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:11.284783  4683 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.341533  4728 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.313462  4643 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:11.339792  4671 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:11.421295  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.254271  4750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.026s	sys 0.044s
I20260709 13:02:11.507643  1720 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:11.397388  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1310) took 1483 ms. Trace:
I20260709 13:02:11.516525  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:11.494370  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 205068us
W20260709 13:02:11.524222  4714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.012s	sys 0.024s
W20260709 13:02:11.524556  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.336972  4741 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:11.348248  4733 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:11.526185  1647 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:11.277737  4526 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:11.514482  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:11.504891  1651 rpcz_store.cc:276] 0709 13:02:10.059708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.059897 (+   189us) service_pool.cc:224] Handling call
0709 13:02:10.061297 (+  1400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:10.119675 (+ 58378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.119721 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:10.127111 (+  7390us) log.cc:844] Serialized 52 byte log entry
0709 13:02:10.268503 (+141392us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:11.048266 (+779763us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.067306 (+ 19040us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.283732 (+216426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.283850 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.283909 (+    59us) raft_consensus.cc:1685] finished
0709 13:02:11.283928 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.285891 (+  1963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51396,"prepare.queue_time_us":353856,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":7609,"raft.queue_time_us":118385,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":32920,"thread_start_us":979736,"threads_started":3,"wal-append.queue_time_us":359463}
I20260709 13:02:11.613473  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:11.537109  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:11.552222  1721 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
W20260709 13:02:11.553686  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.003s	sys 0.034s
I20260709 13:02:11.542531  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:11.349640  4781 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:11.619091  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 1965) took 1012 ms. Trace:
I20260709 13:02:11.621333  2378 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:11.626513  1644 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:11.318370  1646 rpcz_store.cc:276] 0709 13:02:10.079709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.475202 (+395493us) service_pool.cc:224] Handling call
0709 13:02:10.485515 (+ 10313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.493664 (+  8149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.493703 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.494348 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 13:02:10.881768 (+387420us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:10.965278 (+ 83510us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.215490 (+250212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.215594 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.215652 (+    58us) raft_consensus.cc:1685] finished
0709 13:02:11.215671 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.215834 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7515,"prepare.queue_time_us":147627,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":96653,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":24587,"thread_start_us":624882,"threads_started":2,"wal-append.queue_time_us":576963}
I20260709 13:02:11.504045  4752 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:11.349565  4713 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:11.475626  4730 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.980s	user 0.008s	sys 0.007s
I20260709 13:02:11.631748  1649 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:11.343938  2375 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:12.156399  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 640292us
I20260709 13:02:11.709398  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:11.477030  4311 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:12.029367  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 395) took 2099 ms. Trace:
I20260709 13:02:12.135912  1722 rpcz_store.cc:276] 0709 13:02:10.526654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.527435 (+   781us) service_pool.cc:224] Handling call
0709 13:02:10.527824 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.528269 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.528299 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.528932 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.077485 (+548553us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:11.523710 (+446225us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.538577 (+ 14867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.538693 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.538757 (+    64us) raft_consensus.cc:1685] finished
0709 13:02:11.538776 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.538935 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110677,"prepare.queue_time_us":726742,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":94641,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":36035,"thread_start_us":918238,"threads_started":2,"wal-append.queue_time_us":759587}
I20260709 13:02:11.720947  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:11.475533  4799 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.139196  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1967) took 1611 ms (client timeout 1620 ms). Trace:
I20260709 13:02:12.163239  1643 rpcz_store.cc:276] 0709 13:02:09.930488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.933352 (+  2864us) service_pool.cc:224] Handling call
0709 13:02:09.971876 (+ 38524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.522215 (+1550339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.028983 (+506768us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400306e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:12.029274 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3009519872}
I20260709 13:02:11.951552  2374 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:11.721411  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:12.145053  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.631912  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:12.197398  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.197995  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.198060  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 567134us
W20260709 13:02:11.334477  4790 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:11.708771  4732 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.158113  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:11.533569  4604 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.644s
I20260709 13:02:11.531919  4793 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:11.614315  4756 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.159379  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.159737  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.242463  1724 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:11.543725  4795 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:12.161796  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.248997  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.162420  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.164604  1575 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:11.542976  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:12.251214  2377 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:11.982096  4659 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.895s	user 0.011s	sys 0.004s
I20260709 13:02:12.131872  1642 rpcz_store.cc:276] 0709 13:02:09.914201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.915285 (+  1084us) service_pool.cc:224] Handling call
0709 13:02:10.177590 (+262305us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:02:10.177872 (+   282us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:10.202665 (+ 24793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.202713 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:10.220805 (+ 18092us) log.cc:844] Serialized 52 byte log entry
0709 13:02:10.319940 (+ 99135us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18003420a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:02:10.489410 (+169470us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:11.089326 (+599916us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.289750 (+200424us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.396821 (+107071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.396967 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.397028 (+    61us) raft_consensus.cc:1685] finished
0709 13:02:11.397048 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.397287 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":485847,"prepare.queue_time_us":191710,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":222401,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":21217,"spinlock_wait_cycles":40264832,"thread_start_us":913198,"threads_started":3,"wal-append.queue_time_us":355795}
W20260709 13:02:12.170078  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.183306  2375 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 368) took 1609 ms (client timeout 1675 ms). Trace:
W20260709 13:02:12.142778  1716 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.001s
I20260709 13:02:12.269404  1716 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:11.619771  4542 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.180842  1720 rpcz_store.cc:269] 0709 13:02:10.527309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.532291 (+  4982us) service_pool.cc:224] Handling call
0709 13:02:12.139117 (+1606826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.165783  4760 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:12.192788  4744 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.165800  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:12.198354  4783 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.199229  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:02:12.278174  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 79055us
W20260709 13:02:12.279289  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.067s	user 0.061s	sys 0.209s
W20260709 13:02:12.279605  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.061s	sys 0.209s
W20260709 13:02:12.279865  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 1.06767e+06us
W20260709 13:02:12.150002  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.019s	sys 0.055s
I20260709 13:02:11.728021  4796 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.281749  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.267822  1724 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:12.210246  1717 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:11.993698  4804 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:12.283535  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.284132  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.285341  1648 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:12.286154  1568 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:11.827004  4699 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:11.806147  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 268059us
I20260709 13:02:12.184448  4311 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:12.214146  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 56224.3us
W20260709 13:02:12.158957  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 644589us
I20260709 13:02:12.288492  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:12.288915  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:11.333423  4806 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.847s
I20260709 13:02:11.857002  4614 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.294355  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.260028  4803 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:11.350455  4644 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.211198  4741 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:11.560974  4592 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.182300  4802 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.318636  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:12.320531  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1319) took 2279 ms. Trace:
I20260709 13:02:12.287763  4783 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.156723  4526 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.182804  4752 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:12.340234  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:12.230366  4604 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:12.230783  4671 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:12.147334  4648 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.168704  4776 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:12.257732  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.837903  4791 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 13:02:12.240059  4745 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.930s	user 0.010s	sys 0.000s
I20260709 13:02:11.824919  4805 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.417s
I20260709 13:02:12.275763  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.161211  4730 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.283996  1569 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:12.287344  4796 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.284736  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:12.211488  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.081s	sys 0.130s
I20260709 13:02:12.280470  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:11.829272  4737 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:12.290746  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.373572  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.375711  1721 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:12.288460  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:12.376484  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96090.3us
I20260709 13:02:12.292882  1539 leader_election.cc:400] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10.
I20260709 13:02:12.377136  1539 leader_election.cc:403] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:02:12.377233  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.377640  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:12.378000  1720 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:12.378115  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 85363.1us
I20260709 13:02:11.929229  4715 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:11.814174  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:12.299293  4714 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:12.209142  4799 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.305991  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.332962  4614 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:12.285322  4683 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.236876  4732 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.291576  4804 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.310976  1643 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:12.296463  4751 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.042s	user 0.009s	sys 0.005s
I20260709 13:02:12.311468  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:12.323745  1642 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:12.179246  4680 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:02:12.287901  4699 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:12.417459  4699 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.418808  4699 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:12.287292  4760 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:12.420557  4796 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.421124  4760 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.422078  1720 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:12.422782  4648 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.423365  1720 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:12.424058  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2015) took 1106 ms. Trace:
I20260709 13:02:12.424513  4680 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:12.425020  4680 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:12.425797  4680 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.426523  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2016) took 1109 ms. Trace:
I20260709 13:02:12.426976  4680 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:12.426872  1722 rpcz_store.cc:276] 0709 13:02:11.317580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.425704 (+1108124us) service_pool.cc:224] Handling call
0709 13:02:12.426016 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.426190 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.426219 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.426278 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.426309 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.426359 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.426469 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.426941  4735 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:12.428480  4735 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:12.335393  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:12.271246  2375 rpcz_store.cc:269] 0709 13:02:10.570406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.573840 (+  3434us) service_pool.cc:224] Handling call
0709 13:02:12.179103 (+1605263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.273614  4659 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.429337  4735 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.431087  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2092) took 1605 ms. Trace:
I20260709 13:02:12.431757  4659 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:12.341156  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 246600us
I20260709 13:02:12.431702  1575 rpcz_store.cc:276] 0709 13:02:10.826141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.882613 (+ 56472us) service_pool.cc:224] Handling call
0709 13:02:10.885720 (+  3107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.250660 (+1364940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.430825 (+180165us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54000c1a54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:12.430988 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2940793728}
I20260709 13:02:12.434392  4804 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:12.434928  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.839s (SENT)
W20260709 13:02:12.347350  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 67228.5us
I20260709 13:02:12.301858  4807 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:12.368368  4745 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.436753  4807 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:12.437211  1724 raft_consensus.cc:2368] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8.
I20260709 13:02:12.284312  4542 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:12.208441  1645 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.893s	user 0.011s	sys 0.003s
I20260709 13:02:12.340170  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:12.372123  4754 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:12.390064  4737 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.438834  4754 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.439457  4542 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.440060  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2082) took 1914 ms. Trace:
I20260709 13:02:12.445885  4799 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.446055  1570 rpcz_store.cc:276] 0709 13:02:10.526030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.561643 (+ 35613us) service_pool.cc:224] Handling call
0709 13:02:10.996388 (+434745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.041907 (+ 45519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.439872 (+1397965us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54001257d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:12.440003 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998498688}
I20260709 13:02:12.448073  4751 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.253432  4733 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.346818  4671 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:12.317567  4741 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.386010  4643 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.390568  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:12.463369  1721 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:12.376498  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.307724  4803 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.331476  4800 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:02:12.398152  1646 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:12.400331  4526 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.401278  4614 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:12.403884  4795 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.404290  4592 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.404422  4810 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:12.389783  4811 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.405133  4793 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.342552  4802 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.408578  4311 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.395500  4801 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:12.393970  4683 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.405180  4756 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.413720  4730 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.484731  4614 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:02:12.414795  1647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1898) took 2260 ms (client timeout 1866 ms). Trace:
I20260709 13:02:12.301666  4713 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:12.437176  4807 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.321063  1650 rpcz_store.cc:276] 0709 13:02:10.029598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.029815 (+   217us) service_pool.cc:224] Handling call
0709 13:02:10.042764 (+ 12949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.150905 (+108141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.150945 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:10.158005 (+  7060us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.166063 (+1008058us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:11.205706 (+ 39643us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.206069 (+   363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.724033 (+517964us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b54000910d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:11.724100 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:11.836910 (+112810us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b18001bd900. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:12.308491 (+471581us) raft_consensus.cc:1685] finished
0709 13:02:12.308520 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.308687 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7785,"prepare.queue_time_us":56055,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":601,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":41168,"spinlock_wait_cycles":203312128,"thread_start_us":936340,"threads_started":1,"wal-append.queue_time_us":1011807}
I20260709 13:02:12.438335  4745 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.440616  1716 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:12.440876  4754 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.373270  2378 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:12.438336  1645 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:12.441632  4737 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.443821  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:12.444597  2374 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.444718  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:12.429353  1649 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:12.375406  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.130s	user 0.082s	sys 0.131s
I20260709 13:02:12.392516  4783 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:12.424443  1720 rpcz_store.cc:276] 0709 13:02:11.317299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.422884 (+1105585us) service_pool.cc:224] Handling call
0709 13:02:12.423208 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.423765 (+   557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.423991 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.461573  1722 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:12.462122  1725 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:12.466724  1718 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:12.472879  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:12.475504  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.477192  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.482167  1721 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:12.482074  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:12.467163  4733 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.456981  4671 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:12.492486  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.501032  4707 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:12.501930  4801 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:12.503644  4813 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.289s	user 0.000s	sys 0.000s
I20260709 13:02:12.505738  4741 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.505944  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 400) took 1284 ms. Trace:
I20260709 13:02:12.509822  4802 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.491865  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:12.494117  4750 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 13:02:12.539253  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 193677us
W20260709 13:02:12.751421  1647 rpcz_store.cc:269] 0709 13:02:10.154852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.272968 (+118116us) service_pool.cc:224] Handling call
0709 13:02:12.414522 (+2141554us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b5400389fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:12.414686 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169869568}
I20260709 13:02:12.752302  4828 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:12.752638  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:12.752449  4730 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:12.753098  4828 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.753986  4730 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:12.754261  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.754791  4713 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:12.755252  1574 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:12.756053  4828 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.756904  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.754395  4730 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:12.758103  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:12.608460  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.753s (SENT)
W20260709 13:02:12.492005  2372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 367) took 1923 ms (client timeout 2164 ms). Trace:
I20260709 13:02:12.758551  4713 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.758986  4730 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.759212  1646 rpcz_store.cc:276] 0709 13:02:11.221098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.157333 (+936235us) service_pool.cc:224] Handling call
0709 13:02:12.157642 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.481561 (+323919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.505668 (+ 24107us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54003fc354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:12.505876 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542045952}
I20260709 13:02:12.760528  4730 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.760854  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2017) took 1426 ms. Trace:
I20260709 13:02:12.761150  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:12.761693  4801 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:12.761936  1644 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:12.762506  1568 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:12.526319  4752 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:12.763391  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.764143  1568 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:12.764227  4752 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.765341  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.766048  1575 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:12.766825  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.759220  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:12.767915  1573 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:12.767385  1575 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:12.768497  4707 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:12.541235  4739 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:12.769307  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.769631  4739 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:12.769814  4707 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.770470  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.770588  4739 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.771200  4707 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:12.771720  4739 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:12.620836  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.773592  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.582016  4815 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:02:12.759353  2372 rpcz_store.cc:269] 0709 13:02:10.565170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.568902 (+  3732us) service_pool.cc:224] Handling call
0709 13:02:12.486076 (+1917174us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54005fee54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:12.488454 (+  2378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260152576}
I20260709 13:02:12.494944  4810 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.777751  4761 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:12.778458  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:12.778373  4761 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.779446  1718 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:12.779923  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:12.678944  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 289267us
I20260709 13:02:12.793174  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:12.793424  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:12.794605  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:12.624253  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.761296  1722 rpcz_store.cc:276] 0709 13:02:11.334235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.428138 (+1093903us) service_pool.cc:224] Handling call
0709 13:02:12.461396 (+ 33258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.760488 (+299092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.760794 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.522929  4756 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.781095  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 398) took 1560 ms (client timeout 1866 ms). Trace:
W20260709 13:02:12.797823  1645 rpcz_store.cc:269] 0709 13:02:11.220554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.313003 (+ 92449us) service_pool.cc:224] Handling call
0709 13:02:12.781033 (+1468030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.616168  1723 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:12.737053  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.780624  4741 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:12.774288  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.781463  4828 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:12.787359  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 178957us
I20260709 13:02:12.782162  1720 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:12.544927  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:12.780902  4761 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.761855  1570 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:12.778053  4733 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:12.791144  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 347418us
I20260709 13:02:12.721215  4776 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10.
I20260709 13:02:12.805750  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:12.826503  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:12.831342  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.835335  1568 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:12.679174  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 207928us
I20260709 13:02:12.792636  4737 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:12.849522  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.800745  4716 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:12.850462  4716 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:12.850598  4810 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.819666  4741 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:12.851449  4716 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.851966  4741 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.801456  1722 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:12.853406  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2020) took 1502 ms. Trace:
I20260709 13:02:12.853785  1722 rpcz_store.cc:276] 0709 13:02:11.351302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.800931 (+1449629us) service_pool.cc:224] Handling call
0709 13:02:12.801290 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.853143 (+ 51853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.853349 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.816641  4732 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:12.796209  2378 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:12.854883  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:12.855640  1722 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:12.856971  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2021) took 1505 ms. Trace:
I20260709 13:02:12.857381  1722 rpcz_store.cc:276] 0709 13:02:11.351647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.856173 (+1504526us) service_pool.cc:224] Handling call
0709 13:02:12.856505 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.856674 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.856696 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.856744 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:12.856779 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.856826 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.856924 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.808149  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2019) took 1457 ms. Trace:
I20260709 13:02:12.860008  1720 rpcz_store.cc:276] 0709 13:02:11.350895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.781706 (+1430811us) service_pool.cc:224] Handling call
0709 13:02:12.781994 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.807874 (+ 25880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.808094 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.861323  4683 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:12.862952  1722 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:12.863612  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2022) took 1511 ms. Trace:
I20260709 13:02:12.863864  1722 rpcz_store.cc:276] 0709 13:02:11.351988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.862402 (+1510414us) service_pool.cc:224] Handling call
0709 13:02:12.862749 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.863375 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.863555 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.816884  4614 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.801237  4823 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.866046  4614 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:12.808735  1572 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:12.780073  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2018) took 1436 ms. Trace:
I20260709 13:02:12.801826  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:12.808519  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.180s (SENT)
W20260709 13:02:12.870890  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62371.4us
W20260709 13:02:12.871515  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.845s (SENT)
I20260709 13:02:12.871865  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:12.819305  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:12.809280  1539 leader_election.cc:400] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7.
I20260709 13:02:12.810720  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:12.875604  1539 leader_election.cc:403] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:02:12.806221  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.876601  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.869419  1718 rpcz_store.cc:276] 0709 13:02:11.350301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.778957 (+1428656us) service_pool.cc:224] Handling call
0709 13:02:12.779274 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.779851 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.780022 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.820747  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:12.758903  1721 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:12.795153  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:12.879379  1644 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:12.833367  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:12.880542  1642 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:12.824572  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.877545  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:12.817874  4801 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:12.820070  4592 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:12.889153  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55884.1us
I20260709 13:02:12.890440  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:12.894241  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:12.842643  1570 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:12.843783  4761 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:12.808385  4756 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.756121  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.901988  1574 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:12.902990  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.816156  4802 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:12.838353  1573 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:12.868235  4750 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:12.915513  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:12.898236  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 89086.7us
W20260709 13:02:12.905558  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 94958.2us
I20260709 13:02:12.916849  1647 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:12.917791  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:12.919728  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50351.4us
I20260709 13:02:12.908525  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.912484  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.906816  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:12.897226  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.923966  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2014) took 1594 ms. Trace:
I20260709 13:02:12.923480  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.882936  4809 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.920331  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:12.928222  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:12.924371  1719 rpcz_store.cc:276] 0709 13:02:11.316820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.316968 (+   148us) service_pool.cc:224] Handling call
0709 13:02:11.341904 (+ 24936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.342438 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.342470 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.343225 (+   755us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.636138 (+292913us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:11.636213 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.636325 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.503021 (+866696us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:12.897534 (+394513us) raft_consensus.cc:1685] finished
0709 13:02:12.897554 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.911009 (+ 13455us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1592,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"spinlock_wait_cycles":5213056,"thread_start_us":292395,"threads_started":1,"wal-append.queue_time_us":647053}
I20260709 13:02:12.929430  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:12.932211  1574 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:12.935760  1567 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:12.897038  4737 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.938436  1574 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:12.901428  4526 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.910377  4542 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:12.939961  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.820036  4828 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.910657  4795 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:12.903610  4793 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:12.914947  4730 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:12.948124  1649 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:12.907953  4821 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.922318  4823 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:12.954118  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.907024  4800 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 13:02:12.970582  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.989364  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:12.980355  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:12.980115  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.911823  4761 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:12.992357  1572 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:12.931394  4760 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:12.990046  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:12.906472  4713 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:02:12.949621  4542 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.978264  4793 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.940752  4715 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:02:12.980856  4680 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:12.943871  4714 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:13.015139  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.034973  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.055538  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.034786  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:12.998253  4542 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:13.031945  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:12.987231  4795 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:13.029909  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:12.996371  4760 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.045526  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.002223  4730 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:13.209242  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 179403us
I20260709 13:02:13.014678  4793 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.047583  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.227114  1646 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.219475  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:12.952230  4737 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:12.986385  4687 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:13.002079  4742 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:13.047624  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.240862  1572 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:13.060132  1649 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:13.247546  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.062825  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:13.057351  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 133913us
I20260709 13:02:13.249305  1647 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:13.250474  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.251474  1647 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:13.077555  1642 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.078703  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.089483  4713 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:13.256958  1644 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:13.256839  4713 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:13.178445  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.259152  4713 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:13.076900  1650 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:13.005270  4801 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:02:13.003953  4822 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:02:13.248385  1649 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:13.228909  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:13.225726  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.208511  4795 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.210362  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:13.210232  4760 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:13.210630  4714 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:13.218817  4699 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.800s	user 0.015s	sys 0.000s
I20260709 13:02:13.263096  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:13.280562  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.059576  4807 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.286506  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.287266  2375 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:13.287458  4735 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.292964  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.294979  4745 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.295519  4745 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.296473  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.297222  1572 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:13.298004  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.298934  4803 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:13.299137  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.299705  1572 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:13.300446  4745 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:13.302688  4834 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
W20260709 13:02:13.304457  4671 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.524s	user 0.003s	sys 0.006s
I20260709 13:02:13.305003  4671 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.305733  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.306114  1574 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:13.306869  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.307570  1575 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:13.308740  4756 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:13.309001  4832 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:02:13.309688  4671 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:13.310235  4830 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.365s	user 0.000s	sys 0.000s
I20260709 13:02:13.310482  4761 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.312384  4839 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:13.312788  4839 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:13.313575  4839 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.314690  4839 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:13.314857  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51845.6us
W20260709 13:02:13.315950  2374 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.001s
I20260709 13:02:13.308207  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.218146  4838 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.318317  4648 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:13.318765  4648 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:13.318893  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:13.318358  4837 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:13.320040  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.320637  4648 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.321095  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.321214  4828 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.321614  4648 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.322129  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.323434  1568 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:13.325723  1570 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:13.326656  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 373) took 1013 ms. Trace:
I20260709 13:02:13.327127  1570 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:13.327023  2378 rpcz_store.cc:276] 0709 13:02:12.313289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.313462 (+   173us) service_pool.cc:224] Handling call
0709 13:02:13.326482 (+1013020us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b54006fe0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.326572 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74480256}
W20260709 13:02:13.328745  4833 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 13:02:13.329051  4714 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:13.332911  4835 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.287s	user 0.000s	sys 0.000s
I20260709 13:02:13.333273  1722 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:13.335624  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2030) took 1457 ms. Trace:
I20260709 13:02:13.237329  4526 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:13.336493  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2023) took 1984 ms (client timeout 2011 ms). Trace:
W20260709 13:02:13.336936  1725 rpcz_store.cc:269] 0709 13:02:11.352320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.618979 (+266659us) service_pool.cc:224] Handling call
0709 13:02:13.332756 (+1713777us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54003dccd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.336434 (+  3678us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656011136}
I20260709 13:02:13.209649  4592 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:02:13.339480  4807 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:13.340462  1576 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:13.343286  4739 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.000s
I20260709 13:02:13.343616  1720 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:13.345484  4737 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.317035  4732 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.354918  1646 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:13.356791  1568 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:13.350838  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.352308  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:13.352355  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 338618us
I20260709 13:02:13.364921  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:13.336026  1722 rpcz_store.cc:276] 0709 13:02:11.878838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.881716 (+1002878us) service_pool.cc:224] Handling call
0709 13:02:13.332879 (+451163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.333686 (+   807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.335299 (+  1613us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54003dccd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.335525 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037001600}
I20260709 13:02:13.371804  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.347492  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.372977  1644 raft_consensus.cc:2376] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 7. Re-sending same reply.
W20260709 13:02:13.373792  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.373936  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:13.374382  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.374979  1644 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:13.285398  1647 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:13.376528  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.344209  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2029) took 1580 ms. Trace:
I20260709 13:02:13.378257  1720 rpcz_store.cc:276] 0709 13:02:11.764064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.342963 (+1578899us) service_pool.cc:224] Handling call
0709 13:02:13.343293 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.343965 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.344144 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:13.379141  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.385522  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.386556  1572 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 5: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:13.271979  4644 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.407899  1646 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:13.424616  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.425678  1568 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:13.419816  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:13.427119  1644 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:13.427770  2378 raft_consensus.cc:2376] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7. Re-sending same reply.
I20260709 13:02:13.435835  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:13.415422  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.441999  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:13.409731  4441 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:13.443086  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:13.424163  4745 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:13.447646  4745 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.448081  4745 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.450124  4735 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:13.450675  4735 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:13.451086  4735 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.455382  4739 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:13.457764  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.373055  2374 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:13.460497  4699 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:13.461439  4745 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.462383  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 371) took 1163 ms. Trace:
I20260709 13:02:13.462826  2374 rpcz_store.cc:276] 0709 13:02:12.298872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.301516 (+  2644us) service_pool.cc:224] Handling call
0709 13:02:13.462218 (+1160702us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b54005c30d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.462324 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13700480}
I20260709 13:02:13.427059  4756 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.464450  4745 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.466117  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2102) took 1184 ms. Trace:
I20260709 13:02:13.466863  4441 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:13.466696  1576 rpcz_store.cc:276] 0709 13:02:12.281953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.284640 (+  2687us) service_pool.cc:224] Handling call
0709 13:02:12.300261 (+ 15621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.341045 (+1040784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.465767 (+124722us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540027d654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:13.466038 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391837952}
I20260709 13:02:13.467952  4441 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.471475  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.471628  4793 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.472393  1568 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:13.472555  4730 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.472983  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.473295  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.474896  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.475878  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.476779  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.478205  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.478894  1647 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:13.482872  1642 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:13.443425  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:13.492666  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.508322  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.509176  1642 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:13.514424  1644 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:13.486406  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.524312  1572 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:13.527037  1642 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:13.521497  1576 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:13.528426  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.535272  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:13.536387  2378 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:13.542893  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.548269  1575 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:13.538489  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:13.551721  4821 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 13:02:13.551864  2376 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:13.552311  4821 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7.
I20260709 13:02:13.520466  4526 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.559018  4526 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.561131  2378 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:13.561439  4439 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:13.561825  4526 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.567092  4838 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:13.568140  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.503257  4739 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.570288  4739 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.570766  4795 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.572576  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.576402  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2008) took 2272 ms. Trace:
I20260709 13:02:13.576944  1717 rpcz_store.cc:276] 0709 13:02:11.304112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.308469 (+  4357us) service_pool.cc:224] Handling call
0709 13:02:11.308907 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.282765 (+973858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.576038 (+1293273us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b540010e0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:13.576282 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072193408}
I20260709 13:02:13.577566  4760 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.585340  1573 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:13.578414  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2031) took 1672 ms (client timeout 2035 ms). Trace:
I20260709 13:02:13.579402  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 397) took 2359 ms. Trace:
W20260709 13:02:13.593851  1724 rpcz_store.cc:269] 0709 13:02:11.905712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.290930 (+385218us) service_pool.cc:224] Handling call
0709 13:02:13.578183 (+1287253us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b54001a77d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.578349 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114636544}
I20260709 13:02:13.594375  1648 rpcz_store.cc:276] 0709 13:02:11.220123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.220245 (+   122us) service_pool.cc:224] Handling call
0709 13:02:11.247451 (+ 27206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.288177 (+1040726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.579126 (+1290949us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001ba3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:13.579333 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2384268672}
W20260709 13:02:13.597663  1651 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.073s	sys 0.135s
W20260709 13:02:13.597987  1651 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.073s	sys 0.135s
I20260709 13:02:13.522125  4542 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.619037  2371 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:13.602622  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:13.621827  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:13.624193  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:13.625821  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9960143900000276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 13:02:13.626479  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9906143900000188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 13:02:13.628522  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:13.627069  1539 connection.cc:256] RPC call timeout handler was delayed by -1.8961143899999797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 13:02:13.629487  1539 connection.cc:256] RPC call timeout handler was delayed by -1.8115143900000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 13:02:13.630177  1539 connection.cc:256] RPC call timeout handler was delayed by -1.7935143899999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 13:02:13.630650  1539 connection.cc:256] RPC call timeout handler was delayed by -1.787214390000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
W20260709 13:02:13.631111  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6747143900000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 13:02:13.631572  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5865143899999907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:02:13.632015  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3966143900000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
I20260709 13:02:13.633039  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:13.644464  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.645637  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.561846  4439 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:13.524888  4680 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:13.651491  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:13.652236  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2027) took 2110 ms (client timeout 2101 ms). Trace:
I20260709 13:02:13.655622  1570 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:13.682298  1642 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:13.647603  4799 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:13.629007  4614 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:13.657125  4671 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:13.657804  1645 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.005s	sys 0.011s
I20260709 13:02:13.652298  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.663368  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:13.657266  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:13.680305  1723 rpcz_store.cc:269] 0709 13:02:11.541886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.215357 (+673471us) service_pool.cc:224] Handling call
0709 13:02:13.652074 (+1436717us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.652175 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569323776}
I20260709 13:02:13.627811  4754 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:13.647187  4741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.006s	sys 0.009s
W20260709 13:02:13.659610  4820 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.012s	sys 0.022s
I20260709 13:02:13.734261  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.654423  4809 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:13.684587  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 274313us
W20260709 13:02:13.735466  1645 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.005s	sys 0.015s
I20260709 13:02:13.647966  4840 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.751535  1722 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:13.734522  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71284.3us
I20260709 13:02:13.662228  4821 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:13.734329  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77169.9us
I20260709 13:02:13.752899  1717 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 8. Ops: []
I20260709 13:02:13.753185  1720 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:13.740901  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:13.684013  4752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.000s	sys 0.016s
W20260709 13:02:13.736114  4741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.006s	sys 0.012s
I20260709 13:02:13.683872  4737 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:02:13.755975  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:13.757305  4716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.006s	sys 0.017s
W20260709 13:02:13.737370  4836 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:13.736181  4824 raft_consensus.cc:2764] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:02:13.750387  4754 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:13.806758  4754 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:13.810921  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.780597  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:13.701004  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.750803  4707 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:13.753654  4850 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.756816  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.750677  4739 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:13.750867  4793 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.953814  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:13.780937  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 141927us
I20260709 13:02:13.781368  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:13.783334  4644 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:13.784228  4680 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.785204  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.960098  1575 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.785480  1647 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:13.785888  4737 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:13.786104  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 480) took 1325 ms. Trace:
I20260709 13:02:13.786761  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:13.962081  2378 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.786981  4542 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:13.787635  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:13.789139  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:13.791988  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.752592  4742 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:02:13.966986  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:13.961892  2371 rpcz_store.cc:276] 0709 13:02:12.460398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.463119 (+  2721us) service_pool.cc:224] Handling call
0709 13:02:13.785901 (+1322782us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400457754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.786040 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485224832}
I20260709 13:02:13.794461  1574 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:13.797291  4760 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:13.763576  4795 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:13.797039  4735 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.799852  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:13.933038  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:13.973510  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 173759us
I20260709 13:02:13.974326  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:13.812734  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.976698  1644 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:13.943483  4848 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.977638  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:13.916996  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:13.775192  4752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.001s	sys 0.018s
I20260709 13:02:13.945134  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:13.946292  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:13.983232  2378 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:13.983628  1718 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:13.984027  2380 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:13.968168  4754 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.814194  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:13.994825  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:13.814610  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2037) took 1842 ms (client timeout 1915 ms). Trace:
I20260709 13:02:13.912391  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:13.958602  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.964852  1568 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:13.996295  1568 pending_rounds.cc:85] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:13.996208  1717 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:13.996783  1568 pending_rounds.cc:107] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:13.997058  1568 raft_consensus.cc:2889] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:13.995872  1716 rpcz_store.cc:269] 0709 13:02:11.972650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.277151 (+304501us) service_pool.cc:224] Handling call
0709 13:02:13.814450 (+1537299us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540010ead4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:13.814552 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335013632}
I20260709 13:02:13.969646  4793 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.964867  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 184384us
I20260709 13:02:13.999928  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:14.003675  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:14.004455  1647 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:14.009764  1717 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:02:13.943163  4439 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.988721  4707 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:13.986966  4739 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:02:13.987892  4799 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:14.010653  1648 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:14.010540  1575 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:13.966966  2377 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:13.967826  1719 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:13.970060  1723 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:14.026851  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:14.027442  1575 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:13.780606  4716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.006s	sys 0.017s
I20260709 13:02:13.988919  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:13.817430  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:13.989547  4683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.014s	sys 0.016s
I20260709 13:02:13.965463  4850 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:13.990458  1724 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:13.967480  4737 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:13.814528  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:14.022360  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:14.011201  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1431) took 1533 ms. Trace:
I20260709 13:02:14.089783  1644 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:14.029065  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:14.029351  1569 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:13.972476  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:14.038225  1575 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:14.104751  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.104315  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 99267us
W20260709 13:02:14.091501  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62535.2us
I20260709 13:02:14.143087  1572 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:14.161731  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:14.153137  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 163783us
I20260709 13:02:14.090173  1571 rpcz_store.cc:276] 0709 13:02:12.477895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.478039 (+   144us) service_pool.cc:224] Handling call
0709 13:02:12.488906 (+ 10867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.489172 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:12.791617 (+302445us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.302167 (+510550us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:14.002192 (+700025us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b180015a2a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287
0709 13:02:14.010814 (+  8622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.010882 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:14.010942 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.011013 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.011142 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8011,"prepare.queue_time_us":230702,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":3012,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":13492,"spinlock_wait_cycles":29663872,"thread_start_us":439795,"threads_started":3,"wal-append.queue_time_us":235958}
I20260709 13:02:14.159370  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:13.988546  4614 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 13:02:13.988379  4840 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.154s
I20260709 13:02:14.167471  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:14.199247  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:14.089267  4683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.022s	sys 0.027s
W20260709 13:02:14.277891  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 113864us
I20260709 13:02:14.170027  1720 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:14.166811  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:14.280402  1724 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:14.165129  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 138862us
I20260709 13:02:14.283655  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:14.297595  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:14.186511  1716 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:14.299163  1720 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:14.300065  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.168179  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.168558  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.195999  4714 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:14.305972  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1458) took 1004 ms. Trace:
I20260709 13:02:14.306375  1574 rpcz_store.cc:276] 0709 13:02:13.301283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.307985 (+  6702us) service_pool.cc:224] Handling call
0709 13:02:14.305806 (+997821us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b5400364cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:14.305919 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097047936}
I20260709 13:02:14.308386  1644 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:14.309072  1720 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:14.041668  4752 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:14.316603  1719 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:14.027626  4824 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:13.992146  4848 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.310892  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:14.304248  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:13.990792  4648 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.003s
W20260709 13:02:14.019732  4732 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.003s
I20260709 13:02:14.314886  1724 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:14.310356  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 113786us
I20260709 13:02:14.142932  4783 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.017362  4439 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.158839  4776 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:14.020052  4707 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.355640  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:14.351249  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:14.357008  1642 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:14.361420  1643 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:14.364794  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:14.366384  1719 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:14.366567  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:14.039067  4821 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:14.385372  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.412190  1724 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:14.020460  4793 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.342736  4799 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:14.196667  4756 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 13:02:14.042600  4741 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.312s
I20260709 13:02:14.316113  4714 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:14.421185  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.430804  1644 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:14.435545  1651 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:14.143596  4739 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.235s
I20260709 13:02:14.438905  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.003929  4754 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.364475  4707 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.364696  4730 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:14.448742  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.403800  4810 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:14.335089  4806 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.102s
I20260709 13:02:14.365187  4801 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:02:14.359588  4687 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:02:14.316269  4737 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.401710  4783 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:14.024286  4760 consensus_queue.cc:1175] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd6d377efe514bc58ae49c3f4a107232" responder_term: 6 status { last_received { term: 5 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:14.421567  4732 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:14.468703  4849 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.127s
I20260709 13:02:14.590875  4783 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:14.427596  4817 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:14.494313  4735 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:14.591792  4817 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:14.430902  4824 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:14.530504  4866 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 13:02:14.463047  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.600486  1718 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:14.602058  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.602798  1718 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:14.561960  4867 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:02:14.439821  4870 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:14.441031  4846 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.470185  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.424242  4855 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.500093  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 202577us
I20260709 13:02:14.616279  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:14.439311  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:14.447913  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 285009us
I20260709 13:02:14.601311  4824 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:14.588268  1717 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:14.591106  4805 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:14.503824  4441 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.469280  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.592569  1725 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:14.499815  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 220214us
I20260709 13:02:14.557634  1722 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:14.621407  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:14.625669  4875 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:14.626206  4875 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:14.627117  4875 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:14.628916  4860 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:02:14.630515  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 409) took 1351 ms. Trace:
I20260709 13:02:14.608233  4870 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:14.630985  1649 rpcz_store.cc:276] 0709 13:02:13.279188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.351379 (+ 72191us) service_pool.cc:224] Handling call
0709 13:02:13.362731 (+ 11352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.374717 (+ 11986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.374745 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.409528 (+ 34783us) log.cc:844] Serialized 26 byte log entry
0709 13:02:13.870446 (+460918us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:13.870505 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.870608 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.630209 (+759601us) raft_consensus.cc:1685] finished
0709 13:02:14.630236 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.630427 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18103,"raft.queue_time_us":12114,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":17581,"thread_start_us":388150,"threads_started":1,"wal-append.queue_time_us":933435}
I20260709 13:02:14.633908  4441 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.595629  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:14.454963  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.636122  4793 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:14.636941  4754 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:14.637110  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:14.639562  4805 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:14.640532  4783 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:14.641043  4783 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.641589  4783 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:14.640926  4805 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.642760  4805 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:14.646803  1719 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:14.654796  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 217484us
W20260709 13:02:14.562482  4861 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 13:02:14.593290  4859 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:02:14.605018  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.701514  1718 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:14.552094  4863 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.392s	user 0.001s	sys 0.000s
I20260709 13:02:14.561185  4807 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:14.544664  4854 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:02:14.495216  4850 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:14.617790  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:14.477582  1720 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:14.607404  4810 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:14.620445  4735 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:14.700642  4592 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:14.695439  4793 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.617297  4848 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:14.721724  1720 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:14.716836  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 99139.4us
I20260709 13:02:14.722855  4592 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:14.557344  4799 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.728488  4592 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:14.729880  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:14.706645  4807 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:14.733074  4839 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:14.733748  4807 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.734403  4839 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:14.597940  4817 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.736578  4807 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:14.736510  4873 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:14.737607  4671 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:14.737998  4671 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:14.722997  4441 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:14.738883  4671 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.633452  4870 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.728399  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:14.723064  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:14.740741  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1348) took 1661 ms. Trace:
I20260709 13:02:14.751204  1650 rpcz_store.cc:276] 0709 13:02:13.078845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.260043 (+181198us) service_pool.cc:224] Handling call
0709 13:02:13.341238 (+ 81195us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:02:13.558275 (+217037us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49e9c 00007fc830c49a5b 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:02:13.558477 (+   202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.558818 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.558843 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.559314 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:02:13.778805 (+219491us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:13.908532 (+129727us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.942918 (+ 34386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.943000 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.740422 (+797422us) raft_consensus.cc:1685] finished
0709 13:02:14.740481 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.740662 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68164,"prepare.queue_time_us":127907,"raft.queue_time_us":45486,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":15088,"spinlock_wait_cycles":119112064,"thread_start_us":219022,"threads_started":1,"wal-append.queue_time_us":416000}
W20260709 13:02:14.741657  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.866s (SENT)
I20260709 13:02:14.499881  4714 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.743569  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:14.698120  4687 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.743837  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:14.742930  4744 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:14.721437  4735 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:14.749354  1720 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:14.618602  4846 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.619266  4855 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.720889  4876 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:14.701054  4741 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:02:14.761066  4592 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.703528  4806 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:14.701203  4739 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:02:14.744421  4683 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:14.773911  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 472408us
I20260709 13:02:14.776638  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:14.741137  4793 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:14.790634  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.773141  4776 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:14.794773  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.773699  4778 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.782384  4839 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:14.784966  4671 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.799005  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:14.812870  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:14.778455  4744 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.785790  4752 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:02:14.779057  4687 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.822438  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:14.822706  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:14.818799  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:14.860986  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65040.4us
I20260709 13:02:14.875278  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:14.878921  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.880398  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:14.882246  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:14.883104  1642 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:14.885380  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:14.819453  4882 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:14.938473  4882 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:14.816124  4839 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:14.942513  4839 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:14.817557  4756 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:14.948040  4756 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:14.949358  4756 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.951131  4756 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:14.812235  4614 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:02:14.823449  4828 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:14.954555  4799 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:14.972970  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.696s (SENT)
I20260709 13:02:14.814374  4683 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:14.975409  1643 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:14.977738  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:14.811877  4732 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:14.989877  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.828821  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:14.881269  1647 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:15.025190  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:15.004076  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.806243  4439 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:02:15.007031  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:15.026886  1651 raft_consensus.cc:2410] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:15.029471  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51835.1us
I20260709 13:02:15.031338  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:15.032615  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.976137  4856 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.045643  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 226805us
W20260709 13:02:15.016233  4878 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 13:02:15.059253  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:15.073465  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 485) took 1115 ms. Trace:
I20260709 13:02:15.074054  2380 rpcz_store.cc:276] 0709 13:02:13.958011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.958144 (+   133us) service_pool.cc:224] Handling call
0709 13:02:13.983799 (+ 25655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.984440 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.073166 (+1088726us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b54004acc50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:15.073373 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36371328}
I20260709 13:02:14.807739  4778 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.035989  4888 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.092648  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:15.035519  4803 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:15.037266  4441 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.095842  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:15.157802  1568 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.051168  4604 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:14.874181  4735 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.051015  4683 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:14.874504  4810 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.055711  4880 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:15.063525  4871 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.446s	user 0.001s	sys 0.000s
W20260709 13:02:15.215009  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 122438us
I20260709 13:02:15.055254  2374 raft_consensus.cc:2376] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6. Re-sending same reply.
I20260709 13:02:15.063186  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:15.152930  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:15.089273  4839 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.028389  4799 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:14.939779  4882 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.101080  4850 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:15.114255  4876 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:15.239907  4542 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.304438  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2127) took 1348 ms. Trace:
I20260709 13:02:15.262964  4751 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.065521  4856 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.247701  4880 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:15.216308  4680 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:15.318851  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68003us
I20260709 13:02:15.341620  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:15.316709  4901 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:15.092447  4776 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:15.272800  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.275820  4855 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:15.159080  4803 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:15.330926  4846 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:15.335757  4783 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.694s	user 0.009s	sys 0.000s
I20260709 13:02:15.304628  4592 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:15.218957  4441 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:15.378990  4807 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.642s	user 0.013s	sys 0.001s
W20260709 13:02:15.339777  4890 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:02:15.375324  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:15.360255  4880 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:15.337069  4896 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:02:15.415711  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.288458  4839 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:15.406980  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 254161us
I20260709 13:02:15.329401  4751 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:15.414309  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.168074  4888 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:15.427767  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.552s (SENT) [suppressed 16 similar messages]
I20260709 13:02:15.428157  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:15.405203  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.002s	sys 0.006s
W20260709 13:02:15.429518  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.003s	sys 0.006s
W20260709 13:02:15.429885  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 554700us
I20260709 13:02:15.433542  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:15.434651  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:15.436522  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:15.442625  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.445116  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:15.445533  1651 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:02:15.417212  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 264679us
I20260709 13:02:15.421460  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:15.412081  4846 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.407404  4889 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:15.379165  4901 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.396068  4783 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.422492  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 484) took 1457 ms. Trace:
I20260709 13:02:15.562369  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1922) took 1780 ms. Trace:
I20260709 13:02:15.604540  1648 rpcz_store.cc:276] 0709 13:02:13.782284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.784433 (+  2149us) service_pool.cc:224] Handling call
0709 13:02:13.784757 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.038785 (+254028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.562129 (+1523344us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540005c8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:15.562303 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502541568}
I20260709 13:02:15.557528  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:15.540853  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 119455us
I20260709 13:02:15.607601  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:15.469223  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.477335  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.541376  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.429093  4751 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.556510  4900 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:15.450575  4739 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.007s	sys 0.001s
W20260709 13:02:15.613147  4739 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.009s	sys 0.006s
W20260709 13:02:15.556655  4892 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:02:15.603503  1568 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:15.619975  1575 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:02:15.624493  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.555285  4891 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:02:15.627692  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.628077  1722 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:15.557628  4897 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.629791  1719 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:15.561792  4735 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.625129  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.621661  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.402288  4683 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.425779  4687 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.439147  4850 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:15.637370  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.637873  1722 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:15.438586  4849 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.626801  4901 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:15.587239  4846 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.641536  1575 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:15.557405  4887 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:15.614607  4894 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:15.664808  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45899.3us
W20260709 13:02:15.639608  2345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.002s	sys 0.014s
W20260709 13:02:15.681023  2345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.003s	sys 0.015s
I20260709 13:02:15.661250  4778 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:15.667268  1571 raft_consensus.cc:2410] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:15.593642  4614 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.645195  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 210614us
W20260709 13:02:15.685707  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 69276us
I20260709 13:02:15.686414  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:15.622228  4761 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.670737  1722 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:15.652209  4739 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:15.678773  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:15.682273  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.665868  4855 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.665567  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 237812us
I20260709 13:02:15.687486  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.425294  4876 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.676838  4901 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:15.690807  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:15.695298  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.928s (SENT)
I20260709 13:02:15.681213  4900 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.595345  4889 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.699652  4900 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:15.699739  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92183.2us
I20260709 13:02:15.701735  4889 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.701968  4900 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.673055  4849 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.705899  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:15.682063  4799 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.716951  4903 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.694334  4614 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.720312  4901 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:15.683015  4894 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.723306  4754 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:15.725178  1722 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:02:15.734697  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56008.6us
I20260709 13:02:15.717552  4913 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.734874  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 48545.3us
I20260709 13:02:15.778954  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:15.719098  4912 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:15.729491  4778 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:15.734658  4754 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:15.807760  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.696228  4855 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.618340  2375 rpcz_store.cc:276] 0709 13:02:13.957628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.963956 (+  6328us) service_pool.cc:224] Handling call
0709 13:02:13.991180 (+ 27224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.020277 (+ 29097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.020310 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.021071 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.468821 (+447750us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:14.469965 (+  1144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.521689 (+ 51724us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:15.397314 (+875625us) raft_consensus.cc:1685] finished
0709 13:02:15.401462 (+  4148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.415184 (+ 13722us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":422726,"threads_started":1,"wal-append.queue_time_us":531275}
I20260709 13:02:15.790614  4542 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:15.789680  4761 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:02:15.778262  4856 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:15.781553  4912 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.734645  4903 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.783296  4913 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:15.928115  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.305035  1567 rpcz_store.cc:276] 0709 13:02:13.956173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.005112 (+ 48939us) service_pool.cc:224] Handling call
0709 13:02:14.005376 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.005551 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:14.785117 (+779566us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.303975 (+518858us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.304054 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:15.304115 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.304184 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.304325 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87786,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":76282,"thread_start_us":964547,"threads_started":3,"wal-append.queue_time_us":510568}
W20260709 13:02:15.993237  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.033094  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:16.002629  4817 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.034825  2374 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:16.035401  1570 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:02:15.987087  4824 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.284s	user 0.006s	sys 0.007s
I20260709 13:02:16.036373  2372 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:16.036721  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:16.036944  4824 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.037844  4928 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:16.038187  1573 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:15.992973  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:15.826387  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47571.5us
I20260709 13:02:16.041057  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:16.041177  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:16.040659  1718 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:16.042459  1570 raft_consensus.cc:2410] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:02:15.889897  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 202561us
I20260709 13:02:15.867223  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:16.047679  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 268694us
I20260709 13:02:16.049081  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:16.015628  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:16.049580  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:16.050091  2377 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:16.028995  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.051322  1648 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:16.009784  4732 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.957s	user 0.010s	sys 0.003s
W20260709 13:02:16.006532  4671 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.197s	user 0.004s	sys 0.008s
I20260709 13:02:16.058503  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:16.059341  1570 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:15.786432  4441 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:15.962864  4756 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.941s	user 0.006s	sys 0.006s
I20260709 13:02:16.052855  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 381) took 1226 ms. Trace:
I20260709 13:02:16.038453  4928 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.789263  4901 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.055213  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:15.787674  4898 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:15.959434  4805 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.316s	user 0.013s	sys 0.001s
W20260709 13:02:15.956606  4707 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.494s	user 0.010s	sys 0.005s
W20260709 13:02:15.987416  4880 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
W20260709 13:02:15.795256  4828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.006s	sys 0.005s
I20260709 13:02:15.823756  4849 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.058799  4848 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:15.948700  4846 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.066197  1643 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:16.066815  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:16.131631  1567 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:16.002734  4614 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.962771  4894 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.792361  4889 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:15.792507  4739 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.962729  4716 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:15.885037  4744 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:15.870410  4897 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.791211  4783 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:16.071875  2377 rpcz_store.cc:276] 0709 13:02:14.826856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.827025 (+   169us) service_pool.cc:224] Handling call
0709 13:02:16.015396 (+1188371us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:02:16.052772 (+ 37376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5631872}
I20260709 13:02:16.040063  4912 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:15.999377  4934 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.797168  4839 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:15.883363  4850 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:15.789583  4904 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:15.789410  4888 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.058260  4732 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:16.186895  4934 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.191570  4850 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.927321  4919 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:16.175860  4614 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.958410  4751 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:15.789769  4918 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:16.098951  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65976us
W20260709 13:02:16.127985  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 432728us
I20260709 13:02:16.202423  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:16.202726  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.923430  4929 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:16.087693  4901 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.099502  4848 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.925024  4754 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.055168  4847 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:02:16.092723  4880 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.090083  4707 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.176453  4739 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.205772  4614 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.180692  4716 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:16.088959  4828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.007s	sys 0.012s
I20260709 13:02:16.197410  4919 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:16.198513  4928 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.174203  4889 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:16.203701  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:16.129096  4898 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:16.217234  4847 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:16.209317  4848 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.217674  4934 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.219735  4880 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.220626  4716 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:16.245005  4716 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:16.221557  4874 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:16.223351  4913 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.229935  4751 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:16.215361  4754 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.224715  4828 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:16.232009  4803 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.265434  4828 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.204591  4918 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.227151  4919 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.241360  4898 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.249068  4824 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.251117  4889 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.242514  4847 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:16.246201  4716 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:02:16.209610  4901 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.259133  4707 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:16.279937  4865 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.874s	user 0.003s	sys 0.002s
I20260709 13:02:16.410279  4809 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.277586  4918 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.275512  4855 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:16.412916  4809 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.324144  4739 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.369941  4848 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.391994  4741 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:16.408541  4870 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.415518  4741 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:16.416850  4848 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:16.278337  4735 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.002s
I20260709 13:02:16.381670  4912 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.272280  4874 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:16.357714  4648 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:16.408607  4793 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:16.375422  4737 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.279105  4903 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.359573  4614 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.398756  4803 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.422966  4741 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.370929  4882 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.438592  4741 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.424988  4809 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.365938  4823 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:16.440877  4823 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:16.441943  4823 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.443369  4823 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.427696  4901 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.428941  4912 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.420185  4847 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.430012  4874 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.447665  4874 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.447850  4847 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.430361  4648 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.448900  4648 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:16.477455  4439 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.478236  4439 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.478513  4919 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.503849  4919 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.031703  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:16.101074  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.600750  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 568681us
I20260709 13:02:16.054286  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:16.607120  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.130563  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.607956  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 553830us
I20260709 13:02:16.085939  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:16.603775  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:16.619370  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 533409us
I20260709 13:02:16.616143  1648 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:16.620141  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:16.620970  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:16.126641  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2059) took 1435 ms. Trace:
W20260709 13:02:16.696465  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 76790.3us
I20260709 13:02:16.705344  4793 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:16.706768  4805 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.733258  4870 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:16.723397  4824 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:16.727663  4873 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:16.727823  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:16.722365  4880 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.739799  4880 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.741389  1722 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:16.743721  4880 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.744760  4803 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:16.745417  4803 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.746289  4803 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.748778  4855 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:16.750169  4903 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.750236  4821 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:16.719246  4752 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.711644  4847 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.749182  4882 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:16.749488  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:16.717512  4754 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:16.756829  4873 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.758415  4754 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.734812  4870 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:16.754714  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 134023us
I20260709 13:02:16.751348  4903 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.759130  4873 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.766230  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.767848  1646 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:16.769069  4671 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:16.772593  4752 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.776559  1721 rpcz_store.cc:276] 0709 13:02:14.691211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.565440 (+874229us) service_pool.cc:224] Handling call
0709 13:02:15.565785 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.782340 (+216555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.783943 (+  1603us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.822541 (+ 38598us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.084154 (+261613us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.084195 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.084277 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.126138 (+ 41861us) raft_consensus.cc:1685] finished
0709 13:02:16.126176 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.126443 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119574,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4805,"thread_start_us":348735,"threads_started":2,"wal-append.queue_time_us":201411}
I20260709 13:02:16.780186  4805 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:16.730779  4793 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.792606  4671 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:16.795342  4439 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.795516  4873 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.797319  4919 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:02:16.798750  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49401.8us
I20260709 13:02:16.799813  4754 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.755376  4874 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.815393  4870 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.815723  4874 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.821835  4803 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:16.824816  4805 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:16.826396  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.833503  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:16.834129  1648 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:16.826993  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:16.836279  1646 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.837021  4855 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:16.837702  1650 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.006s
I20260709 13:02:16.838639  1650 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:16.829331  1649 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:16.829450  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.842271  1644 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:16.831322  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.842654  4903 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:16.877074  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.804392  4793 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:16.881204  1646 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:16.882519  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:16.893106  4847 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:16.900655  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 67474.9us
I20260709 13:02:16.788486  4752 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:16.905133  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55601.4us
I20260709 13:02:16.906369  4874 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.907467  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:16.922513  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.922283  4910 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.943951  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:16.953504  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:16.973932  1642 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
W20260709 13:02:17.020490  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 67285.6us
I20260709 13:02:16.976662  4752 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:17.029246  1646 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
W20260709 13:02:17.035611  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.962714  4910 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:17.047451  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.008s	sys 0.035s
W20260709 13:02:17.052484  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.009s	sys 0.036s
I20260709 13:02:17.032897  4752 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:17.039258  4847 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:16.989753  4855 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:02:17.060251  4752 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:17.093000  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52267.7us
W20260709 13:02:17.029733  4776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.092s	user 0.045s	sys 0.090s
W20260709 13:02:17.104420  1720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.016s	sys 0.088s
W20260709 13:02:17.105072  1720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.017s	sys 0.089s
I20260709 13:02:17.105222  4932 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:02:17.094573  4810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.059s	sys 0.083s
W20260709 13:02:17.107318  4810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.316s	user 0.059s	sys 0.084s
W20260709 13:02:17.107720  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.021s	sys 0.018s
W20260709 13:02:17.108371  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.021s	sys 0.018s
I20260709 13:02:17.108837  4810 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:17.109501  4810 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.109941  4810 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:17.105089  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 197898us
W20260709 13:02:17.112886  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.093073  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:17.120142  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:17.112942  4776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.046s	sys 0.091s
W20260709 13:02:17.124821  1691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.017s	sys 0.037s
W20260709 13:02:17.150420  1691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.018s	sys 0.038s
W20260709 13:02:17.124882  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.017s	sys 0.012s
W20260709 13:02:17.128918  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87954.4us
I20260709 13:02:17.116552  4803 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:17.124531  4683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.026s	sys 0.023s
W20260709 13:02:17.156037  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.017s	sys 0.013s
I20260709 13:02:17.124161  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:17.124404  4907 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.063s	sys 0.093s
I20260709 13:02:17.142656  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 416) took 1265 ms. Trace:
I20260709 13:02:17.160234  1643 rpcz_store.cc:276] 0709 13:02:15.866808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.866956 (+   148us) service_pool.cc:224] Handling call
0709 13:02:17.131403 (+1264447us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b5400306e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:17.131621 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43154304}
I20260709 13:02:17.125100  4876 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:17.161180  4683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.026s	sys 0.023s
W20260709 13:02:17.124270  4648 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.013s
W20260709 13:02:17.168864  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 1.07344e+06us
W20260709 13:02:17.166906  4907 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.346s	user 0.063s	sys 0.095s
I20260709 13:02:17.165482  4932 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:17.190394  4945 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:17.191321  4945 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:17.195178  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 1974) took 3028 ms (client timeout 1513 ms). Trace:
W20260709 13:02:17.196426  4875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.022s	sys 0.045s
W20260709 13:02:17.196079  1717 rpcz_store.cc:269] 0709 13:02:14.167064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.167187 (+   123us) service_pool.cc:224] Handling call
0709 13:02:17.194425 (+3027238us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:17.194868 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830073216}
I20260709 13:02:17.186360  4932 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:17.188230  4903 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:17.203888  4875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.023s	sys 0.045s
I20260709 13:02:17.211086  4793 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.185279  4648 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.207605  4932 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.221328  4941 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 13:02:17.221850  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97859.2us
I20260709 13:02:17.223495  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:17.223385  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.231753  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2045) took 3316 ms. Trace:
I20260709 13:02:17.233175  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:17.234514  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.226253  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2001) took 1777 ms. Trace:
I20260709 13:02:17.236172  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:17.237661  1725 rpcz_store.cc:276] 0709 13:02:13.914915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.915259 (+   344us) service_pool.cc:224] Handling call
0709 13:02:13.966650 (+ 51391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.620903 (+654253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.231104 (+2610201us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400372b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:17.231504 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439077120}
I20260709 13:02:17.238121  1718 rpcz_store.cc:276] 0709 13:02:15.443712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.053009 (+609297us) service_pool.cc:224] Handling call
0709 13:02:16.053408 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.053663 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.053702 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.060590 (+  6888us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.130618 (+1070028us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:17.130697 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.130811 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.214084 (+ 83273us) raft_consensus.cc:1685] finished
0709 13:02:17.214108 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.220860 (+  6752us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":959831,"threads_started":1,"wal-append.queue_time_us":993577}
W20260709 13:02:17.241432  4889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.008s	sys 0.020s
I20260709 13:02:17.253193  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:17.260713  1570 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:17.261442  4648 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.262313  4847 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.262567  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:17.263087  4847 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.263825  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:17.264492  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2128) took 3308 ms. Trace:
I20260709 13:02:17.266010  4847 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.265182  1569 rpcz_store.cc:276] 0709 13:02:13.956415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.109422 (+153007us) service_pool.cc:224] Handling call
0709 13:02:14.309267 (+199845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.782845 (+473578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.782985 (+   140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:14.783039 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:14.783088 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.783161 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.264109 (+2480948us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400364cd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:17.264292 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13230,"raft.queue_time_us":99736,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6045,"spinlock_wait_cycles":458781056,"thread_start_us":426593,"threads_started":1}
I20260709 13:02:17.264443  4903 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:17.267482  4946 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.255733  4889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.010s	sys 0.020s
I20260709 13:02:17.268217  4946 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.261332  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:17.266597  4847 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.258200  4941 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:17.284523  4941 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:17.296085  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.085s	user 0.000s	sys 0.052s
W20260709 13:02:17.298689  4898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.072s	sys 0.093s
W20260709 13:02:17.300444  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.000s	sys 0.053s
I20260709 13:02:17.293382  4946 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.295516  4941 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.307260  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2000) took 1864 ms. Trace:
W20260709 13:02:17.310771  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87644.7us
I20260709 13:02:17.307758  1722 rpcz_store.cc:276] 0709 13:02:15.443038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.735782 (+292744us) service_pool.cc:224] Handling call
0709 13:02:15.736327 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.742282 (+1005955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.302875 (+560593us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b540010ead4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:17.307176 (+  4301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2210630016}
I20260709 13:02:17.313189  4793 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.296563  4889 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:17.310941  1688 connection.cc:256] RPC call timeout handler was delayed by -2.0192080310000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:02:17.298764  4945 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:17.305482  4799 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.285s	user 0.010s	sys 0.026s
I20260709 13:02:17.358547  1651 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:17.359846  4850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.002s	sys 0.039s
I20260709 13:02:17.308245  4648 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:17.309134  4898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.072s	sys 0.094s
I20260709 13:02:17.318161  4737 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:17.369434  4741 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.930s	user 0.003s	sys 0.021s
W20260709 13:02:17.372923  1574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2134) took 2631 ms (client timeout 1825 ms). Trace:
W20260709 13:02:17.373541  1574 rpcz_store.cc:269] 0709 13:02:14.741792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.743542 (+  1750us) service_pool.cc:224] Handling call
0709 13:02:17.372213 (+2628671us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002a76d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:17.372638 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2961492864}
I20260709 13:02:17.321507  4875 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.306583  4847 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:17.360414  4850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.003s	sys 0.039s
W20260709 13:02:17.388358  4799 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.460s	user 0.010s	sys 0.027s
I20260709 13:02:17.364506  4737 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.396003  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:17.409227  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 148076us
I20260709 13:02:17.401371  4737 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:17.388985  4648 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:02:17.400825  4821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.027s	sys 0.033s
W20260709 13:02:17.443478  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.397874  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 1.1897e+06us
W20260709 13:02:17.495070  4821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.027s	sys 0.033s
W20260709 13:02:17.506381  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.418416  4824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.025s	sys 0.040s
I20260709 13:02:17.403398  4945 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:17.524483  4828 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.017s	sys 0.035s
W20260709 13:02:17.525096  4828 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.017s	sys 0.035s
I20260709 13:02:17.526010  4648 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.507735  4903 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:17.537194  4929 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 13:02:17.513293  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 215135us
W20260709 13:02:17.509927  4824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.025s	sys 0.041s
I20260709 13:02:17.418629  4875 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.590353  4944 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.592418  4944 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:17.593519  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 228731us
I20260709 13:02:17.554162  4824 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:17.592932  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:17.598079  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 83078.5us
W20260709 13:02:17.601403  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 192799us
I20260709 13:02:17.593523  4855 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.630091  1642 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:17.639736  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.616726  4947 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.645720  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:17.638496  4752 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.005s
W20260709 13:02:17.647141  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.644654  4742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.183s	user 0.026s	sys 0.040s
I20260709 13:02:17.680766  4957 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:17.679217  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2058) took 2969 ms. Trace:
I20260709 13:02:17.630259  4945 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:17.683459  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44707.9us
W20260709 13:02:17.685362  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74759.5us
I20260709 13:02:17.685743  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.686807  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.689673  4855 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.688503  4947 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.675108  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:17.647957  4943 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:17.699931  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:17.700429  4742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.373s	user 0.027s	sys 0.040s
W20260709 13:02:17.704923  4955 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:17.706431  4875 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.707487  4824 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.708905  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.710470  2380 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:17.682144  4847 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:17.681703  4949 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:17.712692  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.711323  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.712393  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.625588  4737 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.730271  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:17.724491  1721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.008s	sys 0.010s
W20260709 13:02:17.733268  1721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.008s	sys 0.011s
I20260709 13:02:17.735747  1642 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:17.663165  4752 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.732380  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:17.721920  4956 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:17.751832  2372 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:17.765209  1690 leader_election.cc:336] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.874s (SENT)
W20260709 13:02:17.769582  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2060) took 3078 ms (client timeout 1825 ms). Trace:
W20260709 13:02:17.770231  1719 rpcz_store.cc:269] 0709 13:02:14.691520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.713366 (+ 21846us) service_pool.cc:224] Handling call
0709 13:02:17.762558 (+3049192us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b54001a7cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:17.769382 (+  6824us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021078400}
W20260709 13:02:17.771320  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 125167us
I20260709 13:02:17.772548  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:17.780799  4803 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.781469  4803 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.727061  4943 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.784036  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:17.785020  2379 raft_consensus.cc:2376] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7. Re-sending same reply.
I20260709 13:02:17.682922  1720 rpcz_store.cc:276] 0709 13:02:14.690897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.781103 (+ 90206us) service_pool.cc:224] Handling call
0709 13:02:14.781592 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.784206 (+  2614us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:14.796781 (+ 12575us) log.cc:844] Serialized 14 byte log entry
0709 13:02:15.965524 (+1168743us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b1800045ee0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:02:17.105620 (+1140096us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.106730 (+  1110us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.130515 (+ 23785us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:17.626002 (+495487us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.626236 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.639254 (+ 13018us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.651142 (+ 11888us) raft_consensus.cc:1685] finished
0709 13:02:17.651175 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.660094 (+  8919us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70851,"prepare.queue_time_us":13216,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":99278,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":26405,"spinlock_wait_cycles":2855808,"thread_start_us":1581618,"threads_started":3,"wal-append.queue_time_us":1262925}
I20260709 13:02:17.772111  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.766142  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.815021  4847 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.817270  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.817997  2377 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:17.817751  4847 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.819103  4941 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.820518  2374 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:17.821213  4855 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.821547  4941 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.822587  4945 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.824004  4752 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:17.824482  4932 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.005s
I20260709 13:02:17.824913  4932 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.826015  2371 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:17.826910  4810 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.716s	user 0.006s	sys 0.009s
I20260709 13:02:17.827117  4932 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.827359  4810 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.850795  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 417) took 1978 ms. Trace:
I20260709 13:02:17.851329  1651 rpcz_store.cc:276] 0709 13:02:15.867854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.868049 (+   195us) service_pool.cc:224] Handling call
0709 13:02:15.868456 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.359355 (+1490899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.828269 (+468914us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54003fc354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:17.845919 (+ 17650us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3223647616}
I20260709 13:02:17.820218  4847 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:17.869968  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 183325us
I20260709 13:02:17.868289  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:17.868644  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:17.767254  4949 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.870972  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:17.880816  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 108403us
W20260709 13:02:17.881235  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 242465us
I20260709 13:02:17.862411  4944 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.866606  4875 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:17.885190  4803 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.887921  4910 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.857239  4793 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:02:17.889700  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 124642us
I20260709 13:02:17.864631  4945 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:17.874461  4737 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.892252  1574 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:17.896783  1574 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:17.898972  1724 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:17.899060  1716 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:17.900484  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:17.901265  1723 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:17.902740  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1332) took 5103 ms. Trace:
I20260709 13:02:17.904752  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:17.906765  4947 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:17.909348  1722 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:02:17.553447   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:17.914227  1720 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:17.922680  1574 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.923861  1574 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.924538  4810 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:17.925067  4943 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.926084  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:17.926355  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.928660  4911 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:17.913491  4949 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:17.930090  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:17.913712  4875 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:17.931679  4875 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:17.932587  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2135) took 3015 ms. Trace:
I20260709 13:02:17.932951  1573 rpcz_store.cc:276] 0709 13:02:14.916892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.917095 (+   203us) service_pool.cc:224] Handling call
0709 13:02:14.917542 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.038599 (+1121057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.932094 (+1893495us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540022d8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:17.932432 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576942976}
I20260709 13:02:17.934823  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:17.935462  1643 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:17.911554  4960 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:02:17.930220  4911 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:17.917075  4793 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.939267  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:17.937075  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.943285  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 376) took 2425 ms. Trace:
I20260709 13:02:17.947279  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.947523  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:17.954276  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:17.946185  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:17.956992  1651 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.957213  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:17.956697  4956 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:17.959645  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 64652.2us
I20260709 13:02:17.960289  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:17.961202  4956 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:17.961638  4956 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:17.961790  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:17.962054  4956 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:17.965123  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 377) took 2453 ms. Trace:
I20260709 13:02:17.965485  1724 rpcz_store.cc:276] 0709 13:02:15.512079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.228882 (+1716803us) service_pool.cc:224] Handling call
0709 13:02:17.229520 (+   638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.899564 (+670044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.964874 (+ 65310us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:17.965053 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434171136}
I20260709 13:02:17.970238  4956 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:17.972815  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:17.973345  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:17.973599  1568 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:17.974424  4911 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:17.975310  4945 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:17.975796  4945 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.976187  4945 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:17.977694  1646 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:17.903107  1645 rpcz_store.cc:276] 0709 13:02:12.799042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.799239 (+   197us) service_pool.cc:224] Handling call
0709 13:02:12.799526 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.887814 (+ 88288us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:13.755231 (+867417us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.156035 (+400804us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.588973 (+432938us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b2c00056608. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:02:14.589366 (+   393us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.589502 (+   136us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:14.741130 (+151628us) log.cc:844] Serialized 14 byte log entry
0709 13:02:14.742764 (+  1634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.902094 (+3159330us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b5400411c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:17.902211 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.902293 (+    82us) raft_consensus.cc:1685] finished
0709 13:02:17.902310 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.902525 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400999,"prepare.queue_time_us":1086345,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":222792,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":70417,"spinlock_wait_cycles":332429952,"thread_start_us":1109367,"threads_started":3,"wal-append.queue_time_us":561797,"wal-append.run_cpu_time_us":10248,"wal-append.run_wall_time_us":1529357}
I20260709 13:02:17.944491  1717 rpcz_store.cc:276] 0709 13:02:15.511772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.241344 (+1729572us) service_pool.cc:224] Handling call
0709 13:02:17.256142 (+ 14798us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:17.364385 (+108243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.364430 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.396845 (+ 32415us) log.cc:844] Serialized 52 byte log entry
0709 13:02:17.850407 (+453562us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:17.882301 (+ 31894us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.883636 (+  1335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.883711 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.936684 (+ 52973us) raft_consensus.cc:1685] finished
0709 13:02:17.936714 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.936893 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25741,"prepare.queue_time_us":32085,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":85374,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":21148,"spinlock_wait_cycles":14168960,"thread_start_us":434365,"threads_started":1,"wal-append.queue_time_us":509998}
W20260709 13:02:17.979187  1576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.065s	sys 0.091s
I20260709 13:02:17.985729  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:17.985927  1576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.065s	sys 0.091s
I20260709 13:02:17.988699  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:17.988826  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:17.991979  2373 raft_consensus.cc:2368] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
W20260709 13:02:17.992839  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1363) took 3167 ms (client timeout 1532 ms). Trace:
W20260709 13:02:17.993258  1650 rpcz_store.cc:269] 0709 13:02:14.821957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.822091 (+   134us) service_pool.cc:224] Handling call
0709 13:02:17.958183 (+3136092us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002e2054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:17.989489 (+ 31306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651742976}
W20260709 13:02:18.000188  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.001524  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:18.004983  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76443.3us
I20260709 13:02:17.976629  4949 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:18.007642  1643 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:18.008205  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:18.014017  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.010396  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:18.017733  1573 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:18.019181  1570 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:18.006158  4956 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:18.010541  4648 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:18.021606  1568 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:18.025246  1569 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:18.030229  1567 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:18.030807  1567 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:18.000581  4737 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:02:18.032668  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:18.033141  4737 consensus_queue.cc:1243] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:18.033455  2375 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:18.033711  4737 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:18.034122  4737 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:18.034994  4737 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.036603  4737 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:18.039271  4648 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:18.040621  4648 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.041953  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.042665  1723 raft_consensus.cc:2376] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7. Re-sending same reply.
I20260709 13:02:18.023435  4949 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:18.042680  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.044780  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.045008  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:18.045210  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:18.046021  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.046087  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.046665  1716 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:18.046811  1720 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.048460  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 375) took 2537 ms. Trace:
I20260709 13:02:18.049813  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 414) took 2897 ms. Trace:
I20260709 13:02:18.048830  1721 rpcz_store.cc:276] 0709 13:02:15.511404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.928515 (+1417111us) service_pool.cc:224] Handling call
0709 13:02:17.037431 (+108916us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:17.736232 (+698801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.736263 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.788832 (+ 52569us) log.cc:844] Serialized 52 byte log entry
0709 13:02:17.917401 (+128569us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:17.981709 (+ 64308us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.047806 (+ 66097us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18003f96a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:02:18.048008 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.048090 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.048138 (+    48us) raft_consensus.cc:1685] finished
0709 13:02:18.048237 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.048404 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6251,"prepare.queue_time_us":67288,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":85960,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":3599,"spinlock_wait_cycles":2919808,"thread_start_us":628627,"threads_started":2,"wal-append.queue_time_us":183980}
I20260709 13:02:18.050171  1647 rpcz_store.cc:276] 0709 13:02:15.152189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.152295 (+   106us) service_pool.cc:224] Handling call
0709 13:02:15.247395 (+ 95100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.476828 (+229433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.476873 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.147497 (+1670624us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b2c00056608. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:02:17.148271 (+   774us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.659504 (+511233us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:17.674860 (+ 15356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.674998 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.911084 (+236086us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b24002777a8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830b079c4 00007fc830b05acb 00007fc830cc0dc9 00007fc830cc0a55 00007fc830cc009d 00007fc8338f210e 00007fc83390988c 00007fc833908bb6 00007fc8339087d8 00007fc833912221
0709 13:02:18.049462 (+138378us) raft_consensus.cc:1685] finished
0709 13:02:18.049484 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.049640 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":308390,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":306604672,"thread_start_us":685825,"threads_started":2,"wal-append.queue_time_us":576438}
I20260709 13:02:18.051590  4959 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:18.043794  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.020931  4911 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:18.052810  1723 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:18.046533  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.053947  4793 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:18.054095  1719 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:18.054651  1572 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:18.057734  1716 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:18.058650  1570 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:18.023974  1722 raft_consensus.cc:1217] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[7.7-7.7]   Dedup: 7.7->[]
W20260709 13:02:18.065062  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74248.4us
I20260709 13:02:18.065443  1722 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:18.060014  1717 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:18.060808  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69231.7us
I20260709 13:02:18.060992  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.061826  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:18.031733  1567 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.072892  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.077484  1574 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:02:18.078074  1651 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:18.079430  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:18.078027  1574 pending_rounds.cc:85] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:02:18.083081  1576 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
W20260709 13:02:18.085654  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.080775  4742 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.091849  4742 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.094308  4742 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:18.095122  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.099479  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.100106  1717 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:18.080974  1574 pending_rounds.cc:107] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:18.098946  1724 raft_consensus.cc:2376] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7. Re-sending same reply.
I20260709 13:02:18.101126  1574 raft_consensus.cc:2889] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:18.104977  1651 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:18.113287  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:18.115576  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.125598  4977 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.126417  4977 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.099610  4742 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.132077  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.135936  1717 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:18.138180  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:18.045003  4949 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.143585  4760 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:18.144234  4760 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:18.145239  4760 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.146188  4793 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:18.147281  4793 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:18.148051  4761 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:18.148931  4793 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:18.149048  4761 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.149555  4793 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:18.149911  4761 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.150640  4793 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.151974  4793 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:18.153278  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:18.132652  4880 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:02:18.174500  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.103225  4911 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:02:18.182988  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:18.197804  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.207621  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.208519  1716 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:18.152554  4944 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:18.211592  1722 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:18.220198  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.205240  4907 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:18.181643  4977 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:18.224468  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.218993  4803 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.225399  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:18.240026  1724 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:18.227568  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.228818  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:18.229427  4974 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:02:18.229470  4975 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:18.211617  4944 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:18.141904  4742 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.231951  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:18.241277  1569 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:18.206215  4910 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:18.230427  4911 consensus_queue.cc:1243] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:18.255096  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.259295  1720 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:18.259413  1576 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:18.239617  4907 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:18.278541  4803 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.246865  4963 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:18.281157  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.242462  4977 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.226145  4956 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 13:02:18.291136  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.298132  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.303115  1720 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:18.297906  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.310228  1722 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:18.250160  4981 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.241084  4946 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:02:18.289757  4963 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:18.342125  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.265064  4752 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
W20260709 13:02:18.369103  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 128819us
I20260709 13:02:18.367760  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.297250  4989 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.380764  1722 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:18.399168  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:18.366106  4946 consensus_queue.cc:1243] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:18.343984  4963 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:18.407663  1572 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:18.413110  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:18.416450  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 115666us
W20260709 13:02:18.419191  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 391234us
I20260709 13:02:18.420753  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:18.431172  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.365252  4977 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.365594  4981 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.467265  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.466120  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.475159  1718 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:18.433048  4752 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:18.497754  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.497881  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.498574  1722 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.501411  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.503063  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.503793  1718 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:18.512213  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.514703  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.519874  4999 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:18.521543  4994 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.522554  4994 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.526031  4988 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.526907  4988 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.532847  4803 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:18.535410  4981 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:18.532682  4742 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:18.540961  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:18.427246  4944 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.544852  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.491199  4910 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:18.545315  1720 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:18.536124  1567 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.559428  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.575500  1576 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:18.576886  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 156282us
I20260709 13:02:18.577796  1718 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:18.508725  4956 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:18.559435  4976 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.304s	user 0.000s	sys 0.002s
I20260709 13:02:18.605686  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:18.617472  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 195145us
W20260709 13:02:18.620664  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.448707  4989 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.464618  4907 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.624445  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.586621  4956 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:18.621754  1616 connection.cc:256] RPC call timeout handler was delayed by -1.6241105380000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
I20260709 13:02:18.626642  1719 raft_consensus.cc:2376] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 7. Re-sending same reply.
I20260709 13:02:18.635362  1716 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:18.640717  1720 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:18.641858  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.643776  1723 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:18.665797  1724 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:18.583781  4910 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:18.669713  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.669428  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.673655  1716 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:18.636108  4944 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:18.679729  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:18.684823  4793 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.005s
W20260709 13:02:18.686453  4737 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.003s
I20260709 13:02:18.687675  1642 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:18.693825  2373 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:18.707712  4994 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:18.713116  1650 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:18.624580  4999 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:02:18.715312  4999 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:18.715806  4999 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:18.725195  4989 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:18.687687  4979 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:02:18.729174  4977 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:18.705129  4960 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:18.742012  4910 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:18.744913  1567 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:18.756620  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:18.757613  1717 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:18.759616  1568 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:18.760046  1568 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:18.761075  1568 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.762252  1568 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:18.777617  4945 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.783s	user 0.012s	sys 0.002s
I20260709 13:02:18.778108  4945 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:18.787580  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 83544.8us
W20260709 13:02:18.794855  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.801082  1722 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
W20260709 13:02:18.792515  4761 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.007s
I20260709 13:02:18.717103  4999 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:18.806274  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 376534us
W20260709 13:02:18.812489  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.821949  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:18.825073  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:18.828112  5001 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:02:18.827836  4999 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.829062  1576 pending_rounds.cc:85] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:18.831995  1576 pending_rounds.cc:107] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:18.832201  1575 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:02:18.832387  1576 raft_consensus.cc:2889] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:18.832640  1575 pending_rounds.cc:85] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:18.837762  1575 pending_rounds.cc:107] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:18.838025  1575 raft_consensus.cc:2889] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:18.853189  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:18.855448  4810 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:18.856230  4810 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:18.857751  4810 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:18.861887  4741 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:18.863651  4847 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.864056  4959 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:18.864322  4847 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.867828  1688 leader_election.cc:400] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:18.868330  1688 leader_election.cc:403] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:18.843539  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:18.864821  4959 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.869499  1573 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
W20260709 13:02:18.895236  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.900292  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:18.815572  4761 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.895124  4959 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:18.912710  1644 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
W20260709 13:02:18.935577  4978 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.672s	user 0.006s	sys 0.000s
I20260709 13:02:18.832768  4956 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.955425  5008 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:18.935093  4994 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.965942  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:18.966310  4994 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.946357  4977 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:18.968668  4977 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.969123  4716 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.976773  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:18.944056  4875 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:18.978607  4875 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:18.961835  4742 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:18.962806  4898 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.977808  1651 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:18.961323  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 136360us
I20260709 13:02:18.939113  4988 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:18.980805  5005 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 13:02:18.989120  1614 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:18.985181  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:19.014209  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:18.991919  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.015820  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.017747  2373 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:18.985270  4742 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.048658  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:19.049666  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.056668  2378 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:02:19.020277  4988 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:19.070564  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.072743  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:19.072945  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.041285  4742 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.023461  4761 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:19.080123  1567 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:19.082598  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.082582  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.083698  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.086747  1723 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:19.012841  4898 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.085530  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.087656  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.089895  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.106676  1651 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:19.089210  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:19.089579  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.110302  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.089408  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.112066  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:19.114487  1568 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:19.117380  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:19.120770  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.121320  1717 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:19.122381  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.079913  4988 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.123014  1720 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:19.123668  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:19.113329  1725 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:19.087181  4824 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.130026  1651 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.132221  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:19.132048  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.135277  1644 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:19.135936  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:19.141402  5013 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:19.141484  4977 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:19.129475  4875 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:19.144440  1718 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:02:19.144629  4963 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.005s
I20260709 13:02:19.146013  1718 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:19.125111  4994 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:19.135375  1645 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:19.152843  1720 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:19.158645  1567 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:19.164008  1650 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:19.164602  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81070.9us
I20260709 13:02:19.165161  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.159943  1725 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 13:02:19.145527  4963 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:19.165351  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:19.128567  4988 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:19.176551  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.177488  1647 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:19.161136  1722 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:19.180127  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.181243  1651 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:19.181916  1720 raft_consensus.cc:2368] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:19.185534  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:19.184857  5009 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:19.187443  5009 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.191617  1723 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:19.194253  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.195015  1651 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:19.187814  5009 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.203847  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:19.183139  4824 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.213495  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:19.214746  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.218796  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:19.220094  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:19.220441  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.221670  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.225066  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.227057  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:19.228116  4990 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:02:19.229003  4824 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:19.230178  1644 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:19.231649  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:19.232303  1643 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.233673  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:19.233918  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.205879  5000 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:02:19.237632  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.243772  1643 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:19.245369  1651 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:19.249058  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:19.245078  4648 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.259865  4898 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:19.270851  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.262099  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:19.264065  2373 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:19.265873  1642 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:19.269690  4929 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:02:19.273679  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.272434  1645 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:19.276947  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:19.277763  2378 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:19.279985  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.280714  1720 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
W20260709 13:02:19.285136  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.285775  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.286319  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.293905  1642 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:19.319633  1567 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:02:19.320811  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.326527  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:19.327864  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4651468529999647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
I20260709 13:02:19.320191  1567 pending_rounds.cc:85] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:02:19.332610  1567 pending_rounds.cc:107] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:19.335353  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.335806  1567 raft_consensus.cc:2889] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.329862  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:19.337563  1644 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:19.338569  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:19.336889  4995 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:19.342869  2373 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [3.5-6.7]
I20260709 13:02:19.267668  4648 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:19.345286  5020 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:02:19.346998  1642 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:19.318778  4932 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:19.320525  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:19.362003  1720 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:19.361892  1725 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
W20260709 13:02:19.339515  4952 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.040s	user 0.007s	sys 0.005s
I20260709 13:02:19.368286  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:19.339636  4824 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.366695  4648 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:19.394093  2373 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:19.395180  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.395690  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.388084  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:19.353917  4932 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:19.404451  1644 raft_consensus.cc:2368] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:19.409670  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:19.416190  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:19.417171  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49017.2us
I20260709 13:02:19.419364  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:19.420109  1567 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:19.421346  1651 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:19.428382  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:19.405859  4977 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:19.420739  4990 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:19.427932  2375 raft_consensus.cc:2410] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:02:19.430770  5018 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.244s	user 0.002s	sys 0.000s
I20260709 13:02:19.433072  4929 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:19.450917  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:19.411496  4824 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:19.458845  4824 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:19.446365  4994 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:02:19.460734  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:19.462888  4981 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:19.463294  4981 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.463677  4981 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.468833  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:19.470654  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:19.471860  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:19.472442  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.473068  4994 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:19.473498  4994 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.473838  4994 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.430989  4963 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:19.476788  4977 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:19.479351  4648 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:19.479803  4648 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.480137  4648 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:19.455601  4932 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.485894  4932 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.443477  5000 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:02:19.456804  4959 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 13:02:19.489969  4959 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.492311  4959 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:19.445866  4947 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:19.491184  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:19.497571  4947 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.498183  4947 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:19.498320  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:19.492699  4898 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:19.499800  4898 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:19.500026  5006 raft_consensus.cc:2764] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:19.501173  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:19.501078  4898 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:19.501937  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.504521  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62416.2us
I20260709 13:02:19.507815  2373 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:19.508188  1567 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:19.513123  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.515841  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.440810  4995 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:19.528375  2373 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:19.529958  1642 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:19.453490  4929 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.502074  2375 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:19.506850  4977 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.546517  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:19.551265  1642 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:19.542980  4929 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.509305  4824 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:02:19.565240  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.564836  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66622.8us
I20260709 13:02:19.531848  4990 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:02:19.581507  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 390) took 1038 ms. Trace:
I20260709 13:02:19.581876  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:19.582073  1724 rpcz_store.cc:276] 0709 13:02:18.542725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.542878 (+   153us) service_pool.cc:224] Handling call
0709 13:02:19.581266 (+1038388us) spinlock_profiling.cc:243] Waited 96.1 ms on lock 0x7b54002959d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:19.581428 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221143296}
I20260709 13:02:19.535120  4988 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.563854  4761 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:19.721895  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.723124  4995 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:19.728878  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 147047us
I20260709 13:02:19.723363  5009 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:19.724547  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.726852  1642 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:19.566625  4855 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:19.553447  4742 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:19.753902  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.753588  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 164547us
W20260709 13:02:19.762688  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.783403  1645 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:19.783275  5029 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:19.792593  5025 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:19.792734  4648 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.792325  5029 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.798966  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.802541  4956 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.805965  1720 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:19.811162  4932 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:02:19.785120  5032 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:02:19.743440  4761 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:19.815532  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.552s (SENT)
I20260709 13:02:19.819105  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 1936) took 1034 ms. Trace:
I20260709 13:02:19.826397  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.820513  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:19.815843  1647 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:19.827088  1642 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:19.826812  1650 rpcz_store.cc:276] 0709 13:02:18.784935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.792173 (+  7238us) service_pool.cc:224] Handling call
0709 13:02:19.163836 (+371663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.164765 (+   929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.818797 (+654032us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54000915d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:19.819020 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827340160}
I20260709 13:02:19.754662  4855 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:19.830302  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:19.828131  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 248084us
I20260709 13:02:19.764132  4981 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:19.833366  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:19.816001  4824 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:19.843861  4847 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:19.843084  5009 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:19.727912  5016 raft_consensus.cc:2764] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:19.835906  4898 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:19.782521  4990 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:02:19.866175  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:19.866531  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51030.3us
W20260709 13:02:19.865617  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102680us
I20260709 13:02:19.858734  4742 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:19.885255  1539 leader_election.cc:400] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:19.886994  1539 leader_election.cc:403] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.886760  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:19.887490  1647 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:19.863466  4898 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:19.793502  5033 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:02:19.863062  4847 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:19.893934  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.900923  1717 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:19.884570  4648 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:19.860893  5037 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:02:19.862246  4824 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:19.908629  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:19.911105  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 80922.1us
W20260709 13:02:19.923863  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.928799  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.931818  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:19.933737  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66307.3us
I20260709 13:02:19.901638  4898 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.903869  4988 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:19.901439  5041 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:19.955715  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 95789.7us
I20260709 13:02:19.904443  4847 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:19.963836  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.969189  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.971349  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:19.973347  4847 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:19.977180  5035 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:02:19.968115  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:19.985776  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:19.989048  5030 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:02:19.967139  4648 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:19.990080  4648 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.990911  4648 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:19.995652  5031 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:02:19.904704  5004 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:02:19.949708  5039 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.009261  4981 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.011824  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.012635  1646 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.012667  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.013263  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.013609  1642 raft_consensus.cc:2376] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8. Re-sending same reply.
I20260709 13:02:20.014622  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.014729  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.000562  5040 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.015419  1642 raft_consensus.cc:2376] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8. Re-sending same reply.
I20260709 13:02:20.015731  5040 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:20.016033  5050 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:02:20.016237  5040 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.016805  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.017710  1643 raft_consensus.cc:2410] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:20.016819  4761 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:20.018261  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.019590  1651 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:20.004060  5045 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:20.003846  5029 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:19.967520  5032 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:19.917114  4996 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:02:20.020103  5045 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.020982  4945 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:20.021164  5045 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.021310  4996 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:20.024704  4996 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:20.025736  4996 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.027963  4996 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:20.015522  4994 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 13:02:19.904197  5008 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:02:20.029084  4994 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.029701  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:20.030427  4824 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:20.030702  4990 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:20.031883  4990 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:20.032135  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:20.032528  4990 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:20.033847  4855 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:20.003868  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:20.035902  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.036777  4988 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:20.036949  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:20.037626  4988 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:20.039271  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.040460  1571 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:20.041342  4999 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:20.041898  4999 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:20.013217  1644 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:02:20.043375  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.044601  1571 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:20.046432  1647 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:20.046873  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:20.048557  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.049333  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.050745  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.052675  4981 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:20.053051  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:20.065240  1575 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:20.068782  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:20.053282  5031 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.057s	user 0.004s	sys 0.000s
I20260709 13:02:20.053361  1576 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:02:20.055235  4947 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.556s	user 0.004s	sys 0.009s
W20260709 13:02:20.060402  5022 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.852s	user 0.004s	sys 0.011s
I20260709 13:02:20.012832  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.102077  1649 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.034073  4990 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.031075  4824 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:20.043103  4999 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.062206  1570 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:20.073161  5006 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.080587  1646 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:20.089329  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:20.105657  5039 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
W20260709 13:02:20.110601  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.133687  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.140046  5051 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.141086  5051 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.104573  4947 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:20.336258  4981 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:20.378703  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.152876  1472 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 217) took 31443 ms (client timeout 15000 ms). Trace:
I20260709 13:02:20.159657  4999 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.384445  4981 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:20.371495  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.250181  4956 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:20.252892  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.148s (SENT)
I20260709 13:02:20.333545  5016 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.388214  4956 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:20.143448  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.216081  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2092) took 1145 ms. Trace:
I20260709 13:02:20.144182  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.391348  1645 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:20.332903  4929 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.761s	user 0.012s	sys 0.001s
I20260709 13:02:20.143795  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.250701  5040 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.144796  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:20.390504  1721 rpcz_store.cc:276] 0709 13:02:19.070853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.070981 (+   128us) service_pool.cc:224] Handling call
0709 13:02:19.085371 (+ 14390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.110555 (+ 25184us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:19.989998 (+879443us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b18004108c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf
0709 13:02:19.990157 (+   159us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.167278 (+177121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.215593 (+ 48315us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:20.215657 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.215762 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.215978 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14574,"prepare.queue_time_us":53235,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":1732,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":21847,"spinlock_wait_cycles":7999872,"thread_start_us":219753,"threads_started":2}
I20260709 13:02:20.414204  5040 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.416029  2371 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:20.416882  1647 raft_consensus.cc:2368] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
W20260709 13:02:20.418952  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.143935  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 212273us
W20260709 13:02:20.152897  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.388320  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:20.379073  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.128458  5006 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.389904  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:20.429487  1649 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:20.434916  4932 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:20.384272  1472 rpcz_store.cc:269] 0709 13:01:48.660017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.664615 (+  4598us) service_pool.cc:224] Handling call
0709 13:02:20.097589 (+31432974us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:08.964981 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:09.240170 (+275189us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:09.256525 (+ 16355us) write_op.cc:435] Acquired schema lock
0709 13:02:09.261202 (+  4677us) tablet.cc:662] Decoding operations
0709 13:02:09.568479 (+307277us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:09.568587 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:09.568627 (+    40us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:02:09.569543 (+   916us) tablet.cc:701] Row locks acquired
0709 13:02:09.569605 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:02:09.569867 (+   262us) write_op.cc:270] Start()
0709 13:02:09.570097 (+   230us) write_op.cc:276] Timestamp: P: 1783602129569779 usec, L: 0
0709 13:02:09.570126 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:02:09.571390 (+  1264us) log.cc:844] Serialized 2094 byte log entry
0709 13:02:09.727587 (+156197us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b
0709 13:02:11.203579 (+1475992us) op_driver.cc:464] REPLICATION: finished
0709 13:02:12.148512 (+944933us) write_op.cc:301] APPLY: starting
0709 13:02:12.165851 (+ 17339us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:12.166073 (+   222us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:12.166110 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:12.166973 (+   863us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:12.167268 (+   295us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:02:12.167317 (+    49us) write_op.cc:312] APPLY: finished
0709 13:02:12.167625 (+   308us) log.cc:844] Serialized 47 byte log entry
0709 13:02:12.344550 (+176925us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:12.351269 (+  6719us) write_op.cc:454] Released schema lock
0709 13:02:12.365573 (+ 14304us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:12.402725 (+ 37152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:12.402758 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41180,"reported_tablets":50,"spinlock_wait_cycles":12544,"child_traces":[["op",{"apply.queue_time_us":730625,"apply.run_cpu_time_us":63808,"apply.run_wall_time_us":391923,"num_ops":5,"prepare.queue_time_us":55735,"prepare.run_cpu_time_us":106785,"prepare.run_wall_time_us":1075025,"raft.queue_time_us":704456,"raft.run_cpu_time_us":45616,"raft.run_wall_time_us":676385,"replication_time_us":1594475,"spinlock_wait_cycles":83202816,"thread_start_us":1587763,"threads_started":5,"wal-append.queue_time_us":870129,"wal-append.run_cpu_time_us":72939,"wal-append.run_wall_time_us":3171772}]]}
I20260709 13:02:20.428375  4929 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:20.429723  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.885s (SENT) [suppressed 26 similar messages]
W20260709 13:02:20.440514  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50716.4us
I20260709 13:02:20.375849  4990 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:20.444731  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 355527us
I20260709 13:02:20.249584  4996 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:20.217767  5039 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.389827  5016 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.475314  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:20.434151  5006 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.455389  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.485571  1649 raft_consensus.cc:2376] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9. Re-sending same reply.
I20260709 13:02:20.319063  4994 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.408638  5045 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.485690  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:20.491473  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:20.449322  4761 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:20.466724  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.292783  4898 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.453297  5029 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:20.478554  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 225716us
I20260709 13:02:20.492660  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:20.459307  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.429093  4648 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:20.442289  4716 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:02:20.480628  5016 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.521559  1724 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:20.522262  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:20.528014  2372 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:20.531805  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:20.533229  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.535135  1721 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.536295  1722 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:02:20.555079  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.556839  2378 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:20.559629  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.560048  1645 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:20.391955  4956 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:20.481494  5039 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:20.485216  4996 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.533207  5004 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.535650  4994 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:20.572459  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.580422  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.549952  4824 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:20.520810  5029 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.553236  4932 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.553177  4947 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.580229  4994 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:20.636055  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.588845  4956 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:20.644366  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:20.646489  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.647066  1572 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:20.622133  4847 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.003s
I20260709 13:02:20.590842  5040 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:20.654027  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:20.654285  5045 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.607225  5059 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:20.659017  4929 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.650442  4994 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:20.700747  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:20.701578  1723 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:20.702488  1572 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:20.593170  5004 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.573645  4949 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:02:20.655740  5039 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:20.672866  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.673713  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.674932  4956 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:20.678033  4996 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.678129  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:20.720747  1724 raft_consensus.cc:606] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:20.679451  4648 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.680365  4824 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.682089  5051 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.682399  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:20.725450  1722 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:20.684923  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:20.697088  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.702590  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.727756  1568 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:20.659053  1575 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:20.717607  1569 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 5.
I20260709 13:02:20.716887  1576 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:20.714771  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:20.733373  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.706532  5059 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:20.731446  5053 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:20.743049  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:20.692561  4847 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.715842  4716 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:20.731359  4898 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.705286  5040 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.763922  5040 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.713383  4932 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:02:20.757544  1644 raft_consensus.cc:2368] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:20.759028  4847 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:20.744762  5006 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:20.769141  1721 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:20.742062  5057 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.769219  4949 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:20.769325  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.775149  1575 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:20.779747  5004 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:20.740208  5059 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:20.742030  5053 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:20.808600  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:20.808892  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:20.814272  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.815166  1576 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:20.866374  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:20.869810  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 75969us
W20260709 13:02:20.881335  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.882807  1569 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:20.821440  4847 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:20.891566  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:20.896003  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:20.896803  1569 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 5.
I20260709 13:02:20.897763  4981 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:20.899106  5045 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:20.899470  5045 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:20.900380  5045 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:20.902032  5063 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:02:20.903368  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.865200  4648 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:20.903271  5057 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:20.878208  1523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54597.1us
I20260709 13:02:20.896075  4898 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.928792  1569 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:20.931289  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.937330  1569 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:20.910019  5029 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:20.910274  4648 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:02:20.949579  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.954677  1720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.004s	sys 0.012s
W20260709 13:02:20.959620  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.004s	sys 0.012s
W20260709 13:02:20.960052  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.959983  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 466449us
I20260709 13:02:20.963891  2378 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:20.936687  5040 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:20.924669  5057 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:20.969609  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:20.964579  4945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.002s	sys 0.018s
W20260709 13:02:20.980682  1688 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.532s (SENT)
W20260709 13:02:20.997232  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.967283  5062 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:02:20.977255  1576 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:21.010732  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:21.011363  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 327610us
I20260709 13:02:20.974694  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 398) took 1104 ms. Trace:
I20260709 13:02:20.982474  5057 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.021565  4742 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.023116  4988 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:02:21.024899  5016 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.029459  1471 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:21.030943  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.080572  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:21.100651  1573 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:21.030579  4761 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:21.100957  2378 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:20.966090  4956 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:21.036379  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.042461  4994 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.043471  5040 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:20.979223  4945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.002s	sys 0.018s
W20260709 13:02:21.044775  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64151.8us
I20260709 13:02:21.044870  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.046047  1649 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:20.957680  5067 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 13:02:21.005739  1569 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:20.905495  5053 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:21.066308  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48647.9us
I20260709 13:02:21.069446  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.053566  5045 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.073465  5004 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.074074  5006 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.074872  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:21.081941  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.083022  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.083469  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.132722  1471 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 4, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:21.084172  4648 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.088919  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:21.088994  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:21.067096  4999 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.008s
I20260709 13:02:21.100826  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.164950  1644 raft_consensus.cc:2368] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:21.035346  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.102769  1574 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:21.097834  1720 rpcz_store.cc:276] 0709 13:02:19.870837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.881449 (+ 10612us) service_pool.cc:224] Handling call
0709 13:02:19.896643 (+ 15194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.896936 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:20.074243 (+177307us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.974163 (+899920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.974248 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:20.974312 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.974441 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.974615 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13764,"prepare.queue_time_us":101725,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":88104,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":141,"thread_start_us":1062545,"threads_started":2,"wal-append.queue_time_us":1021769}
I20260709 13:02:21.117689  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:21.122044  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:21.167915  2378 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:21.084431  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.161013  1571 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:21.160104  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:21.163260  1576 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.163842  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:21.189981  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.191143  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:21.193578  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.196004  1573 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:21.201884  1569 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.190114  2371 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:21.197005  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.205410  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.208134  1718 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:21.209194  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:21.210529  1642 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.213968  1570 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:21.214488  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96889.8us
I20260709 13:02:21.215888  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:21.249884  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:21.216620  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.216877  5067 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.218293  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.218483  5006 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:21.205902  5059 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.220826  1721 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:02:21.221565  4990 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.738s	user 0.013s	sys 0.000s
I20260709 13:02:21.204103  5053 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.222621  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.268774  1716 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:21.191469  4907 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.228708  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.229471  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.230156  1724 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:21.268853  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 252227us
W20260709 13:02:21.232896  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.232954  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:21.234418  5084 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 13:02:21.214023  2372 raft_consensus.cc:2410] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:02:21.214308  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.237836  4761 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:21.240665  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.279067  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.279495  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.241868  4742 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:21.242914  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:21.246243  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.261652  1719 raft_consensus.cc:2368] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:21.198280  1471 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 5, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:21.196206  4956 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.265393  4988 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:21.270819  1644 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:21.274106  1648 raft_consensus.cc:2368] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:21.271114  1721 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:21.261185  4898 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:21.288769  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.294180  1724 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:21.291531  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.296993  1716 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.302785  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.303638  1719 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
W20260709 13:02:21.316733  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.285023  4945 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:02:21.294368  4956 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.292147  4996 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.324851  4996 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.326916  4956 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:21.327286  4996 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.295080  4990 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.329185  5059 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.329620  4990 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.330722  5040 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:21.284904  4907 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.334708  4907 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.285349  4742 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:21.265388  5068 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:02:21.282256  4994 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.293540  5067 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.350230  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:21.386147  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 378840us
I20260709 13:02:21.351016  1574 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:02:21.352083  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.353250  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:21.357860  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.358815  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 74728.1us
I20260709 13:02:21.360821  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:21.365980  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77750.6us
I20260709 13:02:21.362138  1471 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 5, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 5 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:21.383126  1576 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:21.387977  1575 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:21.392871  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.268115  5006 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.394352  1643 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.394639  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.397034  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.327955  4956 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:21.404145  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.410000  1647 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:21.405988  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.406798  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.407032  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.397149  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.407691  1644 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.411275  1567 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:21.403591  1716 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:21.408049  1569 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:02:21.412526  1569 pending_rounds.cc:85] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:21.412786  1569 pending_rounds.cc:107] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:02:21.413096  1569 raft_consensus.cc:2889] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.405172  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:21.410629  1723 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:21.300891  5051 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:21.417295  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.417151  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:21.359285  5016 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.421384  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.422847  1647 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.420833  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:21.423682  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.424111  1574 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:21.421077  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.426016  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.427295  1642 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.389424  4996 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.428609  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.432557  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:21.438834  5076 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:21.439386  5076 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:21.439297  4742 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.441136  1719 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:21.441375  5088 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:21.442556  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.438856  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:21.392524  5082 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.444475  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:21.444861  5082 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.442619  5008 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:21.446089  5006 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.446177  5008 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:21.447175  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.447888  5082 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:21.448163  4875 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:21.447706  5008 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.448014  4855 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:21.452165  5076 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.409000  5067 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.455471  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:21.456245  5004 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.447995  1723 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:21.466939  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.469125  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.471240  1569 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:02:21.469820  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.472379  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:21.481581  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.482461  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.486058  1725 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.486918  1575 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:21.494227  1724 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:21.486351  1569 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:21.496771  1569 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:21.498008  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0226166850000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 13:02:21.491001  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:21.499426  1647 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.501602  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.458590  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:21.504488  1570 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:21.505334  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:21.506184  2371 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.487349  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:21.519444  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.520191  1573 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:21.519654  1642 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:21.522133  1722 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:21.496927  1471 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 5, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 5 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:21.461838  4875 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:21.519505  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.527385  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:21.528007  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.529567  1723 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.529985  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.530823  1724 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.464241  4855 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:21.485128  4742 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.537693  5065 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:21.526425  5004 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.539661  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.573745  1717 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.495138  5086 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.548969  1471 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:21.532570  1648 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:21.432269  2372 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:21.427277  5051 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.552820  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.554044  1725 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:21.594988  1574 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.502700  4990 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:21.580762  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.583201  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.605120  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.595623  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.465605  4996 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.466322  4956 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.491083  4988 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:02:21.548154  4875 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.520026  5006 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:21.523087  5008 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.552013  4855 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.623247  1576 raft_consensus.cc:2410] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:21.601498  5067 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:21.609866  1471 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 6, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:21.627904  1722 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.630381  2371 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:02:21.634228  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 288323us
W20260709 13:02:21.719794  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96404.2us
I20260709 13:02:21.765050  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:21.691762  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.768998  1573 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:21.672194  4990 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.138s
I20260709 13:02:21.606132  5065 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:21.771525  4875 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.773523  1721 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:21.774837  5065 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:21.690488  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.605250  5086 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.776611  4875 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:21.730773  1725 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:21.777701  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 322501us
I20260709 13:02:21.738965  1723 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:21.779739  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:21.780534  1723 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.784792  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.785722  1725 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:21.787329  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.793645  5080 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:21.797524  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:21.784631  5006 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.640848  1574 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:21.801821  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.800791  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:21.765998  5057 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.000s
W20260709 13:02:21.811842  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.807852  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.811342  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.813052  1722 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:21.817360  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:21.818917  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:21.819906  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.790674  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.821600  4824 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:21.822926  4824 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:21.820218  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:21.825114  4977 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:21.825770  4977 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:21.826972  4977 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.822037  1725 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:21.830384  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:21.830710  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:21.820667  4898 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:21.834666  4898 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:21.835877  1725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:21.836007  4898 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.838810  1723 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:21.842363  5039 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:21.843051  5039 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:21.843144  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:21.843637  5039 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.846053  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:21.848663  4824 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.711006  5068 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:21.850628  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:21.850888  1725 raft_consensus.cc:606] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:21.697973  5029 raft_consensus.cc:2764] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:21.849958  1471 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 5, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:21.713213  5051 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:21.853662  1725 raft_consensus.cc:614] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:21.864782  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:21.874120  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:21.807860  5067 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:21.883550  1718 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:21.803333  5099 raft_consensus.cc:2764] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:21.850998  4990 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.799643  4994 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:02:21.799808  4981 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:02:21.895804  1724 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:21.899878  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:21.900321  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.065779  1573 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.852818  5068 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:22.087410  5045 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.957s	user 0.011s	sys 0.002s
I20260709 13:02:22.088224  1643 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:22.089037  5053 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.817s	user 0.004s	sys 0.006s
W20260709 13:02:21.940755  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.089994  2376 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:02:22.012699  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 166736us
I20260709 13:02:22.095322  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.098624  1573 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:22.101895  1574 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:22.103318  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.103871  1574 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:02:22.104949  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.105902  4956 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:21.930186  1471 catalog_manager.cc:5697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:21.704206  1557 tablet_service.cc:1558] Processing DeleteTablet for tablet b837a5ec9a0d4f2a8635dedf45d51aa2 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:44738
W20260709 13:02:22.053158  5033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.010s
I20260709 13:02:21.882194  5051 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:21.850659  5057 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:21.943501  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:22.064159  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 187799us
W20260709 13:02:21.995693  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.896699  5073 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:21.908723  5067 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:21.912415  4990 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:22.070631  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 171747us
I20260709 13:02:22.074007  5100 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:22.109010  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 209236us
W20260709 13:02:22.112737  1717 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.003s
I20260709 13:02:22.131721  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.144411  1575 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:02:22.173497  1575 pending_rounds.cc:85] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:22.173714  1575 pending_rounds.cc:107] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:22.173998  1575 raft_consensus.cc:2889] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:22.176026  1573 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:22.157665  5073 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:22.185580  5073 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.136632  5110 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:22.158453  5067 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.159279  4855 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.154066  5004 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:22.215183  5086 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:22.216761  1723 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:22.198253  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:22.164464  1717 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:22.166078  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.219066  1569 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:22.157056  1576 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:22.120311  1574 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:22.231755  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:22.201217  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.203646  1573 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:22.208214  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:22.247040  1575 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:22.136369  1471 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 6, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:22.213521  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.256397  5006 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.257673  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:22.259297  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:22.260797  5086 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:22.262537  5088 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:22.263787  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:22.264727  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.265334  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.270129  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.276129  1471 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 3, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.278074  1569 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:22.277964  1688 leader_election.cc:400] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:22.204540  5110 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.160769  4956 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:22.281412  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:22.278395  1471 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:22.283407  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 82276us
I20260709 13:02:22.291031  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:22.144709  5033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.007s	sys 0.010s
I20260709 13:02:22.159878  4990 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.286382  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.293110  1576 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:22.294083  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:22.295318  1574 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:22.237572  4742 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.006s
I20260709 13:02:22.206291  5051 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:22.305505  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:22.309128  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.296371  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:22.338991  1574 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:22.263370  5088 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:22.299897  5110 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:22.362098  5039 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.004s
I20260709 13:02:22.326304  1471 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 6, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.322445  4981 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.153698  5099 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.148s
W20260709 13:02:22.369668  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 495620us
I20260709 13:02:22.175249  4824 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:02:22.374591  1575 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:22.210960  4898 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.125s
W20260709 13:02:22.358454  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53070.5us
I20260709 13:02:22.439108  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:22.285622  5016 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:22.358724  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49772.7us
I20260709 13:02:22.447618  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.263059  5004 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:02:22.345757  5076 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:22.364832  5059 raft_consensus.cc:2764] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:22.366175  5029 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:22.432132  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.428953  1471 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:22.433292  5100 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:22.382687  4855 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:22.457710  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.630s (SENT)
I20260709 13:02:22.366734  5051 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:22.489385  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 291206us
W20260709 13:02:22.505693  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 319557us
W20260709 13:02:22.503141  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64147.9us
I20260709 13:02:22.507349  1688 leader_election.cc:400] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:22.508093  1688 leader_election.cc:403] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:22.509970  1716 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:22.511075  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 371914us
I20260709 13:02:22.506750  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:22.443235  5088 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:22.520972  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.396029  5039 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.458932  5123 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.522951  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65313.9us
I20260709 13:02:22.459146  5086 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:22.528239  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:22.538204  1575 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:22.525413  1539 leader_election.cc:400] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:22.538846  1539 leader_election.cc:403] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:22.541481  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.641s (SENT)
I20260709 13:02:22.557103  1573 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:22.554867  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 107365us
I20260709 13:02:22.510587  4981 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:22.513501  5076 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.547279  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.544535  5113 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:22.504360  5100 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.559723  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.522243  5029 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:22.535391  4855 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:22.567204  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:22.518230  4956 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:02:22.569133  5100 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:22.584102  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.589039  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:22.586721  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:22.587190  1539 leader_election.cc:400] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:22.590524  1539 leader_election.cc:403] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:22.552783  5123 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:22.599232  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:22.600396  4981 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:22.575663  4716 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:22.602207  4716 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:22.555702  5095 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:02:22.605360  5051 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:22.609400  5115 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:02:22.611732  5117 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 13:02:22.613169  5104 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.237s	user 0.000s	sys 0.002s
I20260709 13:02:22.614763  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2025) took 1354 ms. Trace:
I20260709 13:02:22.615932  5129 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:22.617030  4716 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.616962  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:22.617475  5118 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:22.618618  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.600485  2372 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:22.616873  5129 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:22.615204  1720 rpcz_store.cc:276] 0709 13:02:21.260696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.260978 (+   282us) service_pool.cc:224] Handling call
0709 13:02:21.261322 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.282271 (+ 20949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.282326 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.294827 (+ 12501us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.444443 (+149616us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b18003ceec0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:02:21.815474 (+371031us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:21.815524 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.815606 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.614335 (+798729us) raft_consensus.cc:1685] finished
0709 13:02:22.614372 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.614681 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84168,"raft.queue_time_us":22318,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":2165,"spinlock_wait_cycles":47678336,"thread_start_us":370789,"threads_started":1,"wal-append.queue_time_us":866002}
I20260709 13:02:22.594650  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.621024  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:22.621022  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2106) took 1155 ms. Trace:
W20260709 13:02:22.629204  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.793s (SENT)
I20260709 13:02:22.629894  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:22.632583  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.633275  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:22.618652  5086 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.639773  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2103) took 1217 ms. Trace:
I20260709 13:02:22.640236  1716 rpcz_store.cc:276] 0709 13:02:21.422534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.422664 (+   130us) service_pool.cc:224] Handling call
0709 13:02:21.422946 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.510481 (+1087535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.639493 (+129012us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400294fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:22.639705 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499705088}
I20260709 13:02:22.626993  1719 rpcz_store.cc:276] 0709 13:02:21.465521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.482718 (+ 17197us) service_pool.cc:224] Handling call
0709 13:02:21.517841 (+ 35123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.534360 (+ 16519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.534407 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.552309 (+ 17902us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.872030 (+319721us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:21.872071 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.872142 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.620400 (+748258us) raft_consensus.cc:1685] finished
0709 13:02:22.620748 (+   348us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.620944 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166832,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":267345,"threads_started":1,"wal-append.queue_time_us":678139}
W20260709 13:02:22.662549  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 68812.6us
W20260709 13:02:22.627346  5008 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.004s
I20260709 13:02:22.671136  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:22.669680  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51149.2us
I20260709 13:02:22.624209  5099 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:22.672899  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.624753  5004 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.623795  4855 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:22.679195  4716 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:22.679261  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50120.7us
W20260709 13:02:22.680799  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.821s (SENT)
I20260709 13:02:22.681495  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:22.683105  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.827s (SENT)
I20260709 13:02:22.626034  5051 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:22.665412  5139 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.694798  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.674890  5099 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.674479  5113 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:22.714687  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 120160us
W20260709 13:02:22.718784  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.913s (SENT)
I20260709 13:02:22.725207  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:22.738502  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.720851  5099 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:22.699931  5130 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:22.691987  5135 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:22.760250  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.094s (SENT)
I20260709 13:02:22.780045  5073 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:22.775653  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 80972.9us
I20260709 13:02:22.784237  2376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 505) took 1358 ms. Trace:
W20260709 13:02:22.785456  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.210s (SENT)
I20260709 13:02:22.790962  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.792141  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.781538  5067 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:22.792115  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54694.2us
I20260709 13:02:22.791746  2376 rpcz_store.cc:276] 0709 13:02:21.426162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.433187 (+  7025us) service_pool.cc:224] Handling call
0709 13:02:22.783907 (+1350720us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b5400547254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:22.784127 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313103104}
I20260709 13:02:22.798200  5073 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:22.798501  5101 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:22.798998  5073 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.800307  5067 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.801043  5029 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:22.776360  5004 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:22.803073  4996 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.803380  5067 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:22.789630  4855 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.733057  5142 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:22.793550  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55149.9us
I20260709 13:02:22.664824  5088 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.801950  5073 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.812389  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:22.828593  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 157563us
I20260709 13:02:22.704723  5139 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:22.901558  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.835809  5127 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:22.902812  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:22.903942  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 91921.4us
I20260709 13:02:22.907027  4981 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.907648  4981 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.907975  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:22.909564  5149 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:02:22.910511  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.817011  5039 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.906167  5099 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:22.912425  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:22.897778  5073 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:22.910367  4981 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:22.955987  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:22.961230  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:22.929510  5059 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:22.963723  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:22.964717  1724 raft_consensus.cc:606] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:22.965186  1724 raft_consensus.cc:614] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:22.957367  4981 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:22.902874  5127 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:22.929764  5006 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:22.901100  5110 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:22.971005  5039 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:22.931279  5100 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:22.964270  5073 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:02:23.030005  5039 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:22.973604  5127 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.051781  5083 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.981915  5006 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:22.986084  4956 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:23.053516  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c55c47030a294c17ad5561abe541b993"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:23.054821  5110 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:23.121904  2371 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:23.057279  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:23.122713  1572 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:23.061240  5154 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.150s	user 0.002s	sys 0.000s
I20260709 13:02:23.064527  5160 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:02:23.034161  5053 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:23.066648  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.143616  5053 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.088055  1724 raft_consensus.cc:606] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:23.121054  5156 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.125819  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.609s (SENT)
I20260709 13:02:23.146997  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:23.032518  5150 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.088302  5083 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.137485  5139 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:23.071785  5039 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.140450  5130 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.087677  4956 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:23.165728  5138 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.165961  4956 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.166973  5151 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.167603  5151 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.146165  5053 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:23.139319  5099 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.146869  5156 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.174976  5151 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:23.175402  5156 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.077853  5006 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.182488  5142 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.183365  5142 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.192219  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:23.193168  1724 raft_consensus.cc:606] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:23.193567  1724 raft_consensus.cc:614] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:23.207489   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:23.202291  5100 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:23.217633  5083 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:23.222642  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.224210  1568 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:23.229229  1572 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:23.224848   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:23.223953  5142 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:23.245421  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 284268us
I20260709 13:02:23.246769  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:23.237957  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 307097us
I20260709 13:02:23.281795  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:23.282915  1572 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:23.281473  5156 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:23.281252  5016 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:23.283874  5059 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:02:23.295425  5168 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:23.295887  5059 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:23.296725  5059 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:23.298291  5059 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.299954  5059 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:23.290833  5156 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:23.301090  5016 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.296726  5168 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:23.283136  5142 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:02:23.309795  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:23.317448  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:23.324859  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.325192  5166 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:23.319209  5156 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.335386  5168 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.284463  5067 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.350761  5067 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:23.342521  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58848us
I20260709 13:02:23.346632  5016 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.325060  5170 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:23.354202  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:23.356079  1724 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:23.356257  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:23.358265  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.359351  5100 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:23.353397  4956 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.360544  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.361828  4956 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.360870  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:23.352730  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 403187us
I20260709 13:02:23.361869  1647 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:23.362378  1720 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.371287  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:23.383546  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.384730  1720 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:23.385241  1642 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:23.386267  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.387435  1724 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:23.390559  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.388999  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.415513  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.416108  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.418649  2373 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:02:23.420384  1716 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:23.420962  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.422803  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.371277  5016 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:23.423323  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.425662  1642 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:23.422813  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.428069  1649 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:23.417156  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.415230  1460 catalog_manager.cc:5028] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): tablet b837a5ec9a0d4f2a8635dedf45d51aa2 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
W20260709 13:02:23.470335  1460 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57581.3us
I20260709 13:02:23.470469  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:23.470664  5150 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:23.472206  5099 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:23.473165  1649 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:23.475720  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.471220  2377 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.471274  2379 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:02:23.477628  5163 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:02:23.484983  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.486068  1651 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:23.416069  5170 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:23.419514  4956 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.491580  1539 leader_election.cc:400] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8.
I20260709 13:02:23.427363  5067 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:23.430569  5166 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:23.504115  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:23.515832  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:23.517987  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.518510  1572 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:02:23.518648  1651 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.519244  1572 pending_rounds.cc:85] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 3
I20260709 13:02:23.519821  1572 pending_rounds.cc:107] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:02:23.520390  1572 raft_consensus.cc:2889] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.520814  1642 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:02:23.521591  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:23.522311  2379 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:02:23.523336  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.519335  1719 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:23.525925  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.496234  4956 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.527570  1649 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:23.492019  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.491398  5150 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:02:23.529014  2374 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 4.
I20260709 13:02:23.531443  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:23.492758  5016 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:23.537425  5016 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.540607  5171 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 13:02:23.545656  2377 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
W20260709 13:02:23.546330  5164 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:23.546965  1651 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:23.548206  5166 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.550978  5166 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:23.525297  5099 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:23.554685  5099 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.555126  5099 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.494936  5067 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.560695  1649 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:23.562969  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.565683  4956 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:23.568861  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:23.570971  1651 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:23.559258  5170 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.575084  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.575526  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.575917  1649 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:23.576313  1642 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:23.575996  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.581085  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:23.581323  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:23.560766  5067 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.592710  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:23.597553  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 227752us
I20260709 13:02:23.496815  5127 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.572131  5065 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:23.577958  5171 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:23.606932  5065 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.607231  5171 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.608449  5127 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.606964  1650 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.609443  5033 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:23.609898  5171 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.611543  5065 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.571827  5150 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:23.545701  5039 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.576982  5142 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.625762  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 104256us
I20260709 13:02:23.627039  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:23.628741  1646 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:23.631210  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:23.624660  5150 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:23.614488  5156 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.634914  5160 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.640605  5166 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:23.651437  5083 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.626564  5142 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.661749  5166 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:23.708292  5166 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.709672  5156 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:23.698253  4956 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 13:02:23.719539  5126 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.204s	user 0.000s	sys 0.010s
I20260709 13:02:23.720861  5156 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:23.721496  5156 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:23.725687  5121 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.290s	user 0.019s	sys 0.007s
I20260709 13:02:23.727778  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 508) took 2271 ms. Trace:
I20260709 13:02:23.728307  2375 rpcz_store.cc:276] 0709 13:02:21.461100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.480409 (+ 19309us) service_pool.cc:224] Handling call
0709 13:02:21.551629 (+ 71220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.672450 (+120821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.685022 (+ 12572us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.849682 (+164660us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.164136 (+314454us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:22.172843 (+  8707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.205241 (+ 32398us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.724811 (+1519570us) spinlock_profiling.cc:243] Waited 9.49 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:23.727238 (+  2427us) raft_consensus.cc:1685] finished
0709 13:02:23.727286 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.727619 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17548,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":128,"spinlock_wait_cycles":21824896,"thread_start_us":262312,"threads_started":1,"wal-append.queue_time_us":500597}
I20260709 13:02:23.735174  5065 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.736202  5065 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.739693  5065 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:23.741820  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2193) took 1550 ms. Trace:
I20260709 13:02:23.742383  1576 rpcz_store.cc:276] 0709 13:02:22.191147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.379906 (+188759us) service_pool.cc:224] Handling call
0709 13:02:22.431661 (+ 51755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.441860 (+ 10199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.441915 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.442765 (+   850us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.584006 (+141241us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:22.665349 (+ 81343us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.669657 (+  4308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.669773 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.728569 (+1058796us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b180043e520. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:23.741326 (+ 12757us) raft_consensus.cc:1685] finished
0709 13:02:23.741481 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.741676 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11876,"prepare.queue_time_us":87109,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":130805,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":8891,"spinlock_wait_cycles":9703808,"thread_start_us":67273,"threads_started":1,"wal-append.queue_time_us":231305}
I20260709 13:02:23.757807  5051 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:23.758411  5051 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:23.759415  5051 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.760773  5051 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.742502  5150 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.764575  5016 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:23.765667  5016 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:02:23.767956  5174 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:02:23.769531  5123 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:23.772102  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 135363us
I20260709 13:02:23.673653  1572 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:23.699186  5083 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:23.700619  1568 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:23.783097  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.784881  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.785862  1718 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:23.787500  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.788846  1724 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:23.791183  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.795082  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.796550  1716 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:23.800319  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.801642  1716 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:23.802050  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:23.804708  1614 connection.cc:256] RPC call timeout handler was delayed by -1.2804649139999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:02:23.805244  1614 connection.cc:256] RPC call timeout handler was delayed by -1.0968629750000218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
I20260709 13:02:23.805589  1718 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:23.813417  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.823093  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.825883  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.806674  1614 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:23.828951  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:23.831148  1725 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:23.835793  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.837205  1576 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:23.838973  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.843293  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.849956  1570 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:23.851284  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:23.852372  1576 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.856706  1720 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:23.862489  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:23.865643  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:23.866071  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:23.866456  2373 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.866837  1570 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:23.858332  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:23.869549  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:23.870359  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:23.873135  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:23.873994  1725 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.874444  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.874456  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.876094  1724 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:23.823135  5016 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.878973  2379 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:23.872043  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:23.882200  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.885314  1570 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:23.803478  5160 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:02:23.888053  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:23.805414  5118 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:02:23.859788  4996 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:23.840807  5175 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:23.897172  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.899322  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:23.900068  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.900959  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.901535  1724 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:23.878554  5170 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:23.888154  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 109642us
I20260709 13:02:23.903349  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:23.903405  1576 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:23.907778  1716 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:23.895315  4996 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.887501  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:23.913329  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.914243  5181 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:23.895098  5176 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:02:23.926709  1718 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.928973  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:23.970228  1716 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:23.926465  2373 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:23.971720  1719 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:23.929983  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:23.887732  5111 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:23.931603  1567 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:23.931802  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:23.932657  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 152798us
I20260709 13:02:23.934315  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.934579  1725 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:23.937428  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:23.928421  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.979250  1576 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:23.979133  5123 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:23.902513  5039 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:23.980933  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43622.4us
I20260709 13:02:23.990923  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:23.931746  5142 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:23.981853  1718 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:23.982429  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:23.983433  1719 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:23.930591  4996 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.916778  5170 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:23.885462  5016 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:23.931013  5175 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:24.022643  5102 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 2.223s	user 0.009s	sys 0.009s
I20260709 13:02:24.026044  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:23.995617  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63434.5us
I20260709 13:02:24.027282  1576 raft_consensus.cc:2410] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:23.995134  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:24.044664  1570 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:23.989814  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 161011us
I20260709 13:02:24.025228  1567 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.975605  5111 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:24.048688  5111 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:23.988489  5138 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:23.985589  5181 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:24.051728  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 167140us
I20260709 13:02:24.059803  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:24.032671  5170 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.077440  1567 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:24.071928  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.053049  1724 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:24.073318  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:24.023808  4824 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:24.082536  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.084717  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:24.054005  5138 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.087767  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:24.082439  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:24.085577  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:24.089720  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:24.089857  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:24.086822  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:24.059315  5057 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.103751  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:24.106031  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:24.075321  5181 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:24.101046  5138 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:24.087428  5039 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:24.073792  5118 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:02:24.146503  1570 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:24.105585  5160 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:24.153129  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2114) took 2637 ms (client timeout 1567 ms). Trace:
I20260709 13:02:24.129580  5181 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:24.159751  1722 rpcz_store.cc:269] 0709 13:02:21.515228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.550324 (+ 35096us) service_pool.cc:224] Handling call
0709 13:02:24.152767 (+2602443us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b5400233554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:24.152984 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50695040}
I20260709 13:02:24.101663  4824 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:24.181070  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 91462.6us
I20260709 13:02:24.151482  5194 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:24.161180  5178 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:24.183976  1570 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:24.190191  1568 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:24.189806  1570 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:24.169364  5181 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.192109  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:24.197417  5068 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:24.247187  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:24.247496  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55474.2us
I20260709 13:02:24.248803  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.250092  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.250725  1649 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:24.252121  5138 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:24.187974  5051 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:24.212857  5178 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:24.213266  5083 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:24.199342  5192 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.217347  5183 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260709 13:02:24.157471  5175 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:24.189113  5118 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:02:24.221499  5203 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:24.212924  5182 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260709 13:02:24.260176  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:24.231950  5196 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.284693  5203 consensus_queue.cc:1243] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:24.285269  5196 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:24.286046  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:24.266178  5099 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.710s	user 0.017s	sys 0.000s
W20260709 13:02:24.232635  5166 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.006s
I20260709 13:02:24.287840  5099 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:24.288651  5166 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:24.290512  5099 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.233116  5016 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.291028  5166 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.193738  5190 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
W20260709 13:02:24.263477  5067 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.642s	user 0.014s	sys 0.003s
I20260709 13:02:24.292089  5067 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:24.292724  5192 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:24.295085  5192 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.295413  5067 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:24.288066  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.296485  5190 consensus_queue.cc:1243] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:24.265635  4824 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.297605  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:24.266996  5107 raft_consensus.cc:993] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:24.299170  5083 raft_consensus.cc:1081] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:24.288017  5196 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.260093  5057 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:02:24.257206  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.309121  1650 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:24.309419  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 509) took 1732 ms. Trace:
I20260709 13:02:24.183782  5194 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.237814  5181 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:24.259268  5142 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:24.310076  2371 rpcz_store.cc:276] 0709 13:02:22.577212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.586311 (+  9099us) service_pool.cc:224] Handling call
0709 13:02:24.309065 (+1722754us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540057e4d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:24.309327 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221216000}
I20260709 13:02:24.310813  5142 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:24.310271  5194 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:24.312039  5142 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:24.312590  5181 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:24.313026  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2191) took 2122 ms (client timeout 2635 ms). Trace:
W20260709 13:02:24.313491  1573 rpcz_store.cc:269] 0709 13:02:22.190614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.295954 (+105340us) service_pool.cc:224] Handling call
0709 13:02:24.312754 (+2016800us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400001c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:24.312931 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513404160}
I20260709 13:02:24.316669  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.317729  1650 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:24.318784  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.254542  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:24.299000  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.320710  1575 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:24.320839  1646 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:24.321028  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:24.321832  5079 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:24.322485  1642 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:24.245544  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:24.261861  4990 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:24.316757  4907 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:02:24.309763  1648 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:26.043195  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 476) took 2261 ms (client timeout 1759 ms). Trace:
I20260709 13:02:25.170399  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1567) took 3703 ms. Trace:
I20260709 13:02:26.044442  1571 rpcz_store.cc:276] 0709 13:02:21.467181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.470166 (+  2985us) service_pool.cc:224] Handling call
0709 13:02:21.893683 (+423517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:21.893958 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:22.286336 (+392378us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.555757 (+269421us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:22.555848 (+    91us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:22.555942 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.556050 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.169160 (+2613110us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b18002b37e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4
0709 13:02:25.170141 (+   981us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400025dd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:25.170318 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14055,"prepare.queue_time_us":556092,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":122,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":9468,"spinlock_wait_cycles":919576192,"thread_start_us":647237,"threads_started":2,"wal-append.queue_time_us":435118,"wal-append.run_cpu_time_us":1741,"wal-append.run_wall_time_us":1079884}
W20260709 13:02:25.177910  5194 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.865s	user 0.008s	sys 0.008s
I20260709 13:02:26.048305  5194 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.051761  5194 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:25.286839  5210 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.064937  5210 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.065619  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1584) took 3514 ms. Trace:
I20260709 13:02:26.066272  1575 rpcz_store.cc:276] 0709 13:02:22.551071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.552901 (+  1830us) service_pool.cc:224] Handling call
0709 13:02:24.320473 (+1767572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.321285 (+   812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.065350 (+1744065us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400001c50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:26.065546 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4064378752}
I20260709 13:02:25.338826  1570 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:25.366516  5212 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.068724  5212 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.069470  5210 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.070565  1575 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:26.071504  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2208) took 2105 ms. Trace:
I20260709 13:02:25.394707  1647 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:02:25.413666  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1425) took 4005 ms (client timeout 1529 ms). Trace:
I20260709 13:02:26.075260  1575 rpcz_store.cc:276] 0709 13:02:23.966572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.069835 (+2103263us) service_pool.cc:224] Handling call
0709 13:02:26.070364 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.071152 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.071429 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:26.075793  1643 rpcz_store.cc:269] 0709 13:02:21.408740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.409626 (+   886us) service_pool.cc:224] Handling call
0709 13:02:25.413380 (+4003754us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54001ba8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:25.413576 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482879744}
I20260709 13:02:25.416797  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 360) took 1570 ms. Trace:
W20260709 13:02:25.057113  4977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.019s	sys 0.042s
W20260709 13:02:24.782410  5039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.014s	sys 0.034s
I20260709 13:02:24.322583  5079 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.283890  5079 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:25.368743  5127 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.285934  5127 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:25.285748  4875 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.289435  4875 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:25.398582  4907 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.293493  4907 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:25.118511  4996 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.078s	user 0.004s	sys 0.011s
I20260709 13:02:24.372066  5111 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.300778  5127 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:26.302304  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.301901  4875 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.302696  5111 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:24.337565  4990 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:24.418409  5138 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:25.350340  5156 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.322611  5156 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.337101  5160 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:26.326678  5160 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:25.437510  5168 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:26.329597  5168 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:25.163153  5170 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.074s	user 0.009s	sys 0.009s
W20260709 13:02:25.360503  5173 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.878s	user 0.010s	sys 0.006s
W20260709 13:02:26.339532  5173 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 2.858s	user 0.012s	sys 0.007s
W20260709 13:02:25.185811  5175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.036s	sys 0.118s
W20260709 13:02:25.235760  5184 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.259s	user 0.006s	sys 0.000s
I20260709 13:02:25.519693  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2110) took 4050 ms. Trace:
I20260709 13:02:26.343720  1723 rpcz_store.cc:276] 0709 13:02:21.469036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.841094 (+372058us) service_pool.cc:224] Handling call
0709 13:02:21.841674 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.217369 (+375695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.519444 (+3302075us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400294ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:25.519624 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862086528}
I20260709 13:02:25.521968  1574 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:26.345188  1574 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:26.346084  1574 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:25.528327  5225 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.347826  5225 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.350662  5225 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:25.534611  5227 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:26.351641  5227 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:25.538964  5228 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.355147  5228 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.347371   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:25.564257  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2201) took 1613 ms. Trace:
I20260709 13:02:26.358376  1568 rpcz_store.cc:276] 0709 13:02:23.950450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.983283 (+ 32833us) service_pool.cc:224] Handling call
0709 13:02:25.563957 (+1580674us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000276d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:25.564156 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339849088}
I20260709 13:02:26.360790  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2213) took 2268 ms. Trace:
I20260709 13:02:26.361183  1568 rpcz_store.cc:276] 0709 13:02:24.092384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.359811 (+2267427us) service_pool.cc:224] Handling call
0709 13:02:26.360185 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.360427 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.360456 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.360501 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:26.360545 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.360602 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.360717 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:25.583070  2344 connection.cc:256] RPC call timeout handler was delayed by -1.1017176429999722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
W20260709 13:02:25.738240  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.615s	user 0.039s	sys 0.014s
W20260709 13:02:26.362774  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.240s	user 0.040s	sys 0.015s
W20260709 13:02:26.362676  2344 connection.cc:256] RPC call timeout handler was delayed by -1.0216176429999564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260709 13:02:26.363263  2344 connection.cc:256] RPC call timeout handler was delayed by -1.0207176429999549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
W20260709 13:02:26.363293  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 2.24152e+06us
I20260709 13:02:25.749611  1651 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:26.364264  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:26.364965  1651 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:26.365826  5127 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:26.366323  5127 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.366739  5127 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.366536  1651 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.347443  1574 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:25.769203  4945 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.796877  5006 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.370254  5006 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.371426  1570 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.001135  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:26.010622  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 282168us
I20260709 13:02:26.015195  4981 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:26.018406  1724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.883s	user 0.017s	sys 0.049s
W20260709 13:02:26.375203  1724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.240s	user 0.019s	sys 0.050s
I20260709 13:02:26.376273  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2136) took 2293 ms. Trace:
I20260709 13:02:26.025602  5234 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:25.583362  5229 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.378183  5234 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.378504  5229 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.036085  5233 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.380589  5230 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:24.269826  2374 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:25.279021  5204 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:26.053347  1648 rpcz_store.cc:269] 0709 13:02:23.781455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.285706 (+504251us) service_pool.cc:224] Handling call
0709 13:02:26.043104 (+1757398us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.390997  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:26.056546  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.023s	sys 0.059s
W20260709 13:02:26.392678  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.313s	user 0.024s	sys 0.060s
I20260709 13:02:26.389204  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:26.394620  4875 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.395540  4875 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.059125  5146 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.397354  5234 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.071849  5212 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.398072  5110 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.398677  5110 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.389194  5204 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:25.367130  5217 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.400003  5217 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.400542  5130 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.390443  4945 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.401103  5130 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.402448  5217 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.403189  4945 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.077505  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1449) took 2459 ms. Trace:
W20260709 13:02:26.062462  1646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.739s	user 0.050s	sys 0.047s
I20260709 13:02:25.311702  5211 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:26.404053  4875 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.403888  1647 rpcz_store.cc:276] 0709 13:02:23.617970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.393805 (+1775835us) service_pool.cc:224] Handling call
0709 13:02:25.394256 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.075362 (+681106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.077431 (+  2069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.379294  5127 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.404548  5211 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.406355  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 473) took 2628 ms. Trace:
I20260709 13:02:26.077061  1567 rpcz_store.cc:276] 0709 13:02:23.846478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.092146 (+245668us) service_pool.cc:224] Handling call
0709 13:02:24.092598 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.102165 (+  9567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.102222 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.102814 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.181646 (+ 78832us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:25.403699 (+1222053us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.416087 (+ 12388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.416233 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.416317 (+    84us) raft_consensus.cc:1685] finished
0709 13:02:25.416362 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.416659 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9109,"prepare.queue_time_us":15135,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":42395,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":5777,"thread_start_us":36837,"threads_started":2,"wal-append.queue_time_us":97596,"wal-append.run_cpu_time_us":2119,"wal-append.run_wall_time_us":1008712}
I20260709 13:02:26.407509  5211 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.106285  5158 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:26.077250  1575 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:26.083482  5194 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.118033  5235 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.120916  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2134) took 2185 ms. Trace:
I20260709 13:02:26.147375  5068 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:26.380662  5233 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.151935  5016 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:26.165570  1614 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 2.081s	user 0.041s	sys 0.062s
I20260709 13:02:26.185022  4898 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.186408  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2200) took 2236 ms. Trace:
I20260709 13:02:26.194442  1644 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
W20260709 13:02:26.195969  1717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.215s	user 0.053s	sys 0.056s
W20260709 13:02:26.245204  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.117s	user 0.011s	sys 0.018s
I20260709 13:02:26.274416  5199 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:26.276050  1541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.112s	user 0.011s	sys 0.101s
W20260709 13:02:26.277855  5039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.103s	user 0.014s	sys 0.037s
W20260709 13:02:26.265499  4977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.127s	user 0.024s	sys 0.043s
I20260709 13:02:26.299394  4996 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.296264  4907 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.304541  5065 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.065814  5142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.018s	sys 0.051s
I20260709 13:02:26.304221  5067 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.311493  1576 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:26.312702  5073 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:26.315447  1474 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.317776  4742 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:26.318389  4990 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.318943  5088 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:26.319550  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 415) took 4119 ms. Trace:
I20260709 13:02:26.334084  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 472) took 2556 ms. Trace:
I20260709 13:02:26.341259  5168 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:26.342042  5175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.046s	user 0.037s	sys 0.119s
I20260709 13:02:26.342518  5170 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:26.377851  2342 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.759s (SENT) [suppressed 17 similar messages]
I20260709 13:02:26.381081  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:26.381083  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:26.382546  5229 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.374926  4981 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.383914  1651 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.376713  1724 rpcz_store.cc:276] 0709 13:02:24.082802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.086288 (+  3486us) service_pool.cc:224] Handling call
0709 13:02:24.086621 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.091289 (+  4668us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:24.134896 (+ 43607us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.375818 (+2240922us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.375887 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:26.375948 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.376042 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.376207 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":102200,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":91,"raft.queue_time_us":147,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":2281997,"threads_started":2,"wal-append.queue_time_us":1035723,"wal-append.run_cpu_time_us":1848,"wal-append.run_wall_time_us":1020781}
I20260709 13:02:26.384979  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:26.385842  1574 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:02:26.381265  5230 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.397068  5146 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.390942  5225 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.083415  1643 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:26.107191  5232 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:26.107735  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.084256  5198 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:26.409873  5130 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.455014  5228 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.597867  4990 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.455885  1572 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
W20260709 13:02:26.600840  5142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.285s	user 0.020s	sys 0.060s
I20260709 13:02:26.600984  1642 rpcz_store.cc:276] 0709 13:02:23.777237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.269921 (+2492684us) service_pool.cc:224] Handling call
0709 13:02:26.270438 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.331042 (+ 60604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.331085 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.331141 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:26.331184 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.333570 (+  2386us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b54000b9d54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:26.333767 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.333993 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138691968}
I20260709 13:02:26.604934  1571 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:26.605085  5232 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.605278  1474 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 7 to 9, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 9 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.606740  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2206) took 2654 ms. Trace:
I20260709 13:02:26.607213  1571 rpcz_store.cc:276] 0709 13:02:23.952206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.047211 (+2095005us) service_pool.cc:224] Handling call
0709 13:02:26.047764 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.605580 (+557816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.606415 (+   835us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400125cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:26.606666 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279933312}
I20260709 13:02:26.396054  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:26.606760  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 474) took 2829 ms. Trace:
I20260709 13:02:26.400211  5204 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.610140  5204 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.608907  1642 rpcz_store.cc:276] 0709 13:02:23.780487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.605775 (+2825288us) service_pool.cc:224] Handling call
0709 13:02:26.606160 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.606361 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.606397 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.606483 (+    86us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:26.606517 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.606580 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.606694 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.609578  1474 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 5 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 6 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.600553  1721 rpcz_store.cc:276] 0709 13:02:22.200499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.201805 (+  1306us) service_pool.cc:224] Handling call
0709 13:02:22.236697 (+ 34892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.236970 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:22.553843 (+316873us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.307454 (+3753611us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b1800410980. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:02:26.318685 (+ 11231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.318749 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:26.318799 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.319306 (+   507us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.319472 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":313937,"prepare.run_cpu_time_us":239,"prepare.run_wall_time_us":5232,"raft.queue_time_us":1363,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":4937344,"thread_start_us":224335,"threads_started":2,"wal-append.queue_time_us":3765778}
W20260709 13:02:26.609097  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 213162us
W20260709 13:02:26.461925  5242 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:02:26.390241  5227 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.602177  1574 pending_rounds.cc:85] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:26.615589  1574 pending_rounds.cc:107] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:02:26.615684  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:26.616209  1574 raft_consensus.cc:2889] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.616390  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 478) took 2795 ms. Trace:
I20260709 13:02:26.616923  5211 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:26.429877  5217 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.617393  5211 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:26.507364  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08002dc480 after lost signal to thread 434
I20260709 13:02:26.617604  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2204) took 2666 ms. Trace:
I20260709 13:02:26.618403  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:26.617823  5211 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.619411  4875 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:26.603799  5199 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.621030  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:26.622097  5210 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:26.618780  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08002dcce0 after lost signal to thread 1508
I20260709 13:02:26.625939  5230 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.602545  4742 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:26.629314  5210 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:26.630224  4742 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:26.418725  5194 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:26.649048  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 204) took 32256 ms (client timeout 15000 ms). Trace:
I20260709 13:02:26.553315  5212 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.652753  5212 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:26.658385  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.603621  5088 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.604138  5065 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:26.670279  1717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.690s	user 0.054s	sys 0.063s
I20260709 13:02:26.627249  5146 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:26.669212  4999 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:02:26.683501  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:26.628440  1649 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:26.629405  5232 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.631323  4742 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.630606  5199 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.599977  4898 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.697964  1651 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:26.632594  5067 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.630889  4945 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:26.634120  4990 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:26.634307  5230 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:26.635165  5225 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.635116  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:26.637305  4981 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.637856  5073 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:26.701840  4994 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:02:26.406764  1647 rpcz_store.cc:276] 0709 13:02:23.777544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.405354 (+2627810us) service_pool.cc:224] Handling call
0709 13:02:26.405754 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.405954 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.406002 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.406050 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:26.406087 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.406150 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.406280 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.617642  5217 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.507140  5156 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.742770  5217 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.743261  5156 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.727152  4981 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:02:26.597626  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 219453us
I20260709 13:02:26.418371  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2209) took 2451 ms. Trace:
I20260709 13:02:26.419080  5235 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.424119  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:26.641542  5198 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:26.416972  5158 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:26.642800  1614 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.558s	user 0.045s	sys 0.068s
I20260709 13:02:26.396060  5006 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:26.423550  1646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.100s	user 0.051s	sys 0.047s
W20260709 13:02:26.644217  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.647207  5039 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:26.603245  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2212) took 2614 ms. Trace:
I20260709 13:02:26.759408  1576 rpcz_store.cc:276] 0709 13:02:23.988941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.310561 (+2321620us) service_pool.cc:224] Handling call
0709 13:02:26.311185 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.602991 (+291806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.603191 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.717752  4990 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:26.616734  1642 rpcz_store.cc:276] 0709 13:02:23.821433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.615091 (+2793658us) service_pool.cc:224] Handling call
0709 13:02:26.615588 (+   497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.615960 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.615987 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.616028 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:26.616065 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.616121 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.616272 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:26.664573  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.537s	user 0.019s	sys 0.019s
I20260709 13:02:26.729048  5067 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:26.634380  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 249514us
I20260709 13:02:26.751488  5227 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.769474  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:26.742821  5194 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.673382  5059 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.777786  5204 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.767858  5006 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:02:26.778682  4898 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.673943  5065 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.779367  4898 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:26.779760  4898 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:26.649600  1471 rpcz_store.cc:269] 0709 13:01:54.392570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.392765 (+   195us) service_pool.cc:224] Handling call
0709 13:02:26.648943 (+32256178us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:22.911138 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:22.969573 (+ 58435us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:22.969643 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:02:22.972922 (+  3279us) tablet.cc:662] Decoding operations
0709 13:02:23.174630 (+201708us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:23.174746 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:23.174782 (+    36us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:02:23.188177 (+ 13395us) tablet.cc:701] Row locks acquired
0709 13:02:23.200638 (+ 12461us) write_op.cc:260] PREPARE: finished
0709 13:02:23.235672 (+ 35034us) write_op.cc:270] Start()
0709 13:02:23.244537 (+  8865us) write_op.cc:276] Timestamp: P: 1783602143223610 usec, L: 0
0709 13:02:23.244587 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:02:23.283582 (+ 38995us) log.cc:844] Serialized 5551 byte log entry
0709 13:02:23.481540 (+197958us) op_driver.cc:464] REPLICATION: finished
0709 13:02:25.334408 (+1852868us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc8377a4ef0
0709 13:02:25.343903 (+  9495us) write_op.cc:301] APPLY: starting
0709 13:02:25.344026 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:25.344421 (+   395us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:25.344467 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:25.346598 (+  2131us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:25.347035 (+   437us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:02:25.347100 (+    65us) write_op.cc:312] APPLY: finished
0709 13:02:25.364823 (+ 17723us) log.cc:844] Serialized 101 byte log entry
0709 13:02:25.384179 (+ 19356us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:25.384595 (+   416us) write_op.cc:454] Released schema lock
0709 13:02:25.384772 (+   177us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:26.340064 (+955292us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:26.340108 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1236809,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":1861972,"apply.run_cpu_time_us":16487,"apply.run_wall_time_us":45585,"num_ops":14,"prepare.queue_time_us":68708,"prepare.run_cpu_time_us":16254,"prepare.run_wall_time_us":415755,"raft.queue_time_us":119073,"raft.run_cpu_time_us":20437,"raft.run_wall_time_us":2871516,"replication_time_us":236726,"spinlock_wait_cycles":11958528,"thread_start_us":2928422,"threads_started":5,"wal-append.queue_time_us":48858,"wal-append.run_cpu_time_us":8258,"wal-append.run_wall_time_us":2053070}]]}
I20260709 13:02:26.780916  4990 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:26.675590  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 290892us
I20260709 13:02:26.675040  5212 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.784278  5227 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.676435  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2123) took 2895 ms. Trace:
W20260709 13:02:26.679682  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.600124  1572 pending_rounds.cc:85] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 3
I20260709 13:02:26.787053  1572 pending_rounds.cc:107] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:26.771975  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:26.598448  1573 rpcz_store.cc:276] 0709 13:02:23.950113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.314907 (+364794us) service_pool.cc:224] Handling call
0709 13:02:24.315433 (+   526us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:26.177629 (+1862196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.177670 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:26.178624 (+   954us) log.cc:844] Serialized 52 byte log entry
0709 13:02:26.182373 (+  3749us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:26.182445 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.182568 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.185910 (+  3342us) raft_consensus.cc:1685] finished
0709 13:02:26.185937 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.186325 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":268,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":139,"thread_start_us":2563,"threads_started":1,"wal-append.queue_time_us":2252}
I20260709 13:02:26.686889  1567 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:26.612154  1474 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 5 to 7, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.788511  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:26.691587  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.790045  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.790848  1567 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 4.
I20260709 13:02:26.692451  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1447) took 3097 ms. Trace:
W20260709 13:02:26.686424  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1967) took 3269 ms (client timeout 1915 ms). Trace:
I20260709 13:02:26.618813  1574 rpcz_store.cc:276] 0709 13:02:23.951463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.521351 (+1569888us) service_pool.cc:224] Handling call
0709 13:02:25.521785 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.616887 (+1095102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.617527 (+   640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.791982  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.461550  5127 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.630779  5170 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.595927  5110 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:26.422801  5233 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.669441  5016 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:26.704167  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.809360  1571 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:26.676484  5134 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:26.639246  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1452) took 3019 ms. Trace:
I20260709 13:02:26.813526  1643 rpcz_store.cc:276] 0709 13:02:23.623553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.082588 (+2459035us) service_pool.cc:224] Handling call
0709 13:02:26.083028 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.638926 (+555898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.639183 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.711038  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:26.718571  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 83557.1us
I20260709 13:02:26.702122  4945 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:26.656530  4956 consensus_queue.cc:1175] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd6d377efe514bc58ae49c3f4a107232" responder_term: 8 status { last_received { term: 6 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.5-7.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:26.728264  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 479) took 2906 ms. Trace:
I20260709 13:02:26.820356  4945 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.748791  1575 rpcz_store.cc:276] 0709 13:02:23.966938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.076301 (+2109363us) service_pool.cc:224] Handling call
0709 13:02:26.076973 (+   672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.417902 (+340929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.418276 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.821707  5199 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:26.822088  4945 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.822206  5199 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:26.822860  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.823292  5199 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.753175  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 480) took 2728 ms. Trace:
I20260709 13:02:26.824576  1648 rpcz_store.cc:276] 0709 13:02:24.024441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.752175 (+2727734us) service_pool.cc:224] Handling call
0709 13:02:26.752599 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.752807 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.752833 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.752880 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.752914 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.752988 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.753110 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.826556  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:26.827293  1571 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:26.755477  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 2.67192e+06us
I20260709 13:02:26.756104  5156 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:26.751178  4994 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:26.728017  4996 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.722128  4742 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.828956  5016 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.829493  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:26.829493  4994 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:26.830296  5016 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.830998  4994 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.725428  5146 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.760831  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.832742  4994 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:26.833058  1576 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:26.834352  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.603659  1570 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:02:26.836576  1570 pending_rounds.cc:85] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:26.836915  1570 pending_rounds.cc:107] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:02:26.837234  1570 raft_consensus.cc:2889] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:26.765130  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 2.72223e+06us
W20260709 13:02:26.664876  1541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.500s	user 0.015s	sys 0.105s
I20260709 13:02:26.765656  1720 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:26.767033  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 481) took 2455 ms. Trace:
I20260709 13:02:26.777688  5059 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.785673  1717 rpcz_store.cc:276] 0709 13:02:23.781136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.781295 (+   159us) service_pool.cc:224] Handling call
0709 13:02:23.831051 (+ 49756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.979503 (+148452us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:23.979538 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.980157 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.675948 (+2695791us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:26.676011 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.676103 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.676191 (+    88us) raft_consensus.cc:1685] finished
0709 13:02:26.676204 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.676371 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17205,"raft.queue_time_us":294730,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":12238,"thread_start_us":2843075,"threads_started":2,"wal-append.queue_time_us":1185228,"wal-append.run_cpu_time_us":2049,"wal-append.run_wall_time_us":1003539}
W20260709 13:02:26.627138  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd040 after lost signal to thread 1600
I20260709 13:02:26.787367  1572 raft_consensus.cc:2889] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:26.845187  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd020 after lost signal to thread 1601
I20260709 13:02:26.419225  1716 rpcz_store.cc:276] 0709 13:02:23.936196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.983902 (+ 47706us) service_pool.cc:224] Handling call
0709 13:02:23.984606 (+   704us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:24.053601 (+ 68995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.053662 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:26.036094 (+1982432us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b2c000590a8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:02:26.065607 (+ 29513us) log.cc:844] Serialized 52 byte log entry
0709 13:02:26.101594 (+ 35987us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:26.119774 (+ 18180us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.120308 (+   534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.120446 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.120498 (+    52us) raft_consensus.cc:1685] finished
0709 13:02:26.120609 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.120836 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29942,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":196574,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":1493,"spinlock_wait_cycles":11314560,"thread_start_us":120252,"threads_started":3,"wal-append.queue_time_us":39562}
W20260709 13:02:26.784309  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.871s (SENT)
I20260709 13:02:26.846907  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:26.793387  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.462383  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 512) took 2683 ms. Trace:
I20260709 13:02:26.850214  2374 rpcz_store.cc:276] 0709 13:02:23.778829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.801455 (+ 22626us) service_pool.cc:224] Handling call
0709 13:02:24.269305 (+467850us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:02:24.269520 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.388761 (+2119241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.462279 (+ 73518us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16060288}
I20260709 13:02:26.699069  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1451) took 3079 ms. Trace:
I20260709 13:02:26.805337  5204 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:26.788472  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:26.853358  1573 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:26.853978  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.854780  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2199) took 2905 ms. Trace:
I20260709 13:02:26.794338  1567 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:26.854024  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 69785.6us
I20260709 13:02:26.857722  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:26.858423  2401 debug-util.cc:398] Leaking SignalData structure 0x7b080009c9a0 after lost signal to thread 1675
I20260709 13:02:26.796409  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:26.858544  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.907s (SENT)
I20260709 13:02:26.836189  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.798442  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:26.791877  1649 rpcz_store.cc:276] 0709 13:02:23.595460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.328664 (+733204us) service_pool.cc:224] Handling call
0709 13:02:24.329183 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.684878 (+2355695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.692131 (+  7253us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54000f5ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:26.692379 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5287706880}
I20260709 13:02:26.815032  1568 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:26.734270  5211 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.819037  5225 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.817967  1474 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 1 to 8, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:02:26.861922  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.960s (SENT)
I20260709 13:02:26.840667  1642 rpcz_store.cc:276] 0709 13:02:24.311990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.765129 (+2453139us) service_pool.cc:224] Handling call
0709 13:02:26.765787 (+   658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.766000 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.766035 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.766359 (+   324us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:26.766414 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.766488 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.766965 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.821269  1647 rpcz_store.cc:276] 0709 13:02:23.821767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.726966 (+2905199us) service_pool.cc:224] Handling call
0709 13:02:26.727397 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.727734 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:26.727779 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.727849 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:26.727914 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.727998 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.728172 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:26.866807  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.961s (SENT)
I20260709 13:02:26.848189  1574 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:26.796468  5212 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:26.851852  1651 rpcz_store.cc:276] 0709 13:02:23.619839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.748734 (+2128895us) service_pool.cc:224] Handling call
0709 13:02:25.749394 (+   660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.698614 (+949220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.698928 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:26.855232  1570 rpcz_store.cc:276] 0709 13:02:23.949782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.175654 (+225872us) service_pool.cc:224] Handling call
0709 13:02:24.183803 (+  8149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.338251 (+1154448us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4c00123980. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc83203aa20 00007fc83204ba29 00007fc832011644 00007fc832038f2d 00007fc832038b93 00007fc833d6918b 00007fc833d69cb5 00007fc833d64f30 00007fc833d667de 00007fc833d70b49 00007fc833d71416
0709 13:02:26.371226 (+1032975us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b58004e68d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833cca0aa 00007fc833d4c447 00007fc833d64fe7 00007fc833d667de 00007fc833d70b49 00007fc833d71416 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4
0709 13:02:26.854383 (+483157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.854709 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726619264}
W20260709 13:02:26.792553  1644 rpcz_store.cc:269] 0709 13:02:23.416655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.416914 (+   259us) service_pool.cc:224] Handling call
0709 13:02:26.650154 (+3233240us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b54000b9854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:26.686296 (+ 36142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6267412736}
W20260709 13:02:26.859637  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63356.4us
I20260709 13:02:26.816215  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:26.858676  1567 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:26.859647  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:26.831043  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:26.873521  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.873802  1575 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:26.864029  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2203) took 2913 ms. Trace:
W20260709 13:02:26.864277  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9380 after lost signal to thread 1676
W20260709 13:02:26.879381  2401 debug-util.cc:398] Leaking SignalData structure 0x7b080038e3e0 after lost signal to thread 1749
W20260709 13:02:26.879900  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08002cbd20 after lost signal to thread 1750
I20260709 13:02:26.868399  5199 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.870069  5198 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.884202  5230 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:26.884860  5230 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.885610  5127 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.886565  5127 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.856546  4898 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.888149  4898 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:26.888734  5127 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.889261  4875 raft_consensus.cc:993] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:26.888545  5039 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:26.885689  5230 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.890281  4875 raft_consensus.cc:1081] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
W20260709 13:02:26.892534  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 130716us
W20260709 13:02:26.892867  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94568.8us
I20260709 13:02:26.894533  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:26.878118  1572 rpcz_store.cc:276] 0709 13:02:23.951110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.341981 (+1390871us) service_pool.cc:224] Handling call
0709 13:02:25.342464 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.845429 (+1502965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.863698 (+ 18269us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54000271d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:26.863956 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2559507712}
I20260709 13:02:26.895557  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:26.898357  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:26.899659  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:26.901221  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:26.881863  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08003e96c0 after lost signal to thread 2404
W20260709 13:02:26.907516  2401 debug-util.cc:398] Leaking SignalData structure 0x7b080039d7e0 after lost signal to thread 2405
W20260709 13:02:26.906955  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.867439  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:26.912061  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45319.2us
I20260709 13:02:26.913950  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:26.915383  1650 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:26.844264  4996 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.843870  5156 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.924023  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:26.892555  5178 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:02:26.925037  1568 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:26.926559  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:26.845664  5203 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.919535  1644 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:26.920833  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:26.936650  1651 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:26.921725  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:26.938757  1647 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:26.855727  5170 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:26.870730  5212 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.946539  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:26.948134  1722 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:26.950640  2377 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:02:26.950999  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:26.951894  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:26.952672  1649 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:26.902873  4956 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:26.958140  1649 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:26.923795  5198 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.969620  1644 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:02:26.916404  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 85474.7us
I20260709 13:02:26.929610  1720 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:02:26.970901  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:26.971674  1722 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.870203  5233 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:26.974201  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:26.974891  1718 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.933382  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:26.976048  1642 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:26.880120  5228 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.934367  1573 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:26.940584  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:26.897410  5127 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:26.926725  1716 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:02:26.903291  5217 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:26.945772  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:26.915481  5216 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:26.945762  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.923280  5068 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:26.925693  5006 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:26.928372  5111 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:26.971982  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:02:27.034121  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62244.7us
I20260709 13:02:27.034907  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:26.902976  5065 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:02:26.981773  1721 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:26.902566  5194 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:02:26.941517  5170 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:26.983922  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 120454us
I20260709 13:02:26.984244  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:26.954466  5212 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.968037  5204 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.000383  1718 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:27.007381  1722 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:27.064057  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:27.064558  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.012899  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:26.962816  4956 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:27.029292  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 213205us
I20260709 13:02:27.040583  4994 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:27.066625  4994 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:27.066179  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:27.067031  4994 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.938555  5156 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.029474  5216 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:26.941247  5203 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.982936  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:27.054131  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:26.988202  5217 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.933848  4996 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:26.983449  5233 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.034688  5235 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.079291  5191 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:27.081125  2376 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:27.042146  5067 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:27.037480  5006 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.055872  5170 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.102128  5233 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.074750  5088 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.074326  4956 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.085085  5216 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.098519  5217 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.089363  5073 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.085367  5204 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.113346  5006 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.113940  5067 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:27.136967  5158 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.145766  5156 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.131753  5211 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:27.166711  5230 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.167181  5216 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.170142  5230 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:27.174739  5217 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.180492  5217 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.183593  5216 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.189086  5204 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.193413  5233 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.193627  5212 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.196065  5212 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.196065  5233 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:26.883036  1474 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 3 to 7, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196), now has a pending config: VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193), VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194), VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195), VOTER da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:02:27.235459  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.237365  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.238222  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.238480  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.927922  1568 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:27.242054  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.045245  1643 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
W20260709 13:02:27.254415  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.990860  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.255927  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.257347  1642 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:27.257699  1576 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:27.036059  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:27.000844  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:27.259925  1644 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:27.261950  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 225807us
W20260709 13:02:27.265904  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6841337900000326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
I20260709 13:02:27.266324  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1444) took 3671 ms. Trace:
W20260709 13:02:27.266618  1539 connection.cc:256] RPC call timeout handler was delayed by -1.204433789999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260709 13:02:27.267176  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1054337900000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
W20260709 13:02:27.267624  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0622337899999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:02:27.244144  1568 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:27.266985  1646 rpcz_store.cc:276] 0709 13:02:23.594458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.321599 (+727141us) service_pool.cc:224] Handling call
0709 13:02:24.321975 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.322351 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.322381 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.322915 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.758113 (+2435198us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:26.919342 (+161229us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.265388 (+346046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.265633 (+   245us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.265734 (+   101us) raft_consensus.cc:1685] finished
0709 13:02:27.265753 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.266044 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":321,"prepare.queue_time_us":140549,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":1790,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":12333,"thread_start_us":2919943,"threads_started":3,"wal-append.queue_time_us":1048580,"wal-append.run_cpu_time_us":3193,"wal-append.run_wall_time_us":1021568}
W20260709 13:02:27.274770  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.279932  1644 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:27.280916  1573 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:27.288760  1573 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:27.291793  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:27.302649  1573 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:27.296437  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:27.002782  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:27.328896  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.004288  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.003978  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:26.970965  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:27.078955  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:27.333397  1717 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:27.338372  1648 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:27.338665  1649 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:27.342680  2374 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:27.353328  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 419959us
I20260709 13:02:27.353312  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.355451  1721 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:27.131968  5067 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:27.358294  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.358578  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.361220  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.361054  4999 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:27.390763  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 287989us
I20260709 13:02:27.391268  4999 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.392884  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:27.395378  5233 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:27.395848  5156 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:27.396147  5158 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.443284  1647 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:27.397470  5158 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.447548  5073 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:27.396956  5156 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:27.450398  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.452044  4875 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:27.453172  5232 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:27.480975  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 189237us
I20260709 13:02:27.481801  5232 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:27.395576  4999 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:27.452438  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59823.1us
I20260709 13:02:27.448335  5198 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.486548  5198 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.484025  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:27.489724  5210 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:27.490028  5198 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.454149  4994 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:27.454988  5065 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:27.457219  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 378170us
I20260709 13:02:27.495061  4994 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.494175  5088 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:27.491565  4999 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.496883  5134 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:02:27.495131  4956 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.501154  5235 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:27.501698  5235 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:27.454538  5127 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.464054  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:27.503916  5127 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.504845  4994 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:27.464576  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.004s	sys 0.019s
I20260709 13:02:27.507505  5127 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:27.508083  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.005s	sys 0.019s
I20260709 13:02:27.461052  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:27.508683  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 567264us
W20260709 13:02:27.467774  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.470502  5158 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.516749  4875 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:27.449349  5230 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:27.449256  5199 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:02:27.518879  5230 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:27.520589  5230 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.525908  5199 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:27.470506  5203 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.533911  5232 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.469349  5156 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.497206  5211 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:02:27.534993  4994 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:27.536437  5191 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:27.538707  4994 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:27.542784  4994 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:27.503420  5235 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:27.449851  5204 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.528625  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:27.516587  5210 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:02:27.545415  5204 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.546005  1647 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:27.549840  5235 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.554543  4994 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.503311  4956 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.557657  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.558264  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.558629  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.559123  1721 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:27.559886  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.560624  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.564862  5204 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.566521  5251 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:27.569389  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:27.575908  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:27.576076  5204 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:27.577431  2374 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:27.578305  5204 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:27.581581  5204 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:27.589637  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.599582  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.606750  2376 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 12. Ops: [10.9-10.9]
I20260709 13:02:27.609485  5227 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:27.610229  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.608841  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:27.614013  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 72006.6us
I20260709 13:02:27.616884  1721 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:27.617220  1573 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:27.629902  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:27.637022  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.639778  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:27.640975  2377 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.648893  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.650206  1575 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:27.651770  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:27.652810  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.654464  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.659503  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:27.661602  5146 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.829s	user 0.007s	sys 0.017s
I20260709 13:02:27.662358  5146 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.663582  1722 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:27.663208  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:27.664969  1573 raft_consensus.cc:2376] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 9. Re-sending same reply.
W20260709 13:02:27.665393  1571 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.837s	user 0.019s	sys 0.007s
I20260709 13:02:27.665912  1571 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:27.666529  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:27.667460  1573 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:27.667537  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.668555  1571 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:27.670799  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.671113  2376 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:27.672881  4996 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.673794  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.691565  1568 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:27.693907  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.696020  1646 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:27.699326  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:27.707542  1644 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:27.721187  1571 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.720204  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.727684  1568 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:27.729116  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:27.729923  1575 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:27.722649  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:27.744961  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:27.745893  2380 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:02:27.748461  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.745234  5067 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:27.751495  5067 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:27.754529  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 471) took 3977 ms (client timeout 2976 ms). Trace:
W20260709 13:02:27.755278  1650 rpcz_store.cc:269] 0709 13:02:23.776891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.318490 (+541599us) service_pool.cc:224] Handling call
0709 13:02:27.753817 (+3435327us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b540005c3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:27.754112 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5939375488}
W20260709 13:02:27.756661  5170 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.637s	user 0.021s	sys 0.001s
I20260709 13:02:27.757467  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 429) took 2220 ms. Trace:
W20260709 13:02:27.757763  4742 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.893s	user 0.022s	sys 0.003s
I20260709 13:02:27.758212  1720 rpcz_store.cc:276] 0709 13:02:25.536757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.000673 (+463916us) service_pool.cc:224] Handling call
0709 13:02:27.756987 (+1756314us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54001a6dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:27.757232 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903619456}
I20260709 13:02:27.761453  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.763398  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.771348  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.772279  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.772511  1722 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:27.775144  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.775471  5175 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:27.776795  1721 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:27.778373  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.779284  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:27.780566  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.784281  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:27.784587  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2207) took 3818 ms (client timeout 2871 ms). Trace:
I20260709 13:02:27.786346  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.791723  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:27.800526  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.813316  1643 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.806993  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.814476  1573 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 5.
I20260709 13:02:27.816623  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.822010  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:27.810866  1569 rpcz_store.cc:269] 0709 13:02:23.966170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.076673 (+110503us) service_pool.cc:224] Handling call
0709 13:02:27.783989 (+3707316us) spinlock_profiling.cc:243] Waited 3.68 s on lock 0x7b54000cba54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:27.784258 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8454089088}
I20260709 13:02:27.825740  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.826826  1724 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.833684  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.834014  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:27.838605  1573 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:27.809571  5175 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:27.850248  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.859486  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75422.9us
I20260709 13:02:27.862844  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.757472  5170 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.865337  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.865255  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.868232  2377 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:27.865743  1568 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:27.865228  1718 raft_consensus.cc:2376] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8. Re-sending same reply.
I20260709 13:02:27.876580  1569 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:27.871531  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:27.880021  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.872244  2372 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:27.883733  1614 connection.cc:256] RPC call timeout handler was delayed by -3.0950225189999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3165s for a response
I20260709 13:02:27.886091  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.887429  1644 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:27.889371  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:27.893769  5257 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:27.893970  1722 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:27.894220  5257 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.894728  5257 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:27.894941  5006 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.895789  1570 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:27.896466  1720 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:27.897271  5006 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:27.898255  5175 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:27.899504  5175 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.901592  5059 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:27.905211  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.907248  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.908641  4996 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:27.909091  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:27.911623  5260 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.913004  5260 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.913771  5260 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.914373  1721 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:27.915578  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.889617  1723 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:27.913920  1569 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.919860  1721 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:27.926366  1570 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:27.918314  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2053) took 1007 ms. Trace:
I20260709 13:02:27.909252  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.927352  1717 rpcz_store.cc:276] 0709 13:02:26.910773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.910913 (+   140us) service_pool.cc:224] Handling call
0709 13:02:26.942151 (+ 31238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.338749 (+396598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.874598 (+535849us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54003dd1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:27.918252 (+ 43654us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896395136}
I20260709 13:02:27.936218  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.937871  5068 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:27.938172  1718 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:27.938474  5068 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.939069  5068 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:27.939373  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.936865  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.940862  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:27.942296  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.942667  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.940743  1568 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:27.943531  1719 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:27.945578  2380 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:27.937258  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.947319  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1435) took 4363 ms. Trace:
I20260709 13:02:27.935544  5039 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:02:27.880288  1573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:02:27.947747  1645 rpcz_store.cc:276] 0709 13:02:23.583354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.583518 (+   164us) service_pool.cc:224] Handling call
0709 13:02:26.650305 (+3066787us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:26.651041 (+   736us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:26.757430 (+106389us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.941948 (+184518us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:26.942659 (+   711us) log.cc:844] Serialized 78 byte log entry
0709 13:02:26.942811 (+   152us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:26.943014 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.954326 (+ 11312us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.974341 (+ 20015us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.259080 (+284739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.259494 (+   414us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.259621 (+   127us) raft_consensus.cc:1685] finished
0709 13:02:27.259654 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.766360 (+506706us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b1800273460. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:27.890737 (+124377us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b1800273460. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:27.946923 (+ 56186us) spinlock_profiling.cc:243] Waited 3.07 s on lock 0x7b54000b9850. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:27.947073 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58946,"prepare.queue_time_us":1813327,"prepare.run_cpu_time_us":175,"prepare.run_wall_time_us":181,"raft.queue_time_us":45812,"raft.run_cpu_time_us":710,"raft.run_wall_time_us":27814,"spinlock_wait_cycles":7089895424,"thread_start_us":549095,"threads_started":3,"wal-append.queue_time_us":332180}
I20260709 13:02:27.950067  1573 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:27.950587  1573 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:27.952963  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:27.869742  5227 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.933739  1643 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:27.956993  1642 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:27.958215  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:27.965986  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.962879  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 98323us
I20260709 13:02:27.973059  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:27.977313  5232 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.978262  5232 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:27.980894  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.982564  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:27.984218  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.984855  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:27.985745  2375 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:27.986083  5067 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:27.987088  5233 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:27.987540  5006 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.988581  5006 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:27.972891  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.990136  2380 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:27.991951  1718 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:27.989197  5006 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.990528  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:27.926657  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:27.972224  5156 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.986692  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:27.999466  1571 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:27.986546  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 463) took 1020 ms. Trace:
I20260709 13:02:27.984148  4742 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.008268  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.011348  1725 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:27.972041  5059 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:28.007301  4742 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:28.015247  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:28.019918  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51395.3us
I20260709 13:02:28.022007  1647 raft_consensus.cc:2410] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:28.017211  4742 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.015472  5059 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.036671  5158 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.037279  5158 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.037657  5158 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.041074  1572 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.010010  5156 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.004786  2375 rpcz_store.cc:276] 0709 13:02:26.966356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.968285 (+  1929us) service_pool.cc:224] Handling call
0709 13:02:27.983832 (+1015547us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:02:27.986449 (+  2617us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92598144}
I20260709 13:02:28.044973  5156 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.046113  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:28.047092  2371 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:28.048092  5059 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:28.027935  5266 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:28.049937  5266 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:28.044147  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:28.057328  1616 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:28.062680  2372 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:28.064910  2379 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.065770  5233 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:28.065954  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:28.067330  2380 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:28.035578  2376 raft_consensus.cc:2410] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:02:28.072834  4875 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.046145  5170 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.074059  4875 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:28.074993  5269 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:02:28.079023  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2233) took 1326 ms. Trace:
I20260709 13:02:28.079521  1576 rpcz_store.cc:276] 0709 13:02:26.752426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.835770 (+ 83344us) service_pool.cc:224] Handling call
0709 13:02:28.078717 (+1242947us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54001252d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:28.078926 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902981504}
I20260709 13:02:28.083616  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.087570  4996 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:28.088096  4996 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.088377   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:28.088726  4996 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:28.090257  5156 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:28.096792  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 88033.2us
I20260709 13:02:28.098084  5232 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:28.084131  2377 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:28.099713  4875 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:28.084708  5233 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:28.086180  5065 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:28.103169  1651 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:28.100939  5233 consensus_queue.cc:1243] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:28.117158  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.111353  1573 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.121691  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 459) took 1166 ms. Trace:
I20260709 13:02:28.126555  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:28.122208  2372 rpcz_store.cc:276] 0709 13:02:26.948286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.964630 (+ 16344us) service_pool.cc:224] Handling call
0709 13:02:27.778834 (+814204us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:02:28.114283 (+335449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12687744}
I20260709 13:02:28.126554  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.135919  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.152395  5170 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:28.152782  5216 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:02:28.153512  5170 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.159376  1723 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:28.162722  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51629.8us
I20260709 13:02:28.164917  1573 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:28.170907  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 44461.5us
I20260709 13:02:28.175175  1725 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:28.180095  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53856.5us
I20260709 13:02:28.051906  5266 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.185860  5233 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:28.198457  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.199611  1569 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:28.186378  5168 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:28.186674  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.188951  4875 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:28.201992  4875 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.189448  5232 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:28.203693  5232 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:28.204164  5232 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.204414  5251 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:28.192641  5272 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.092s	user 0.000s	sys 0.003s
I20260709 13:02:28.205148  5251 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.182723  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.201092  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.196717  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:28.225852  1574 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:28.219442  1576 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:28.234119  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.231277  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.241393  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.242218  1571 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:28.230844  5067 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:28.119432  5257 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:28.251960  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 71104.1us
I20260709 13:02:28.253269  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:28.254002  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.254055  5168 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.255520  1642 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:28.302736  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.256471  1647 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.258225  5073 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.259155  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:28.259981  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78564.3us
I20260709 13:02:28.260527  5233 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:28.262167  5068 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.263718  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.264175  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.264650  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c55c47030a294c17ad5561abe541b993"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:02:28.265698  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.269116  1571 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:28.274005  4996 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.275508  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.275807  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 1986) took 1325 ms. Trace:
I20260709 13:02:28.280900  5158 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.282644  5251 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:28.284596  5170 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.320904  1574 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:28.234496  4994 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.299120  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:28.273849  5006 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.310853  1651 rpcz_store.cc:276] 0709 13:02:26.950109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.970724 (+ 20615us) service_pool.cc:224] Handling call
0709 13:02:28.247588 (+1276864us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400133654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:28.275685 (+ 28097us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036141952}
I20260709 13:02:28.316030  1573 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:28.318604  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:28.321732  1569 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.323915  1569 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.288779  5067 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:28.311275  1724 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:28.329342  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54031.3us
I20260709 13:02:28.280062  5260 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.341506  1576 raft_consensus.cc:2368] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
W20260709 13:02:28.344717  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 45774us
I20260709 13:02:28.323488  4875 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:28.358701  1575 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:28.332356  5233 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:28.320580  5073 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.335605  5156 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.366833  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.369062  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:28.370078  2372 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:28.359544  5175 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:28.379058  5232 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.390422  5006 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.392446  5158 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.398257  5278 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.457374  1572 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:28.457922  5278 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.388109  4742 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.460678  1572 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.462162  4742 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.462496  1572 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:28.379060  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:28.345316  5251 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:02:28.463841  5251 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:28.464119  5278 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:28.393980  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80874.5us
I20260709 13:02:28.465803  5232 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.466967  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.381188  5229 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:02:28.469477  5229 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:28.470085  5229 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:28.404633  5067 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.474215  5067 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.477290  5006 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:28.440066  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:28.404915  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:28.481958  5235 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:28.482039  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77281.1us
I20260709 13:02:28.482633  5235 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.358453  5170 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.483139  5225 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:28.483170  5235 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:28.394627  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.487345  5225 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:28.488876  5278 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:28.489194  5225 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.400534  2377 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:02:28.498338  2377 pending_rounds.cc:85] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 8
I20260709 13:02:28.499007  2377 pending_rounds.cc:107] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:02:28.499377  2377 raft_consensus.cc:2889] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.412519  1725 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:28.502467  1723 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.504752  5170 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.505859  5170 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.413411  5068 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.509999  1569 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.425504  5059 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.426553  5175 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:28.370633  5156 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:28.428519  1574 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.429620  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:28.431870  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:28.397931  4996 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.399577  5257 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:28.392992  1567 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:28.395262  5233 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.406548  5260 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.438266  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:28.441754  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.443611  1573 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.327804  5168 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.464388  5251 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:28.388929  5204 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:28.510198  5079 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:28.415450  5073 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.513981  4875 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.516076  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.517693  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.518086  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.614110  1724 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:28.519045  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.518888  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.519586  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.471707  5229 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.526503  5278 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.528676  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61865.7us
I20260709 13:02:28.537034  5006 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:28.589087  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 157576us
W20260709 13:02:28.589721  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 151627us
W20260709 13:02:28.622051  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97197us
I20260709 13:02:28.589128  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.591140  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.591037  4742 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.593989  5059 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.574542  5156 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.594725  1569 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:28.546499  5230 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:02:28.589880  5232 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:28.629434  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.630302  1575 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:28.626065  5235 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.632234  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:28.648590  4875 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.650993  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.651481  4875 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:28.652182  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.652743  1569 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:28.652890  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:28.633541  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55159.4us
I20260709 13:02:28.654065  5068 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.654562  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.637111  1723 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:02:28.654999  5068 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:28.655575  1718 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:28.657143  5251 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.658625  5251 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.660593  5211 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.661314  5211 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.661789  5211 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.666622  5073 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:28.629761  4742 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.673046  4742 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.676878  5229 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.680403  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.682039  1718 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:28.630407  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:28.642230  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.595254  5175 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.644181  5278 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.645180  5230 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:28.649092  5235 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:28.632867  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.697239  1716 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:28.651254  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:28.685397  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.633481  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 204045us
I20260709 13:02:28.688902  1721 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:28.684680  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.691648  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.701982  1575 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:28.705580  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75316.6us
I20260709 13:02:28.705716  5158 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.708413  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:28.634315  5260 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.714396  1569 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:28.717960  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.638494  5170 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.724835  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.721024  1572 pending_rounds.cc:85] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:02:28.725690  1572 pending_rounds.cc:107] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:28.726030  1572 raft_consensus.cc:2889] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:28.723189  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44658us
I20260709 13:02:28.683944  5204 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:28.727638  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:28.693498  5230 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:28.736963  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48906.4us
I20260709 13:02:28.733222  5260 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:28.696955  5006 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:28.723769  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.759s (SENT)
I20260709 13:02:28.742941  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:28.741231  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.746346  1718 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:28.749167  1569 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:28.753082  1717 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:28.750540  5170 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:28.763533  5170 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:28.764674  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 109479us
I20260709 13:02:28.759402  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.764966  1642 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.752012  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:28.771675  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:28.772675  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.770005  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:28.768126  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.778522  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.780197  1724 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:28.782395  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:28.790169  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.790874  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.791479  1569 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:28.785742  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:28.793404  1651 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:28.797840  1647 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:28.798756  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.764153  5170 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.757632  5227 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:28.771857  5289 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:28.771979  5232 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:28.771946  5158 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:28.803428  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:28.805794  5278 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:28.745831  5059 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:02:28.785804  5257 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.815672  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.816073  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:28.808131  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.833660  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:28.809449  5235 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:28.843230  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:28.789158  5233 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:02:28.845494  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:28.781301  5287 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:02:28.807016  5227 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:28.809746  4875 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.858194  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:28.859395  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 135732us
W20260709 13:02:28.854401  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72081.5us
I20260709 13:02:28.864699  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:28.865475  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:28.866050  1651 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:28.869125  4990 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:28.870301  4990 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:28.873456  2372 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:28.875423  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 105515us
I20260709 13:02:28.877127  5204 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:28.839136  5158 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:28.863116  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:28.845202  5290 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.883206  1571 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:28.884395  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:28.844013  5068 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.846788  5257 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.857121  5235 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.891304  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:28.888011  5158 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.923904  1642 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:28.886910  5290 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.924898  1571 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:28.932277  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.889995  5068 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.895004  5257 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.895426  5235 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:28.935847  5068 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.888458  5006 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:28.937994  5285 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:02:28.934840  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72946.5us
W20260709 13:02:28.950069  5294 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:28.858310  5278 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:02:28.900877  5168 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:28.984467  5260 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:28.941460  5290 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.864389  5230 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:29.013887  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 148759us
I20260709 13:02:28.872107  4990 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:29.014954  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 166824us
W20260709 13:02:28.987442  5288 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.172s	user 0.001s	sys 0.001s
I20260709 13:02:29.027056  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.031133  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:29.040743  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52838.7us
I20260709 13:02:29.042066  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:29.061633  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.069597  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.071455  1642 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:29.035236  5260 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:29.117106  1690 leader_election.cc:400] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:29.117694  1690 leader_election.cc:403] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:02:29.115078  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 221631us
I20260709 13:02:29.066207  5168 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:29.126022  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.126998  1571 raft_consensus.cc:2368] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6.
I20260709 13:02:29.111157  1463 maintenance_manager.cc:419] P d61e0c3f41244926bf144dca0f9acf61: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033377
I20260709 13:02:29.128839  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.140034  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:29.147215  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 108505us
I20260709 13:02:29.124155  5230 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.149080  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.159516  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:29.159634  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.166975  1573 raft_consensus.cc:2376] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 10. Re-sending same reply.
I20260709 13:02:29.168666  1647 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:29.129778  5260 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:29.183032  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.128729  5168 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:29.190402  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:29.196503  1573 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:29.192051  1718 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:29.206704  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.208205  1568 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:29.226804  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.227085  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:29.229651  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64013.1us
I20260709 13:02:29.231271  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.232551  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.235468  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:29.228816  5278 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.240289  5278 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:29.240809  5301 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:29.241461  5301 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:29.241912  5301 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.241598  5278 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.242130  5308 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.243538  5308 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.245024  1647 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:29.246886  2379 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:29.248066  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:29.248790  2374 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.200502  5168 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.252478  1643 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:29.252959  2377 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:29.253592  5297 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:02:29.253856  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.265690  5309 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.273296  1645 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:02:29.276945  5310 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:29.279624  5211 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.010s
W20260709 13:02:29.281014  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49916.8us
I20260709 13:02:29.283918  5073 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.284834  5073 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.287945  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:29.219478  5225 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:29.291453  2374 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:29.295796  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:29.297156  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:29.301578  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:29.298565  2373 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:29.301578  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:29.302438  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:29.302831  2377 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.299925  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:29.300441  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 176774us
W20260709 13:02:29.286118  5229 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.003s
W20260709 13:02:29.286898  4742 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.594s	user 0.014s	sys 0.000s
I20260709 13:02:29.302338  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:29.328665  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.333197  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:29.340943  1716 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:29.333683  1723 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:29.333899  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.334687  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.335426  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:29.301612  5067 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.827s	user 0.017s	sys 0.002s
W20260709 13:02:29.336597  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 134848us
W20260709 13:02:29.303263  5295 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
I20260709 13:02:29.355010  1647 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.344882  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.341325  1716 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:29.349517  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.358072  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.285581  5073 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.339968  5059 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:29.280207  5211 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:29.381260  5170 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.006s
I20260709 13:02:29.382506  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.343763  5300 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.390697  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.284794  5260 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.392172  1570 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:02:29.397593  1649 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:29.421530  1721 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:29.324627  5225 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.426067  1643 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:29.422921  1644 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:29.428144  1723 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.431367  1723 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:29.427098  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278475us
I20260709 13:02:29.417877  1716 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.433738  1723 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.435300  1723 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.438457  5225 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:29.439857  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.350731  4742 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.434595  1716 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.422067  5059 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:29.439730  5156 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.001s
I20260709 13:02:29.455185  2377 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:29.466714  1642 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:29.466522  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76009.8us
W20260709 13:02:29.472662  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 130342us
I20260709 13:02:29.475044  1723 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:29.452525  5225 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:29.417860  5170 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.442126  5300 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.486806  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.487983  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.430171  5067 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:02:29.492995  1649 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:29.480942  5251 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.818s	user 0.017s	sys 0.000s
W20260709 13:02:29.497598  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 56913.9us
I20260709 13:02:29.515132  1651 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:29.482421  5312 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:29.458084  5068 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.465972  5156 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.530368  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.533325  1649 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:29.537653  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.446149  5227 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.522078  5312 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:29.554869  1569 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:29.559585  5326 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.562762  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.560143  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.552639  5204 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.591032  1649 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:02:29.592325  1643 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:29.595275  4742 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:29.596310  5211 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:29.598207  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:29.600016  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.600912  1645 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:29.601868  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.598583  4907 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.604115  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.605165  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.603299  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.591563  1649 pending_rounds.cc:85] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 5
I20260709 13:02:29.612594  1649 pending_rounds.cc:107] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:02:29.543757  5287 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:02:29.612982  1649 raft_consensus.cc:2889] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:29.616153  1573 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:29.562849  5326 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:29.620947  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53340.7us
I20260709 13:02:29.580910  5227 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.615058  5225 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.620131  1575 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:02:29.666213  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 335173us
I20260709 13:02:29.671813  1644 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:29.661935  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:29.674237  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:29.590603  5308 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:29.598327  5156 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:29.685232  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.694139  1645 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:29.554947  5059 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.671022  5204 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.693229  5073 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:29.705053  5073 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.705184  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:29.704835  5312 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.706053  1723 raft_consensus.cc:606] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:29.718125  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 164527us
I20260709 13:02:29.718182  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.719660  2344 leader_election.cc:400] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6.
I20260709 13:02:29.720041  2344 leader_election.cc:403] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:29.723120  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.692812  5065 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:29.694744  5316 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.728358  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.705890  5073 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.744849  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.741618  5175 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:29.743103  5331 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.721005  5300 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:29.758116  5323 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.744625  5225 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 5. Current term is 6.
I20260709 13:02:29.709565  5314 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:02:29.768843  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.737686  5312 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:29.774550  5305 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
I20260709 13:02:29.758952  5323 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.715535  5204 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:29.778232  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:29.778239  1716 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:29.782064  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:29.783658  1723 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:29.787392  1647 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:29.791987  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:29.793048  1721 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:29.796762  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:29.756156  5170 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:29.797842  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:29.765105  5068 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:29.734824  5065 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:29.809559  5321 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:02:29.810111  5204 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:29.761682  5006 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:29.812668  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.820814  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2258) took 1009 ms. Trace:
I20260709 13:02:29.822045  5323 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:29.761451  5331 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.825872  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
W20260709 13:02:29.826283  5257 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.838s	user 0.014s	sys 0.000s
I20260709 13:02:29.826735  5257 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.828107  1571 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:29.797772  1723 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:29.781440  5235 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.770s	user 0.015s	sys 0.000s
I20260709 13:02:29.803114  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.751436  5073 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:29.803298  5329 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:29.776147  4907 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:29.839862  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.840525  5227 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:29.821350  1567 rpcz_store.cc:276] 0709 13:02:28.811218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.904324 (+ 93106us) service_pool.cc:224] Handling call
0709 13:02:28.943152 (+ 38828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.231140 (+287988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.231179 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.250096 (+ 18917us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.479266 (+229170us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:29.479770 (+   504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.479897 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.820473 (+340576us) raft_consensus.cc:1685] finished
0709 13:02:29.820500 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.820726 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182616,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":20570,"thread_start_us":384471,"threads_started":2,"wal-append.queue_time_us":234079}
W20260709 13:02:29.836706  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 44855.9us
W20260709 13:02:29.869946  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79661.2us
I20260709 13:02:29.854775  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:29.902302  1644 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:02:29.902443  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99472.4us
I20260709 13:02:29.903823  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.905861  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2249) took 1199 ms. Trace:
I20260709 13:02:29.906235  1644 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:29.848191  5006 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:29.843043  5068 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.848883  5251 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.915835  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:29.917084  1649 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:29.917637  1644 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [6.5-8.6]
W20260709 13:02:29.909734  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 55045.2us
I20260709 13:02:29.906558  1574 rpcz_store.cc:276] 0709 13:02:28.697775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.705343 (+  7568us) service_pool.cc:224] Handling call
0709 13:02:28.705730 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.705943 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.705964 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.706812 (+   848us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.110556 (+403744us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:29.110603 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.110679 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.896457 (+785778us) raft_consensus.cc:1685] finished
0709 13:02:29.896479 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.896631 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":403282,"threads_started":1,"wal-append.queue_time_us":634912}
W20260709 13:02:29.872720  5309 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.005s
I20260709 13:02:29.915418  1646 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:29.841831  5235 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:29.921502  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 105284us
I20260709 13:02:29.900665  4742 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.900098  5227 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:02:29.949550  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50064.1us
I20260709 13:02:29.952095  1723 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:29.896685  5314 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:02:29.958276  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:29.960618  2377 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:29.961647  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:29.962652  1649 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.915767  5175 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.965147  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:29.969940  1644 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:29.944708  5309 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:29.973637  1721 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:02:29.972519  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:29.978478  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 233726us
I20260709 13:02:29.981995  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:29.982856  1644 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:29.981392  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.984424  1720 raft_consensus.cc:606] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:29.919190  5068 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.002014  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:29.962291  4742 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:29.968407  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:29.993173  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:30.004870  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.015s	sys 0.013s
W20260709 13:02:30.013754  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.015s	sys 0.013s
W20260709 13:02:30.014269  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 518487us
I20260709 13:02:30.006764  1720 raft_consensus.cc:614] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:30.006650  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:29.973513  5326 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:30.000065  5309 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.023774  1725 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
W20260709 13:02:30.025878  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53708.8us
I20260709 13:02:30.025279  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:30.031064  1644 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:30.033016  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:29.991819  5251 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:02:30.008283  5330 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:30.043648  5346 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:30.046943  5168 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.778s	user 0.015s	sys 0.001s
W20260709 13:02:30.049113  5336 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:02:30.002388  5260 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:02:30.060317  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:30.032965  5319 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:30.082172  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 113920us
I20260709 13:02:30.033933  5257 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:30.035631  5326 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.082650  5319 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:30.097509  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90973.9us
I20260709 13:02:30.098747  5319 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.099128  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:30.036808  5314 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 13:02:30.103569  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77519.2us
I20260709 13:02:30.107617  1646 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:30.107165  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:30.072715  5156 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:30.041337  5308 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:02:30.025163  5227 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.100078  5319 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:30.090778  5170 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.081717  5251 consensus_queue.cc:1243] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:02:30.138127  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56251.5us
I20260709 13:02:30.072715  5168 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:30.146085  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2257) took 1354 ms (client timeout 1792 ms). Trace:
W20260709 13:02:30.146916  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.076241  5325 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.181s
I20260709 13:02:30.150787  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.113343  5073 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:30.161906  1569 rpcz_store.cc:269] 0709 13:02:28.791473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.793386 (+  1913us) service_pool.cc:224] Handling call
0709 13:02:30.145782 (+1352396us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54001257d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:30.146008 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634443648}
W20260709 13:02:30.175226  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 115069us
W20260709 13:02:30.179378  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.146384  5346 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:30.182603  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:30.185984  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:30.173671  5158 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.186s	user 0.008s	sys 0.010s
W20260709 13:02:30.187250  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88208us
I20260709 13:02:30.189795  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:30.179636  5059 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.186120  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.113405  5339 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.208s
I20260709 13:02:30.203040  5068 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:30.203785  5290 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.172s	user 0.015s	sys 0.003s
I20260709 13:02:30.204298  5290 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:30.205156  5335 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.277s	user 0.000s	sys 0.002s
W20260709 13:02:30.206849  5301 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.964s	user 0.008s	sys 0.010s
I20260709 13:02:30.207075  5290 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:30.207446  5301 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.183990  5235 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:30.208741  1718 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:30.209336  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77342.6us
W20260709 13:02:30.209774  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.210431  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.180318  5310 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:30.137296  5330 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:02:30.211853  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 469) took 1031 ms. Trace:
I20260709 13:02:30.184854  5073 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.228307  2380 rpcz_store.cc:276] 0709 13:02:29.180570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.180723 (+   153us) service_pool.cc:224] Handling call
0709 13:02:29.224047 (+ 43324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.340730 (+116683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.344695 (+  3965us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.375241 (+ 30546us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.686136 (+310895us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:29.686171 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.686255 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.211484 (+525229us) raft_consensus.cc:1685] finished
0709 13:02:30.211522 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.211755 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28420,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":92,"thread_start_us":399320,"threads_started":2,"wal-append.queue_time_us":323970}
I20260709 13:02:30.189224  5170 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:30.188552  5257 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:30.184561  5346 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:30.234980  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45287.5us
I20260709 13:02:30.240214  1723 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.225775  5308 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.204402  5341 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:30.233623  4875 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 6 is higher.
I20260709 13:02:30.203433  5347 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:30.193567  5314 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:02:30.197986  5251 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:02:30.237466  5170 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.239406  5260 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.196942  5158 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.271252  5006 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:30.199957  5059 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.277149  1643 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:02:30.282510  1643 pending_rounds.cc:85] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 8
I20260709 13:02:30.282814  1643 pending_rounds.cc:107] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:30.283124  1643 raft_consensus.cc:2889] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:30.280942  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.284153  5156 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:30.285717  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.286908  4990 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.287534  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.253970  5341 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.288636  5341 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:30.297050  4990 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.253384  5326 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:02:30.300733  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59843.2us
I20260709 13:02:30.301957  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.248443  5348 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:30.266652  5170 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:30.309085  5338 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:02:30.306576  5348 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:30.318212  5301 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:30.203984  5342 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:02:30.329407  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2084) took 1136 ms. Trace:
I20260709 13:02:30.196652  5349 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.243s
I20260709 13:02:30.334492  1649 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:30.330392  1718 rpcz_store.cc:276] 0709 13:02:29.193286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.226572 (+ 33286us) service_pool.cc:224] Handling call
0709 13:02:30.326012 (+1099440us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54003aaf54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:30.329302 (+  3290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254465920}
I20260709 13:02:30.332115  4990 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:30.351627  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 132898us
W20260709 13:02:30.379020  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 75215.5us
W20260709 13:02:30.354236  5352 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:02:30.380086  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.384902  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.386962  5079 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:02:30.389739  5318 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.956s	user 0.004s	sys 0.002s
W20260709 13:02:30.391892  5230 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.208s	user 0.013s	sys 0.004s
I20260709 13:02:30.334195  5257 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:02:30.394507  5260 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:30.395684  5331 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.571s	user 0.014s	sys 0.001s
I20260709 13:02:30.398554  5168 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:30.398844  5360 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:30.399510  5158 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:30.400096  5356 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:02:30.400378  4990 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.400866  4990 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.401221  4990 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:30.401616  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:30.402287  2380 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:30.406538  5278 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:30.413756  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110893us
W20260709 13:02:30.417855  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 219864us
W20260709 13:02:30.431854  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.433441  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:30.435725  1574 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 6. Ops: []
I20260709 13:02:30.439245  1571 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 8. Ops: []
W20260709 13:02:30.462647  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.465688  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:30.466524  1569 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:30.465713  1567 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:30.473165  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 88366.5us
I20260709 13:02:30.475199  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:30.475955  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.476532  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:30.474720  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:30.433691  5331 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.481367  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:30.486382  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:30.487255  1716 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:30.493108  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:30.494026  1718 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:30.500319  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:30.501384  1573 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:30.501744  1575 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:30.502988  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:30.503774  1568 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:30.511824  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:30.514151  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:30.514564  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:30.515336  1568 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.515558  1573 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:30.517442  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 85774.7us
I20260709 13:02:30.518388  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:30.522699  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:30.521173  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:30.524139  1716 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:30.337932  5346 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.521179  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:30.499532  5339 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:30.543053  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:30.544785  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:30.547842  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:30.572810  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50290.7us
I20260709 13:02:30.537395  5232 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.413453  5348 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.499096  5326 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:02:30.577139  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:30.575615  1721 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:30.569785  5233 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:02:30.591334  5170 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.592810  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:30.593717  1574 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:30.596264  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46583.6us
I20260709 13:02:30.589270  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:30.600198  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:30.575520  5339 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:30.582566  5257 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:30.606411  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88136.9us
I20260709 13:02:30.601203  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:30.586057  5175 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:30.610605  5170 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:30.611756  5369 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.604173  5233 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.612804  5233 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.612502  5369 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.613647  5233 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:30.614233  5350 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.415s	user 0.000s	sys 0.002s
I20260709 13:02:30.614688  5367 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.595575  5232 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:30.615761  5367 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.616705  5232 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:30.601883  5343 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 13:02:30.617201  5232 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:30.618659  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.968s (SENT)
I20260709 13:02:30.619222  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:30.623354  2380 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:30.624815  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:30.601531  5371 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:30.628087  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1628) took 2258 ms. Trace:
I20260709 13:02:30.630316  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:30.632179  1721 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:30.640453  1721 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:30.640897  1721 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:30.640807  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:30.641880  1721 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.644521  1721 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.599078  5059 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:30.633397  1570 rpcz_store.cc:276] 0709 13:02:28.361331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.421104 (+ 59773us) service_pool.cc:224] Handling call
0709 13:02:28.421614 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.436454 (+1014840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.619821 (+1183367us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54002a76d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:30.620066 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224593664}
I20260709 13:02:30.652494  1573 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:30.662379  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1486) took 1064 ms. Trace:
I20260709 13:02:30.662894  1645 rpcz_store.cc:276] 0709 13:02:29.598117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.601574 (+  3457us) service_pool.cc:224] Handling call
0709 13:02:30.662142 (+1060568us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b54001bbcd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:30.662304 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210660608}
I20260709 13:02:30.619887  5175 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.631220  5170 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:30.638152  5251 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:30.682497  1643 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:30.692791  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74188.1us
W20260709 13:02:30.696710  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.026s (SENT)
I20260709 13:02:30.696589  1568 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:30.680850  5079 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 13:02:30.705649  5364 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:02:30.667591  5059 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.704876  5354 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:30.704952  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:30.705505  5373 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:02:30.689496  5357 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:02:30.709741  5251 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:30.688705  5317 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:02:30.722743  5251 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.673005  5175 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:30.725796  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 376) took 1571 ms. Trace:
W20260709 13:02:30.727977  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 87266.9us
I20260709 13:02:30.705534  5366 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.712586  5379 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:30.708534  5059 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.710916  1721 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:02:30.739418  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 112234us
W20260709 13:02:30.742959  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 112770us
W20260709 13:02:30.726397  5312 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.945s	user 0.018s	sys 0.000s
I20260709 13:02:30.727700  5384 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:30.729242  1576 rpcz_store.cc:276] 0709 13:02:29.154552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.162851 (+  8299us) service_pool.cc:224] Handling call
0709 13:02:29.194871 (+ 32020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.195266 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.195301 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.229215 (+ 33914us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.454706 (+225491us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:29.478753 (+ 24047us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.480524 (+  1771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.480643 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.811195 (+330552us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b18003b4880. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:30.350810 (+539615us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b18003b4880. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:30.725511 (+374701us) raft_consensus.cc:1685] finished
0709 13:02:30.725545 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.725727 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23723,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":1524,"raft.run_cpu_time_us":717,"raft.run_wall_time_us":5374,"spinlock_wait_cycles":20122624,"thread_start_us":211691,"threads_started":1,"wal-append.queue_time_us":267137}
I20260709 13:02:30.749217  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:30.732148  5316 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.727066  5257 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:30.710206  5170 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:30.710011  5354 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:30.742529  4742 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:30.820583  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:30.825775  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:30.826980  1724 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:30.841588  1722 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:30.673440  5371 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.745071  5227 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:02:30.782226  5379 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:30.792577  5232 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.795657  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:30.755194  4907 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.781155  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:30.809051  5372 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:30.810328  2377 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:30.810696  5347 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:30.811429  5370 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:02:30.798530  5170 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.813934  5204 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.814356  5354 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.815948  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2088) took 1526 ms. Trace:
I20260709 13:02:30.689965  5339 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:30.790902  5361 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:30.852985  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:30.782465  5059 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:30.896566  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.807816  1718 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:02:30.948068  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54445.6us
W20260709 13:02:31.003211  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:30.950525  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87513.1us
I20260709 13:02:30.979588  1725 rpcz_store.cc:276] 0709 13:02:29.293297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.294415 (+  1118us) service_pool.cc:224] Handling call
0709 13:02:30.815689 (+1521274us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54003dccd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:30.815867 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493805440}
I20260709 13:02:30.893546  4742 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.993026  2374 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:02:31.018303  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 269274us
I20260709 13:02:30.967453  5170 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:30.969333  5225 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.948788  5347 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:31.088115  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 82659.2us
I20260709 13:02:31.089720  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:31.091532  5225 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.092047  5385 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:02:31.093920  5385 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:31.095089  5225 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.095808  5385 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:30.953053  4907 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:30.991155  5372 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:31.001008  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:31.122049  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.016s	sys 0.049s
W20260709 13:02:31.124642  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 343604us
W20260709 13:02:31.147378  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.017s	sys 0.049s
W20260709 13:02:31.162612  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.191953  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.200367  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2181) took 1468 ms. Trace:
I20260709 13:02:31.201313  1719 rpcz_store.cc:276] 0709 13:02:29.723505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.828123 (+104618us) service_pool.cc:224] Handling call
0709 13:02:29.828605 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.976576 (+147971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.976616 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.028384 (+ 51768us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.507940 (+479556us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:30.507987 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:30.508094 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.191120 (+683026us) raft_consensus.cc:1685] finished
0709 13:02:31.191146 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.191332 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9631,"raft.queue_time_us":121527,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":6362,"thread_start_us":563490,"threads_started":2,"wal-append.queue_time_us":571562}
I20260709 13:02:31.122303  5379 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.069837  5361 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.081620  4742 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.123953  5390 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:31.096318  5385 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.212826  5385 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:31.219918  5170 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.223755  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:31.234270  5232 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.236789  5347 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.238241  5347 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:31.239565  1724 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:31.239995  1724 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:31.242254  5369 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:31.250530  5372 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:31.253772  5341 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.964s	user 0.013s	sys 0.005s
I20260709 13:02:31.254359  5341 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.193307  5354 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:31.256698  2374 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:31.259930  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 68855.2us
I20260709 13:02:31.264243  5394 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:31.267004  5312 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:31.269001  5312 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:31.269585  5312 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:31.270900  5312 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.273489  5312 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:31.277098  4990 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.875s	user 0.013s	sys 0.004s
I20260709 13:02:31.278051  4990 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.281651  1724 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.283433  1724 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.263126  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:31.267617  5310 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:31.294348  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.297605  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:31.298419  1574 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.704s	user 0.008s	sys 0.006s
W20260709 13:02:31.305078  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 215455us
I20260709 13:02:31.305622  1574 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:31.305389  1643 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:31.311787  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:31.290983  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:31.318354  5310 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:31.318638  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:31.318899  5310 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.319344  5310 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.320271  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:31.320712  5380 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:31.321161  5380 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.321489  5386 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:31.322016  5386 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.322408  5386 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.321511  5380 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.323709  5369 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.324235  5369 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.317040  5067 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:31.325475  5067 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.325819  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.326522  1645 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:31.328401  1647 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:31.332875  5225 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:31.333683  5225 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:31.137972  5352 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:02:31.357473  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:31.296918  5260 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.821s	user 0.000s	sys 0.012s
I20260709 13:02:31.376780  1719 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:31.101068  4907 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.336989  5225 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.311334  5394 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:31.127414  5257 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.381587  5260 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:31.417985  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:31.423610  1719 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:31.401057  5394 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.412238  1724 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:02:31.431214  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 119540us
W20260709 13:02:31.416602  5233 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.005s
I20260709 13:02:31.443533  1717 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:31.445554  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:31.431936  5235 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:02:31.432979  5394 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:31.443336  5233 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.461947  4990 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.481437  5371 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.483103  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:31.505259  5369 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:31.509693  5367 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:31.511237  5395 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:31.512113  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:31.525789  1719 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:31.466436  5358 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:31.539494  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2185) took 1781 ms. Trace:
I20260709 13:02:31.539263  5310 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.539937  1723 rpcz_store.cc:276] 0709 13:02:29.757921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.971079 (+213158us) service_pool.cc:224] Handling call
0709 13:02:29.981867 (+ 10788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:30.240853 (+258986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.539189 (+1298336us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54001702d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:31.539420 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593660672}
W20260709 13:02:31.541132  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.541785  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.542881  5310 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:31.548738  5323 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:31.552861  5323 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.555819  5323 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:31.552088  5211 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:31.558291  5211 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:31.559478  5211 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.560685  5211 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.567759  5367 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.568897  5367 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:31.577855  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.590709  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:31.593199  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:31.594985  5310 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.595345  1576 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:02:31.595897  5310 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.597653  1574 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:31.598997  5310 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:31.599836  5310 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.600699  5006 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:31.601456  5006 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:31.600180  5371 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 13:02:31.607007  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 297133us
I20260709 13:02:31.594884  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.612666  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:31.612877  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:31.613559  1721 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:31.619411  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:31.619747  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:31.620838  1723 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:31.621367  1721 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:02:31.624928  1725 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:31.616819  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:31.627266  1721 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:31.628541  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:31.631247  2374 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [7.7-8.8]
I20260709 13:02:31.602696  5006 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.610805  5348 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:31.642366  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 506) took 2624 ms. Trace:
I20260709 13:02:31.642571  1719 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:31.643638  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:31.647733  1651 rpcz_store.cc:276] 0709 13:02:29.017909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.018157 (+   248us) service_pool.cc:224] Handling call
0709 13:02:29.018481 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.515947 (+497466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.642042 (+2126095us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54001ba3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:31.642281 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141685632}
I20260709 13:02:31.648384  2380 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:31.611485  5369 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.655376  1576 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:02:31.659723  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.618309  5289 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:31.619649  5233 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.667259  1570 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:31.667320  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:31.669281  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.669860  1574 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:31.652997  5175 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.872s	user 0.012s	sys 0.001s
I20260709 13:02:31.611755  5310 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.652091  5225 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:31.684669  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.618543  5323 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:02:31.685014  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:31.688385  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1465) took 2576 ms (client timeout 2867 ms). Trace:
I20260709 13:02:31.684793  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:31.689975  1642 rpcz_store.cc:269] 0709 13:02:29.112125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.112385 (+   260us) service_pool.cc:224] Handling call
0709 13:02:31.688092 (+2575707us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400389fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:31.688275 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746471296}
I20260709 13:02:31.708966  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.712486  1576 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:31.628276  5405 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:02:31.684640  5225 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.660548  5369 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.689112  5383 raft_consensus.cc:2764] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Leader election decision vote started in defunct term 5: won
I20260709 13:02:31.689611  5379 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.716586  5204 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:31.722999  5310 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:31.725963  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.716725  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.735155  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:31.731401  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:31.737852  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.738819  1651 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:31.739897  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:31.672143  5289 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.686478  5175 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:31.733968  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.202s (SENT) [suppressed 22 similar messages]
I20260709 13:02:31.727834  5225 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:31.745807  1651 raft_consensus.cc:2376] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9. Re-sending same reply.
W20260709 13:02:31.735288  5385 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.522s	user 0.016s	sys 0.000s
I20260709 13:02:31.764132  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.763932  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.765544  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.765755  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:31.745532  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:31.766690  1643 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:31.763549  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.717996  5233 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:31.805051  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:31.747023  1576 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:31.805655  1644 raft_consensus.cc:2368] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7.
I20260709 13:02:31.806766  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:31.807415  1576 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:31.768823  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:31.732911  5418 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:02:31.772680  5367 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.744814  5379 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.739295  5372 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:02:31.787309  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53370.6us
W20260709 13:02:31.805085  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59691.2us
I20260709 13:02:31.770726  5289 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:31.777217  5373 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:31.811805  5059 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.008s
I20260709 13:02:31.828826  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:31.807996  1642 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:31.837008  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.827493  5386 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.004s
W20260709 13:02:31.827075  5354 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.006s
I20260709 13:02:31.855388  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.858803  1569 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:31.865693  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.843720  5406 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:02:31.870983  1648 raft_consensus.cc:2410] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:02:31.871658  5368 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.251s	user 0.009s	sys 0.010s
I20260709 13:02:31.873183  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:31.873730  5354 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:31.874917  5354 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:31.815706  5257 raft_consensus.cc:2764] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:02:31.875036  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:31.876495  5363 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:31.849098  5059 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.859619  5386 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.883565  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.884579  1569 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:31.885200  5251 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:31.885526  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.886036  5406 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:31.886219  5363 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:31.886807  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.886621  5406 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.887384  5363 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.888589  5372 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:31.889038  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:31.889056  5372 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:31.889999  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 61298us
I20260709 13:02:31.890360  5372 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.890910  5299 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:02:31.882496  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.236s (SENT)
I20260709 13:02:31.892059  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:31.893045  5307 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:31.893306  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:31.893679  5307 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:31.893896  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.025s (SENT)
I20260709 13:02:31.894426  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:31.891913  5372 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.894833  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:31.895623  1573 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.896570  5289 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:31.897485  5406 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:31.894874  5307 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.898737  5289 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:31.890972  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:31.891672  5416 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:31.901333  5341 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.901361  5067 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.903787  5416 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:31.902791  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:31.810299  5385 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.906771  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:31.907280  5079 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:31.791148  5358 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:02:31.915504  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:31.888023  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:31.810307  5381 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:02:31.917148  1569 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:02:31.918423  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.814415  5233 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:31.817291  5398 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:31.932914  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:31.923834  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48877.5us
I20260709 13:02:31.907927  5385 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:31.932070  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.914839  5347 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.006s
I20260709 13:02:31.891958  5299 consensus_queue.cc:1243] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:31.831969  5412 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:02:31.847514  5330 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:02:31.946357  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:31.947539  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.908739  5289 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:02:31.937018  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 247352us
I20260709 13:02:31.900532  5059 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:31.918223  5373 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:31.919145  5406 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:31.907881  5079 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.952069  1569 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:31.959241  1573 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:31.968705  5225 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.947599  5251 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:31.965518  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:31.973477  1643 raft_consensus.cc:2410] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:31.969602  5416 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:31.978691  5421 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:31.978425  5416 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:31.963333  2376 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:31.986815  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:31.987334  1688 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.598s (SENT)
I20260709 13:02:31.987857  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:31.991091  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 102132us
I20260709 13:02:31.992519  2374 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:31.992846  1717 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:31.976872  5406 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:32.027293  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 119355us
I20260709 13:02:32.041656  1576 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:31.953981  5289 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.043558  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.045131  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
W20260709 13:02:32.011709  5414 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:02:32.013780  5411 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:02:31.991995  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:32.010867  5175 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.021816  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:32.035759  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:32.079378  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44362.8us
I20260709 13:02:32.046331  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.080842  1576 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:32.081780  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.080210  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:32.085613  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93684.9us
I20260709 13:02:32.086833  5369 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:32.087234  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:32.087697  5425 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.004s
I20260709 13:02:32.088249  1643 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:32.088910  5312 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.002s
I20260709 13:02:32.089272  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:32.089363  5373 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.089568  1724 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:32.087045  5230 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:32.090927  5420 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.227s	user 0.001s	sys 0.001s
I20260709 13:02:32.091118  5230 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:32.093004  5230 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.094100  5416 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:32.090018  5373 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.095466  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.101852  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:32.102365  1574 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:32.044875  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:32.089908  1643 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:32.047138  5347 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:32.103605  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.085350  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:32.105175  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.107473  1576 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:32.108430  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.109028  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:32.109799  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:32.110361  1576 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:32.105232  5175 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:32.112814  5175 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:32.113734  5175 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:32.117345  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72591.8us
I20260709 13:02:32.122941  1574 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:32.123291  1688 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.534s (SENT)
I20260709 13:02:32.123834  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:32.101441  5398 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:32.128307  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:32.105996  5367 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.117476  5312 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:32.116609  5381 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:32.145802  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:32.149272  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59051.6us
I20260709 13:02:32.150976  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:32.146932  2377 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:32.130349  5330 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:32.152940  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:32.166822  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:32.128752  5398 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:32.183416  1643 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:32.173625  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:32.187486  5373 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.167646  5312 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:32.192642  5431 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:32.193578  5431 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.196465  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:32.200459  2379 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:32.220140  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:32.167227  5369 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:32.194926  5398 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:32.193601  5289 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:32.150123  5367 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.181550  5381 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:32.189733  5312 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.194633  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:32.197870  5373 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:32.194140  5431 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.215016  5418 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:32.200479  5233 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.226359  5310 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:32.248442  5373 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.194146  5406 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.250496  5367 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.226064  5225 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:32.262097  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65773.2us
I20260709 13:02:32.269865  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:32.267051  1573 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:32.245055  5418 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.260107  5310 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:32.276149  5394 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.813s	user 0.019s	sys 0.003s
I20260709 13:02:32.289337  5359 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:32.299724  5359 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.291661  5373 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:32.240868  5379 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:32.270272  5426 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 13:02:32.276775  5422 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.175s	user 0.002s	sys 0.000s
W20260709 13:02:32.280268  5309 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:02:32.240028  5251 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:32.297730  1573 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:32.276818  5233 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.281955  5418 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.303897  5359 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:32.304301  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.318993  5432 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.179s
I20260709 13:02:32.284929  5289 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:32.283421  5330 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.295328  5310 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:32.310223  1573 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:32.245205  5369 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:32.343302  5424 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.258s	user 0.002s	sys 0.000s
I20260709 13:02:32.343814  5433 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:32.320005  5432 consensus_queue.cc:1243] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:02:32.343559  5362 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.989s	user 0.006s	sys 0.008s
I20260709 13:02:32.325299  5289 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.329149  5429 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:32.275710  5381 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.349261  5423 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:32.344722  5311 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:32.310705  5233 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.399019  5307 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:32.324067  5394 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.322060  5325 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:02:32.337380  5330 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:32.407213  5380 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.084s	user 0.013s	sys 0.002s
I20260709 13:02:32.410269  5394 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:32.411043  1568 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:32.411190  5346 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:32.411755  5441 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:32.344305  5433 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:32.415922  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:32.417696  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:32.423677  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:32.479246  1718 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:32.476200  5441 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.429674  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:32.427644  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:32.379978  5251 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:02:32.394851  5289 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:32.502614  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:32.415673  5311 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:32.497066  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:32.426815  5233 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.425544  5156 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.510672  5156 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.513384  5156 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.513959  5316 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:32.514235  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:32.514621  5316 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:32.515130  1646 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:32.516198  5316 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.530905  5316 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.424785  5330 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:32.538182  5170 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:32.497157  2379 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:32.503394  1649 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.516243  1646 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:32.479779  1721 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 6.
I20260709 13:02:32.415938  5423 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:32.454876  1568 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:32.546077  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:32.478838  5433 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:32.492950  5347 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:32.557488  1646 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:32.419137  5307 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:32.425338  5380 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.570006  1646 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:32.576229  1649 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:32.546109  5374 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.915s	user 0.006s	sys 0.007s
W20260709 13:02:32.578955  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.580742  4742 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:32.564203  5348 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.907s	user 0.007s	sys 0.004s
I20260709 13:02:32.553356  5423 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.553674  5079 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:32.559481  5330 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:32.601617  1725 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:32.570108  5347 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.608292  5330 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.640733  1718 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:32.456305  5429 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.621655  5441 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:32.648424  1645 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:32.613444  5398 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.635219  5330 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:32.653774  5326 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.050s	user 0.004s	sys 0.014s
W20260709 13:02:32.665477  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.025s	sys 0.031s
W20260709 13:02:32.671854  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.025s	sys 0.032s
I20260709 13:02:32.662870  1646 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:02:32.712919  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.751070  1649 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:32.671373  5326 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.067s	user 0.005s	sys 0.014s
I20260709 13:02:32.927114  5287 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:32.933766  5287 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:32.636677  5311 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:32.974898  5310 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.000s
I20260709 13:02:33.082921  5384 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:33.085325  5307 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:33.088809  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.090564  5448 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 13:02:33.091859  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 473) took 1126 ms. Trace:
I20260709 13:02:33.092214  1719 rpcz_store.cc:276] 0709 13:02:31.965098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.092868 (+127770us) service_pool.cc:224] Handling call
0709 13:02:32.093236 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:32.093605 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.093638 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:32.094183 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.556506 (+462323us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:32.559999 (+  3493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.560127 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.091555 (+531428us) raft_consensus.cc:1685] finished
0709 13:02:33.091574 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.091794 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7791,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":14218,"thread_start_us":461859,"threads_started":1,"wal-append.queue_time_us":568068}
I20260709 13:02:32.968492  5348 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:32.901402  5405 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.165s
I20260709 13:02:33.065636  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46374 (request call id 18) took 3222 ms. Trace:
I20260709 13:02:33.116446  1720 rpcz_store.cc:276] 0709 13:02:29.822856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.825693 (+  2837us) service_pool.cc:224] Handling call
0709 13:02:33.045414 (+3219721us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d68853 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:33.045583 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362748288}
I20260709 13:02:33.072422  5380 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:33.071208  5332 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.043857  5369 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.112100  5311 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:32.888031  5418 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:33.184571  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.224013  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.035s	sys 0.041s
W20260709 13:02:33.224751  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.036s	sys 0.041s
W20260709 13:02:33.261130  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1502) took 1520 ms (client timeout 1961 ms). Trace:
W20260709 13:02:33.261860  1648 rpcz_store.cc:269] 0709 13:02:31.740546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.768501 (+ 27955us) service_pool.cc:224] Handling call
0709 13:02:33.260278 (+1491777us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b54000ba754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:33.261011 (+   733us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113390848}
W20260709 13:02:33.265092  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.016s	sys 0.017s
W20260709 13:02:33.265496  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.016s	sys 0.017s
I20260709 13:02:33.124873  5287 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:33.126941  5430 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:33.113468  5310 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.348762  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1513) took 1233 ms. Trace:
I20260709 13:02:33.352128  1643 rpcz_store.cc:276] 0709 13:02:32.115268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.597160 (+481892us) service_pool.cc:224] Handling call
0709 13:02:32.597668 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.597966 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:32.599867 (+  1901us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.348363 (+748496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.348426 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:33.348481 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.348557 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.348696 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2372,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":81061,"thread_start_us":667010,"threads_started":1}
I20260709 13:02:33.119884  5348 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:33.342090  5406 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.023s	sys 0.069s
I20260709 13:02:33.167982  5332 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.355840  5347 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:33.370123  5406 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.023s	sys 0.070s
I20260709 13:02:33.379729  5348 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.372362  5372 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.406067  2376 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:33.406440  5398 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:33.445276  5331 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:33.368368  5430 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:33.461995  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2205) took 1837 ms (client timeout 2132 ms). Trace:
W20260709 13:02:33.462678  1725 rpcz_store.cc:269] 0709 13:02:31.625000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.670185 (+ 45185us) service_pool.cc:224] Handling call
0709 13:02:33.461677 (+1791492us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b5400294ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:33.461897 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096036736}
I20260709 13:02:33.424518  5429 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:33.446089  5331 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:33.456794  5406 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.485217  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1662) took 1504 ms. Trace:
I20260709 13:02:33.425279  5372 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.485747  1574 rpcz_store.cc:276] 0709 13:02:31.980819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.103233 (+122414us) service_pool.cc:224] Handling call
0709 13:02:33.484979 (+1381746us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b540022d8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:33.485129 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37134208}
W20260709 13:02:33.473093  5175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.033s	sys 0.057s
I20260709 13:02:33.493719  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:33.494302  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 475) took 1294 ms. Trace:
I20260709 13:02:33.494944  1725 rpcz_store.cc:276] 0709 13:02:32.199855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.477366 (+1277511us) service_pool.cc:224] Handling call
0709 13:02:33.493680 (+ 16314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.493920 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.493957 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.494006 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:33.494045 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.494109 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.494236 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.502039  5290 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:33.506249  5290 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:33.506675  5456 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:02:33.506691  5290 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.513640  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 476) took 1308 ms. Trace:
I20260709 13:02:33.514325  1725 rpcz_store.cc:276] 0709 13:02:32.205224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.497242 (+1292018us) service_pool.cc:224] Handling call
0709 13:02:33.512998 (+ 15756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.513226 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.513254 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.513303 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:33.513347 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.513408 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.513561 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:33.520287  5316 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.988s	user 0.006s	sys 0.007s
I20260709 13:02:33.520886  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 477) took 1284 ms. Trace:
I20260709 13:02:33.522503  1725 rpcz_store.cc:276] 0709 13:02:32.236882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.519762 (+1282880us) service_pool.cc:224] Handling call
0709 13:02:33.520209 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.520483 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.520512 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.520560 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.520593 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.520653 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.520818 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.524797  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 478) took 1273 ms. Trace:
W20260709 13:02:33.491065  5289 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.986s	user 0.005s	sys 0.012s
W20260709 13:02:33.520872  5445 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:02:33.525488  1725 rpcz_store.cc:276] 0709 13:02:32.251044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.523667 (+1272623us) service_pool.cc:224] Handling call
0709 13:02:33.524072 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.524276 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.524299 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.524387 (+    88us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:33.524438 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.524523 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.524716 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:33.494787  5175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.033s	sys 0.058s
I20260709 13:02:33.530584  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 481) took 1204 ms. Trace:
I20260709 13:02:33.537321  1725 rpcz_store.cc:276] 0709 13:02:32.326461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.529196 (+1202735us) service_pool.cc:224] Handling call
0709 13:02:33.529716 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.530025 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.530068 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.530140 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:33.530200 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.530301 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.530493 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.538885  5289 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:33.556504  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 516576us
I20260709 13:02:33.493642  5406 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.564674  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:33.568218  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.542698  5290 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.574862  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:33.578970  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:33.583032  2379 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:33.584039  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:33.591729  2377 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:33.593343  5418 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:33.559986  5361 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.596760  5290 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:33.597700  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.599300  5289 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:33.599613  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:33.600975  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.602124  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.602451  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.603044  1574 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:33.603164  1576 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:33.603709  5346 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.135s	user 0.005s	sys 0.012s
I20260709 13:02:33.577790  5332 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:33.572593  5452 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.002s
I20260709 13:02:33.604756  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:33.605471  1571 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.604874  1716 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:33.606734  5211 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.607235  5211 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.608743  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.609019  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 439) took 1288 ms. Trace:
I20260709 13:02:33.609517  1569 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:33.609392  2380 rpcz_store.cc:276] 0709 13:02:32.320194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.321059 (+   865us) service_pool.cc:224] Handling call
0709 13:02:32.329096 (+  8037us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:32.329604 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.329639 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:32.378585 (+ 48946us) log.cc:844] Serialized 52 byte log entry
0709 13:02:32.877591 (+499006us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:32.885708 (+  8117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.885874 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.608655 (+722781us) raft_consensus.cc:1685] finished
0709 13:02:33.608690 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.608913 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":674,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":12727,"thread_start_us":476106,"threads_started":1,"wal-append.queue_time_us":1001012}
I20260709 13:02:33.611084  1569 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:33.611493  1569 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:33.614270  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:33.615103  1568 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.615592  5375 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:33.616041  5375 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:33.624390  5375 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.626389  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1512) took 1519 ms. Trace:
I20260709 13:02:33.626863  1649 rpcz_store.cc:276] 0709 13:02:32.106976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.590818 (+483842us) service_pool.cc:224] Handling call
0709 13:02:32.591133 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.751578 (+160445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.626017 (+874439us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400307354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:33.626270 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367394560}
I20260709 13:02:33.537325  5331 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.628434  1576 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:33.628834  1576 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:33.629086  5331 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:33.629737  1576 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.630946  1576 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.632903  1725 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:33.626432  5375 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:33.617058  1569 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.604243  1574 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:33.651655  2379 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:33.623133  5211 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:33.656283  1571 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:33.656868  1568 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:33.660771  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:33.633422  5301 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:33.663269  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:33.633445  5310 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.664438  1569 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.664940  1568 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:33.667174  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:33.669435  1725 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:33.669544  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.670542  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.671329  1568 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:33.673866  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.675557  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:33.668130  5431 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.420s	user 0.016s	sys 0.004s
I20260709 13:02:33.662456  5301 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:33.678306  1571 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:33.656730  5289 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:33.664644  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.016s	sys 0.044s
I20260709 13:02:33.683524  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:33.689973  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.016s	sys 0.044s
I20260709 13:02:33.690619  1574 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:33.687014  1718 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.044s	sys 0.081s
W20260709 13:02:33.693631  1718 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.044s	sys 0.081s
W20260709 13:02:33.730856  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.699307  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:33.757773  1576 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:02:33.705066  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:33.761507  1576 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:02:33.761850  1576 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:02:33.762115  1576 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:33.764482  1725 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:33.766117  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:33.692306  5230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.228s	user 0.058s	sys 0.130s
I20260709 13:02:33.769400  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:33.769737  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64777.7us
I20260709 13:02:33.788842  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:33.789309  1725 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:33.793406  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:33.794243  1725 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.699144  5290 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.795099  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:33.799212  1574 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:33.788112  5230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.383s	user 0.059s	sys 0.130s
I20260709 13:02:33.759111  5372 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:33.814141  1573 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.050s	sys 0.062s
W20260709 13:02:33.858632  1573 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.050s	sys 0.063s
I20260709 13:02:33.694478  5310 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.694850  5211 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.699635  5332 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.787444  5314 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:33.686497  5431 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.687563  5430 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:33.822830  5301 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:33.823654  5433 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.267s	user 0.009s	sys 0.006s
I20260709 13:02:33.825420  5406 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.792243  5429 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:33.816545  5348 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:33.816126  5418 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.696403  5289 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:33.834249  5158 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:02:33.828212  5346 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.850021  1725 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:33.891534  5433 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.800582  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:33.848567  5330 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.111s	user 0.016s	sys 0.001s
I20260709 13:02:33.860281  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1656) took 1889 ms. Trace:
I20260709 13:02:33.861948  5310 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.862576  5332 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.862798  5314 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.881531  5301 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.862265  5211 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.895319  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1501) took 2155 ms. Trace:
I20260709 13:02:33.882534  5431 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.917574  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1651) took 3412 ms. Trace:
W20260709 13:02:33.858805  5458 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:02:33.921759  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:33.970819  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49263.7us
I20260709 13:02:33.816428  5290 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.936980  1723 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:33.951570  1573 rpcz_store.cc:276] 0709 13:02:31.971187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.340405 (+369218us) service_pool.cc:224] Handling call
0709 13:02:32.340873 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.341177 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:32.383056 (+ 41879us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.859414 (+1476358us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.859725 (+   311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.859801 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:33.859874 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.859983 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.860169 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4674,"prepare.queue_time_us":706934,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":444,"raft.queue_time_us":32032,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":81840,"thread_start_us":1470172,"threads_started":2,"wal-append.queue_time_us":1037560}
W20260709 13:02:33.925657  5373 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:02:33.931339  5423 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.319s	user 0.011s	sys 0.003s
I20260709 13:02:33.962662  5314 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.923677  5429 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:33.965773  1645 rpcz_store.cc:276] 0709 13:02:31.740292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.542686 (+802394us) service_pool.cc:224] Handling call
0709 13:02:32.564993 (+ 22307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.655084 (+ 90091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.894846 (+1239762us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54003fc854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:33.895200 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191293184}
I20260709 13:02:33.929375  5346 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.940932  5158 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:33.979540  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 179052us
I20260709 13:02:34.002388  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:34.002977  5433 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:33.859561  5381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.015s	sys 0.025s
I20260709 13:02:34.004573  5460 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:34.004685  5381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.025s	sys 0.026s
I20260709 13:02:34.005154  5460 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:33.961338  5330 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.996173  5423 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.972486  5289 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:33.962466  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.991870  1725 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:33.928009  5418 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:33.958227  5348 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:34.021879  5211 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.022274  5065 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.023330  5418 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.022697  5348 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.967849  1575 rpcz_store.cc:276] 0709 13:02:30.504982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.505274 (+   292us) service_pool.cc:224] Handling call
0709 13:02:30.505636 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.622431 (+1116795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.622462 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.642756 (+ 20294us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.114564 (+471808us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b1800412960. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:02:32.203447 (+ 88883us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.203506 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.917042 (+1713536us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54004218d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:33.917116 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.917185 (+    69us) raft_consensus.cc:1685] finished
0709 13:02:33.917205 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.917474 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79359,"raft.queue_time_us":138794,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":20307,"spinlock_wait_cycles":2395024384,"thread_start_us":88571,"threads_started":1,"wal-append.queue_time_us":631880,"wal-append.run_cpu_time_us":14334,"wal-append.run_wall_time_us":1106954}
I20260709 13:02:34.005615  5460 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:34.025213  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 472) took 2064 ms (client timeout 1550 ms). Trace:
W20260709 13:02:34.025753  1717 rpcz_store.cc:269] 0709 13:02:31.960877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.986483 (+ 25606us) service_pool.cc:224] Handling call
0709 13:02:34.024619 (+2038136us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b54003516d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.025113 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11528576}
I20260709 13:02:34.027470  1717 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:34.027863  5346 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:33.992712  5316 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:34.028589  5158 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:33.995280  1723 raft_consensus.cc:1217] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Deduplicated request from leader. Original: 5.6->[7.7-8.8]   Dedup: 7.7->[8.8-8.8]
I20260709 13:02:34.030263  1723 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [7.7-8.8]
I20260709 13:02:34.019764  1720 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:34.008718  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.008291  5330 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:34.028885  5316 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.003880  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:34.031167  5158 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:33.991817  5230 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:34.041538  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.042785  1651 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:34.031185  5447 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.675s	user 0.002s	sys 0.000s
I20260709 13:02:34.044734  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.048005  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.000073  5429 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:34.045987  1651 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:34.050833  1649 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:34.016232  5386 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.051365  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:34.018566  5310 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.038978  5316 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.012398  5423 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.048182  5211 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:34.020147  5290 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.056599  1650 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:34.063622  5211 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:34.065371  5211 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.053900  5235 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:02:34.077819  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.078663  1575 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:34.055054  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.035s	sys 0.034s
I20260709 13:02:34.057859  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:02:34.080580  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.035s	sys 0.034s
I20260709 13:02:34.081267  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:34.081535  1645 raft_consensus.cc:606] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:34.082288  1645 raft_consensus.cc:614] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:34.005615  5332 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.103843  1569 pending_rounds.cc:85] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:34.104755  1569 pending_rounds.cc:107] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:34.105254  1569 raft_consensus.cc:2889] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:34.096537  5316 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:34.085424  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.085518  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.086167  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.094358  5460 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.101749  1572 raft_consensus.cc:2410] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:34.102306  5158 raft_consensus.cc:476] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Not starting pre-election -- already a leader
W20260709 13:02:34.083279  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.111277  1575 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:34.092131  5311 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.918s	user 0.017s	sys 0.000s
I20260709 13:02:34.188171  1649 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:34.189522  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.194959  1650 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.190601  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.196048  1573 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:34.190042  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.104820  5386 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:34.225330  1719 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:34.225508  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.234134  5460 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:34.234351  2376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 433) took 2118 ms (client timeout 1520 ms). Trace:
W20260709 13:02:34.234947  2376 rpcz_store.cc:269] 0709 13:02:32.116244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.125140 (+  8896us) service_pool.cc:224] Handling call
0709 13:02:34.234011 (+2108871us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54006d1ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.234216 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2554866432}
W20260709 13:02:34.238441  5462 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 13:02:34.185082  5332 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.240954  5332 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.241902  5006 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:34.242475  5006 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:34.243320  5332 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.243067  5211 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.246205  5316 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.185338  5438 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.247773  5438 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.181932  5423 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:34.249557  5423 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:34.182596  5156 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.252382  5156 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.255810  5156 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:34.253088  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.266250  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.272085  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.277216  1651 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:34.292613  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.225796  5346 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.314208  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.321286  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.324656  5314 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.326668  5211 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.327101  5314 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.326735  1645 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:34.243535  5006 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.329769  5006 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.329955  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.331166  1647 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:34.332695  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.333565  1643 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:34.333896  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.268576  5386 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:34.318224  5346 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.348760  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.324885  5423 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.338380  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.350939  1645 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:34.374315  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.033s	sys 0.024s
W20260709 13:02:34.374992  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.034s	sys 0.024s
I20260709 13:02:34.377091  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 474) took 2409 ms. Trace:
I20260709 13:02:34.377635  1718 rpcz_store.cc:276] 0709 13:02:31.967737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.648113 (+680376us) service_pool.cc:224] Handling call
0709 13:02:32.648587 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.648928 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:33.694125 (+1045197us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.376518 (+682393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.376585 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:34.376643 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.376732 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.376881 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1320216,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2242,"thread_start_us":1699749,"threads_started":2}
I20260709 13:02:34.387833  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.390060  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.409283  1650 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:34.413393  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.414583  1645 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.418325  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.424170  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.427582  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.352214  5386 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.444747  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.449684  1573 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:02:34.456993  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.457619  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.459908  1717 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:34.463382  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:34.466238  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.467561  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.407601  5230 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:34.473268  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:34.434331  5331 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.805s	user 0.008s	sys 0.008s
I20260709 13:02:34.479817  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.430140  5260 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.483680  1717 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:34.435362  5204 raft_consensus.cc:476] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Not starting pre-election -- already a leader
I20260709 13:02:34.429982  5463 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.480178  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.485474  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.425989  5156 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:34.420667  5158 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:34.436805  5406 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.493737  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.507807  1647 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:34.493947  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:34.501959  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.511468  2379 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.513386  1643 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:34.497540  1614 connection.cc:256] RPC call timeout handler was delayed by -2.5667401079999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
I20260709 13:02:34.503793  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.515057  1717 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:34.518366  1688 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.961s (SENT)
I20260709 13:02:34.518975  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:34.520102  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.505000  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.009s	sys 0.005s
I20260709 13:02:34.521412  1647 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:34.527740  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.529206  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.530113  1647 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:34.530900  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.010s	sys 0.006s
I20260709 13:02:34.525079  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:34.533409  1646 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 13:02:34.534153  1646 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 3
I20260709 13:02:34.534579  1646 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 13:02:34.534844  1646 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:34.532967  1688 connection.cc:256] RPC call timeout handler was delayed by -1.292791163000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:02:34.539453  1688 connection.cc:256] RPC call timeout handler was delayed by -1.292791163000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:02:34.539779  1688 connection.cc:256] RPC call timeout handler was delayed by -1.2744752560000165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3023s for a response
W20260709 13:02:34.540064  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0882329460000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2867s for a response
W20260709 13:02:34.519994  5289 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.008s	sys 0.007s
W20260709 13:02:34.533810  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 562167us
I20260709 13:02:34.553992  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.540861  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:34.554721  1650 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:34.517395  5311 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.486688  5429 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:34.558629  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.559183  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.523538  5065 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.533814  5314 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:34.533483  5301 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.537643  5378 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.562790  1643 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:34.561599  1646 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
W20260709 13:02:34.549527  5289 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.008s	sys 0.008s
I20260709 13:02:34.564503  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.574772  1643 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:34.523118  5406 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.569792  5065 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.499914  5260 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.555481  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:34.589933  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.532320  5156 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:34.564601  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.592447  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:34.541517  5348 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.576675  1574 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 13:02:34.593079  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:34.520614  5158 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:34.581820  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.541651  5211 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.583307  1645 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:34.585994  1644 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:34.616264  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.617380  1644 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:34.562146  5311 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.577890  2379 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:34.596309  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:34.524745  5463 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.608296  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:34.648103  1571 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.606859  1575 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:34.594123  1573 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.613385  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.623615  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.651180  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.652992  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:34.614290  5467 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:02:34.683250  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.684691  1570 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:34.685487  1650 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:34.629835  1644 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:02:34.635588  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.623986  1574 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.655947  2376 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:34.656574  2374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 541) took 2741 ms (client timeout 1957 ms). Trace:
I20260709 13:02:34.659273  5463 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.580600  5301 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.587822  5378 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:34.663767  5346 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:34.664582  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2203) took 3162 ms (client timeout 1552 ms). Trace:
I20260709 13:02:34.672016  1573 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:34.672319  1572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1644) took 4207 ms (client timeout 1961 ms). Trace:
W20260709 13:02:34.673336  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2194) took 4153 ms (client timeout 1846 ms). Trace:
I20260709 13:02:34.674006  1717 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:34.674214  1722 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
W20260709 13:02:34.675794  5473 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:34.677166  5429 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:34.716488  1722 pending_rounds.cc:85] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 3
W20260709 13:02:34.694737  2374 rpcz_store.cc:269] 0709 13:02:31.915107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.915254 (+   147us) service_pool.cc:224] Handling call
0709 13:02:34.656393 (+2741139us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b540011f8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.656504 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174539648}
I20260709 13:02:34.717010  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.717052  1722 pending_rounds.cc:107] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:02:34.717906  1717 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:34.718214  1722 raft_consensus.cc:2889] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:34.609287  5348 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.709419  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.679030  5406 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.719309  1643 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:34.684231  5332 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:34.713191  1724 rpcz_store.cc:269] 0709 13:02:31.482304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.482597 (+   293us) service_pool.cc:224] Handling call
0709 13:02:34.644221 (+3161624us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b540010d6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.644424 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311944960}
W20260709 13:02:34.715636  1716 rpcz_store.cc:269] 0709 13:02:30.519770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.544418 (+ 24648us) service_pool.cc:224] Handling call
0709 13:02:34.673081 (+4128663us) spinlock_profiling.cc:243] Waited 3.06 s on lock 0x7b54001a7cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.673251 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7035228032}
W20260709 13:02:34.714943  1572 rpcz_store.cc:269] 0709 13:02:30.465151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.465401 (+   250us) service_pool.cc:224] Handling call
0709 13:02:34.671963 (+4206562us) spinlock_profiling.cc:243] Waited 3.63 s on lock 0x7b5400309b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.672190 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8359939968}
I20260709 13:02:34.733100  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.735844  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.737658  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.649420  5211 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:34.741395  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.741326  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.745848  1716 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:34.608331  5230 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:34.766160  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.731949  5474 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:34.740294  5348 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.726996  5332 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.761260  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.762423  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.769927  5065 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.770964  5158 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:34.771746  5260 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:34.771857  5382 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.777503  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 251376us
W20260709 13:02:34.776033  1614 connection.cc:256] RPC call timeout handler was delayed by -2.2081635569999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 13:02:34.893355  1614 connection.cc:256] RPC call timeout handler was delayed by -2.0929635569999618s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
I20260709 13:02:34.779996  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.894934  1643 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:34.782526  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.898233  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.897225  1644 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:34.795015  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:34.907791  1614 connection.cc:256] RPC call timeout handler was delayed by -1.5583151110000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 13:02:34.908475  1614 connection.cc:256] RPC call timeout handler was delayed by -1.104192463000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
I20260709 13:02:34.908946  1717 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:34.751631  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.910972  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.912273  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2095) took 3296 ms. Trace:
I20260709 13:02:34.913053  1722 rpcz_store.cc:276] 0709 13:02:31.610040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.610198 (+   158us) service_pool.cc:224] Handling call
0709 13:02:34.673741 (+3063543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.748866 (+ 75125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.898047 (+149181us) spinlock_profiling.cc:243] Waited 3.06 s on lock 0x7b54001a7cd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:34.905934 (+  7887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7045275008}
W20260709 13:02:34.917333  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.918111  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.918545  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.919430  1716 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.890200  5332 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:34.790004  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 249193us
I20260709 13:02:34.925060  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:34.847084  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.871057  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 231086us
I20260709 13:02:34.872206  1724 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:34.928773  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.930272  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.930385  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.931348  1717 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.931631  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:34.759491  1574 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:34.921639  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.807199  1571 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:34.923071  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:34.936558  1722 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:34.916196  5476 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:02:34.950299  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.965101  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:34.966143  1644 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:34.952102  1688 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.961s (SENT)
W20260709 13:02:34.969480  1688 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.000s (SENT)
I20260709 13:02:34.984056  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:34.927529  5156 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:34.925933  5260 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:34.937891  5065 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.004922  1718 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:35.005712  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:35.006313  1721 raft_consensus.cc:606] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:34.981245  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:34.945473  5479 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 13:02:34.937318  5483 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.940618  1724 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:34.971911  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 412601us
I20260709 13:02:34.996564  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:34.942622  5463 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:02:34.942158  5373 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:35.032845  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:35.037672  1645 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:34.944010  5311 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.992318  1716 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:35.009167  1644 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:34.934724  5158 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:02:35.017181  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:35.053735  1571 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:34.971289  5482 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:35.020280  1646 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.914042  5382 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.022317  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:35.067474  1643 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:35.032816  1717 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:34.995733  5301 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:34.953891  5257 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
W20260709 13:02:35.023583  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87854.3us
I20260709 13:02:35.069538  1651 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:02:35.078070  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.079506  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 168425us
I20260709 13:02:35.019281  5378 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:35.009986  5260 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.018824  5065 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:35.551265  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:35.132691  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.263705  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 316665us
W20260709 13:02:35.527117  1576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.010s	sys 0.006s
W20260709 13:02:35.535692  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.132954  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.523303  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.006s	sys 0.004s
W20260709 13:02:35.362241  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 287316us
W20260709 13:02:35.552598  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.558739  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.571777  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.579780  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:35.580108  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:35.581370  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:35.581626  1568 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:35.572230  1576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.006s
I20260709 13:02:35.569557  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:35.583590  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:35.572252  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:35.484611  1722 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
W20260709 13:02:35.573298  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.007s	sys 0.004s
I20260709 13:02:35.569201  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:35.595647  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 626227us
W20260709 13:02:35.597150  1688 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.202s (SENT)
I20260709 13:02:35.064971  5482 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:35.550218  5490 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:02:35.217238  5480 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
W20260709 13:02:35.223817  5486 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:02:35.606376  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:35.616987  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 236765us
I20260709 13:02:35.103955  5311 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:35.053125  5483 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.317106  5373 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:35.599911  1688 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.867s (SENT)
I20260709 13:02:35.185472  5491 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:35.591602  1722 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:35.532160  5204 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.618854  1615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.010s	sys 0.015s
W20260709 13:02:35.686257  1615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.011s	sys 0.015s
I20260709 13:02:35.185015  5301 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:35.494395  5382 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:35.391916  5158 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.614722  5378 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.063899  5156 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.543s
I20260709 13:02:35.659211  5375 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:35.127369  5211 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:35.668673  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:35.623013  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:35.705205  5257 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.451612  5497 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:35.713413  5497 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:35.688431  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 82127.8us
I20260709 13:02:35.688832  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2213) took 1226 ms. Trace:
W20260709 13:02:35.691402  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 119235us
I20260709 13:02:35.717124  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:35.716907  1722 rpcz_store.cc:276] 0709 13:02:34.443445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.922729 (+479284us) service_pool.cc:224] Handling call
0709 13:02:35.669314 (+746585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:35.717379  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:35.697320  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.155731  5489 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.591s
I20260709 13:02:35.618912  5482 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.753225  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:35.594503  1575 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:02:35.236506  5495 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.460s
W20260709 13:02:35.710074  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 140592us
I20260709 13:02:35.686056  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1690) took 1114 ms. Trace:
I20260709 13:02:35.744429  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2208) took 1302 ms. Trace:
I20260709 13:02:35.685786  5373 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:35.689354  5502 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.760066  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:35.703116  5375 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:35.770673  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 170976us
I20260709 13:02:35.763243  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:35.773054  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.773415  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:35.764920  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:35.776358  1651 raft_consensus.cc:2376] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 10. Re-sending same reply.
I20260709 13:02:35.763751  1576 rpcz_store.cc:276] 0709 13:02:34.571929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.572464 (+   535us) service_pool.cc:224] Handling call
0709 13:02:34.573004 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.573319 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:34.973878 (+400559us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.685609 (+711731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.685676 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:35.685726 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.685814 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.685973 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":858056,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":168378,"thread_start_us":1111433,"threads_started":3}
I20260709 13:02:35.764218  1725 rpcz_store.cc:276] 0709 13:02:34.441631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.461462 (+ 19831us) service_pool.cc:224] Handling call
0709 13:02:34.469326 (+  7864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.478396 (+  9070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.478450 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.487303 (+  8853us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.720205 (+232902us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:34.720248 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.720319 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.744074 (+1023755us) raft_consensus.cc:1685] finished
0709 13:02:35.744105 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.744316 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8353,"raft.queue_time_us":9264,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":4008,"thread_start_us":169328,"threads_started":1,"wal-append.queue_time_us":366647}
I20260709 13:02:35.685418  5311 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.719377  5506 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:35.696280  5204 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.798905  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:35.787878  1574 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:35.786973  1569 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:35.782281  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:35.785663  5375 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.699476  5301 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:35.870651  1646 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.811s	user 0.010s	sys 0.003s
I20260709 13:02:35.963368  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:35.872684  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:35.873977  1725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:35.751808  5501 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:35.875731  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:35.971933  1717 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:35.973474  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:35.702595  5158 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.876605  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:35.781306  5211 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:35.985549  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1537) took 1009 ms. Trace:
I20260709 13:02:35.985690  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:35.986451  1722 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:35.906140  5499 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:35.883875  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:35.889523  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:35.781620  5502 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.875741  5500 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:35.911276  5451 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 2.085s	user 0.000s	sys 0.012s
I20260709 13:02:36.016912  5065 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:36.017562  5065 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.018015  5065 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.019547  5523 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.736327  5257 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.925874  5497 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.956674  5506 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.022431  5257 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:35.919063  5232 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:35.919462  2405 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:32927 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:32927 timed out after 15.000s (SENT)
I20260709 13:02:36.023777  5232 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.025308  5311 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:35.919893  5517 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.921722  5156 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.923121  5505 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:02:35.934012  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2107) took 3784 ms. Trace:
I20260709 13:02:36.037799  1720 rpcz_store.cc:276] 0709 13:02:32.149266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.243085 (+1093819us) service_pool.cc:224] Handling call
0709 13:02:33.364879 (+121794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.034740 (+669861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.933747 (+1899007us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b5400087854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:35.933948 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504062336}
W20260709 13:02:35.934978  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 161662us
I20260709 13:02:36.040401  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:35.891924  5496 raft_consensus.cc:2764] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:35.869632  1575 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.966475  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2277) took 4230 ms. Trace:
I20260709 13:02:35.885458  5484 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:35.887149  5515 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.985296  5520 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:35.974680  1717 raft_consensus.cc:2376] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 10. Re-sending same reply.
W20260709 13:02:35.978273  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 316583us
W20260709 13:02:36.058611  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.707815  5378 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:35.987771  1722 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:35.986168  1646 rpcz_store.cc:276] 0709 13:02:34.958165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.958689 (+   524us) service_pool.cc:224] Handling call
0709 13:02:35.967422 (+1008733us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:36.008466  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.014046  5301 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:36.000545  5500 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.020257  5489 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:36.098271  5526 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:36.050119  5260 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:35.896391  5373 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.047744  1570 rpcz_store.cc:276] 0709 13:02:31.703530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.703668 (+   138us) service_pool.cc:224] Handling call
0709 13:02:34.681242 (+2977574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.688097 (+  6855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.934143 (+1246046us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b5400309b50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:35.934252 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6818028800}
I20260709 13:02:36.101897  5158 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.057381  5257 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:02:36.036545  1568 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:35.970736  5501 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:02:36.050263  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.976644  1716 raft_consensus.cc:2368] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8.
I20260709 13:02:36.103956  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:36.103832  5516 consensus_queue.cc:1175] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3b57321031416cba9db75450ef302a" responder_term: 8 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:36.045948  5518 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:36.106339  5520 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:36.106624  5530 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:36.108294  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 478) took 3950 ms. Trace:
I20260709 13:02:36.108790  2372 rpcz_store.cc:276] 0709 13:02:32.133551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.133818 (+   267us) service_pool.cc:224] Handling call
0709 13:02:34.656969 (+2523151us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.657203 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.657230 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.657769 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.897328 (+239559us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:34.897368 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.897447 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.222943 (+325496us) raft_consensus.cc:1685] finished
0709 13:02:35.222964 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.053674 (+830710us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b540011f8d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:36.083565 (+ 29891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5767368448,"thread_start_us":238968,"threads_started":1,"wal-append.queue_time_us":465482}
W20260709 13:02:36.054797  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 337499us
I20260709 13:02:35.924379  5483 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.023692  5506 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:36.026549  5232 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.006340  5502 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:36.029237  5211 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:36.096845  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.096120  5006 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:36.097532  5500 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.099294  5526 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:36.103215  5501 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.101912  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.098543  5489 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.100047  5373 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.110934  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:36.122087  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.175982  1646 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:36.049249  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:36.130328  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:36.111503  5230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.008s	sys 0.014s
I20260709 13:02:36.020114  5523 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.151717  5483 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.093389  5499 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:02:36.148609  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.186399  1574 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:02:36.151163  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45183.7us
I20260709 13:02:36.166499  1651 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:36.171518  1576 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:36.052205  5156 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.175031  5489 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:36.174235  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.177361  1644 raft_consensus.cc:2376] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 10. Re-sending same reply.
W20260709 13:02:36.182428  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 133314us
I20260709 13:02:36.215013  1573 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:36.147374  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.184912  2342 connection.cc:256] RPC call timeout handler was delayed by -1.3260311189999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 13:02:36.188421  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77603.4us
I20260709 13:02:36.222812  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:36.164687  5502 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:36.155588  5384 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:36.046046  5517 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.197295  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:36.157063  5505 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:02:36.180701  5230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.009s	sys 0.014s
I20260709 13:02:36.211076  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.081141  5484 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.158471  5515 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:02:36.198922  5006 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:36.190989  5379 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.107685  5520 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.229517  1649 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:36.236801  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.238116  5529 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:36.239562  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.257958  2372 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:36.240240  5519 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:36.244902  5536 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:36.245376  5346 raft_consensus.cc:993] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:36.226053  5310 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:36.176046  5526 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.265103  5375 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.270434  5378 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.271086  5378 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.271493  5526 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:36.271561  5378 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:36.224110  2342 connection.cc:256] RPC call timeout handler was delayed by -1.2538844950000225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33640000000000003s for a response
I20260709 13:02:36.253742  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.256207  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:36.330927  1721 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:36.259861  5438 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.260723  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.245190  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.338677  2380 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:36.239614  5523 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.225996  5502 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:36.355368  5502 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:36.361112  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:36.272429  5375 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.362633  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:36.362905  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.363438  2371 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.363780  2375 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:36.229100  5384 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.365371  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:36.368971  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 171747us
I20260709 13:02:36.365100  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.376191  2373 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:36.376892  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:36.274470  5484 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.277314  5474 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:36.267403  5204 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.265623  5211 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.252154  5006 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:36.290011  5517 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.235559  5514 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:02:36.246377  5156 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:36.294859  5470 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.704s	user 0.001s	sys 0.030s
I20260709 13:02:36.300400  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.301946  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.432317  1644 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:02:36.303073  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.308213  5332 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.363s	user 0.014s	sys 0.002s
W20260709 13:02:36.313804  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1522) took 1771 ms (client timeout 2148 ms). Trace:
W20260709 13:02:36.449822  1650 rpcz_store.cc:269] 0709 13:02:34.542463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.560043 (+ 17580us) service_pool.cc:224] Handling call
0709 13:02:36.313487 (+1753444us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b54001bb2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:36.313700 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163469056}
I20260709 13:02:36.317229  1645 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:36.451408  1645 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:36.330773  1716 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:36.332960  1575 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:36.454452  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.372946  5346 raft_consensus.cc:1081] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:36.338817  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:36.346289  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4910741940000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28809999999999997s for a response
W20260709 13:02:36.459954  1539 connection.cc:256] RPC call timeout handler was delayed by -1.4694741939999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2905s for a response
I20260709 13:02:36.348330  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.349414  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.350435  2377 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:36.350303  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.348171  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:36.462133  1721 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.348178  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.354321  5513 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:36.377391  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:36.361116  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.366446  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.415000  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:36.418349  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:36.423168  2372 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:36.367986  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:36.364208  5497 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:36.430371  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.016s	sys 0.013s
W20260709 13:02:36.498000  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.016s	sys 0.013s
W20260709 13:02:36.498656  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 745596us
I20260709 13:02:36.440964  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.194002  5483 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.372785  5537 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.373286  5331 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:36.371068  5235 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:36.459513  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:36.470111  2378 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:36.503506  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:36.504159  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44789.8us
I20260709 13:02:36.504732  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:36.505677  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:36.470188  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55308.2us
I20260709 13:02:36.411733  5536 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:36.476491  5290 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:36.477425  2375 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:36.479338  2371 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:36.310612  5500 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:36.486758  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2210) took 2044 ms (client timeout 1914 ms). Trace:
W20260709 13:02:36.526984  1723 rpcz_store.cc:269] 0709 13:02:34.442343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.444396 (+  2053us) service_pool.cc:224] Handling call
0709 13:02:36.486522 (+2042126us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:36.486665 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4131825536}
I20260709 13:02:36.488510  5300 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:36.488255  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2872018790000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 13:02:36.494215  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.495270  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:36.533975  1650 raft_consensus.cc:2376] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10. Re-sending same reply.
I20260709 13:02:36.452692  1645 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.535348  1645 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.497251  1570 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:36.499900  1572 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:36.505896  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:36.509161  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:36.412709  5474 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.473253  1571 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:36.510421  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.511551  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.423278  5301 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:36.629940  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2106) took 4492 ms. Trace:
I20260709 13:02:36.630394  1719 rpcz_store.cc:276] 0709 13:02:32.137869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.094085 (+956216us) service_pool.cc:224] Handling call
0709 13:02:33.094505 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.227251 (+1132746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.629701 (+2402450us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003dd1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:36.629885 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2587211520}
I20260709 13:02:36.541067  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:36.633466  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.634753  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.635520  1647 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:36.636260  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:36.516278  1721 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:36.485028  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:36.523288  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.637825  1646 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:36.638350  1717 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 5.
I20260709 13:02:36.639591  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.487193  5513 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:36.528461  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.510454  5382 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.528483  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:36.537132  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:36.521780  5310 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:36.457231  5517 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 13:02:36.654253  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 113333us
I20260709 13:02:36.641117  1717 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:02:36.641204  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 132157us
I20260709 13:02:36.819732  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:36.821264  1722 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:36.783912  1569 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:36.822116  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:36.823101  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.655366  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.824196  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:36.824905  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.825258  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:36.543455  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.826104  1576 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:36.827100  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:36.628304  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.824285  1574 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:36.471102  5332 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:36.711700  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.043s (SENT)
I20260709 13:02:36.628494  1572 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:36.653800  1721 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:02:36.547477  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:36.825503  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:36.833040  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:36.833603  1571 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.828660  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:36.835542  1650 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:36.516876  5375 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.829701  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:36.839222  1575 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:36.832448  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 120829us
I20260709 13:02:36.839222  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.795501  1645 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:02:36.831326  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.841004  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.051s (SENT) [suppressed 33 similar messages]
I20260709 13:02:36.841492  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.835922  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:36.513859  5331 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:36.842140  1645 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:02:36.842911  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:36.629420  5301 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.836297  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.850126  1650 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:36.521587  5519 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.827164  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.517097  5508 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:36.831468  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.851084  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.816910  5526 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:36.841836  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.115s (SENT)
I20260709 13:02:36.541596  5535 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:36.837121  5529 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.003s
W20260709 13:02:36.922582  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.519008  5538 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:36.858275  1723 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.917879  1719 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:36.918180  1576 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:36.918663  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:36.919454  1650 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:36.928891  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.931835  1575 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:36.652009  5532 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:36.862107  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:36.934073  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:36.848133  5482 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.067s	user 0.010s	sys 0.004s
I20260709 13:02:36.963356  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.964082  1576 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:36.965154  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:36.775808  5530 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:36.853343  5499 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:02:36.826246  5510 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:36.875787  5346 raft_consensus.cc:2764] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:36.949190  5493 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.961410  1723 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:36.627844  5300 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.542377  5006 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.522034  5235 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:36.852656  5301 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.866482  5386 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:36.866729  5514 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:37.027439  5514 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.028234  5514 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.853063  5477 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:36.987335  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:36.919400  5508 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:37.033352  5508 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.034076  5546 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.037402  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:37.037997  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.931967  5529 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:36.962764  5502 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 13:02:37.039825  5502 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.042840  5502 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.043596  5502 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:37.044955  5230 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:37.045377  5230 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:37.046291  5230 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.837308  5382 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:36.975767  5482 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.050343  5382 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.941365  5156 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.050750  5482 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.051059  5156 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.051834  5522 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.052431  5522 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.052845  5300 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.054131  5156 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.054581  5522 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.054768  1725 raft_consensus.cc:606] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:37.055383  1725 raft_consensus.cc:614] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.055869  5500 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:36.950210  5536 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.059118  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46374 (request call id 20) took 1320 ms. Trace:
I20260709 13:02:37.054785  5384 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.060482  5520 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:37.061661  1576 raft_consensus.cc:2410] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:37.008531  5535 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:37.069221  5535 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:37.070608  5535 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:37.006742  5501 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.822s	user 0.010s	sys 0.003s
I20260709 13:02:37.072750  5501 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.075817  5501 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.001876  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:37.081122  5539 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:02:37.081543  5384 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:36.988174  5204 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:02:36.946560  5375 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:37.082629  1575 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:37.082521  5375 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.084797  5375 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.085021  1649 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:36.973373  5530 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.985872  5526 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:37.085820  5526 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:37.086781  5526 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.087221  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2310) took 1289 ms. Trace:
I20260709 13:02:37.088912  5483 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.087729  1568 rpcz_store.cc:276] 0709 13:02:35.798349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.798538 (+   189us) service_pool.cc:224] Handling call
0709 13:02:37.087013 (+1288475us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:37.087149 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381499648}
I20260709 13:02:36.983698  5518 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:37.083834  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.091480  1575 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.525108  5537 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:37.002621  5489 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.001s
I20260709 13:02:37.104646  5489 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.107548  5489 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.089831  5331 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.108554  5518 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.109393  5331 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.110143  1720 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:36.955413  1571 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
W20260709 13:02:37.111310  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 269505us
I20260709 13:02:37.121989  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2207) took 2681 ms. Trace:
I20260709 13:02:37.122416  1718 rpcz_store.cc:276] 0709 13:02:34.441062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.462846 (+ 21784us) service_pool.cc:224] Handling call
0709 13:02:34.466672 (+  3826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.005388 (+538716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.121744 (+2116356us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400087354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:37.121928 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237417856}
I20260709 13:02:37.123777  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.111231  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.124724  1568 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.125437  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.127822  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:37.130162  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.024806  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 496566us
I20260709 13:02:36.927001  5542 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.187s
I20260709 13:02:37.082003  5384 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:37.093170  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55917us
I20260709 13:02:36.959950  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.059648  1725 rpcz_store.cc:276] 0709 13:02:35.738872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.871017 (+132145us) service_pool.cc:224] Handling call
0709 13:02:37.058815 (+1187798us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400233a54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d68853 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:37.058995 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499999616}
I20260709 13:02:37.098974  1573 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:37.100032  5378 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.102313  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.008431  5523 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:37.145792  1569 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:02:37.111330  5528 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.882s	user 0.001s	sys 0.003s
I20260709 13:02:37.151865  1569 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:37.033545  5332 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.154264  1576 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:37.016588  5493 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:37.139973  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:37.144423  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:37.100870  5530 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.154405  1575 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:37.026930  5386 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.103000  5477 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.111127  5006 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.852573  5460 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.300s
I20260709 13:02:37.163936  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:37.004830  5510 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.145459  5501 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.156610  5523 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:37.173831  1568 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:37.165263  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:37.021631  5310 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.168658  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:37.169714  2342 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:37.173233  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.170701  5515 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:37.149642  5483 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:02:37.168530  5386 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.165225  1576 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:37.197549  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:37.199165  1568 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:37.184507  5501 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.186335  5513 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:37.255445  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 86896.7us
I20260709 13:02:37.168809  5477 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:37.170106  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 343610us
W20260709 13:02:37.263759  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.265621  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 86963.7us
I20260709 13:02:37.168424  5168 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:37.183857  5316 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:37.273461  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76017.9us
I20260709 13:02:37.273823  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:37.281387  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:37.168246  5346 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.114s
I20260709 13:02:37.286120  1568 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:37.254487  5522 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:37.267249  1572 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:37.282895  5493 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.170230  5502 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:02:37.283425  5301 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.162129  5543 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:02:37.256768  5515 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.291392  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.289333  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.232728  5384 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.308100  1718 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:37.290154  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.314853  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:37.313162  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:37.312577  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.325321  1574 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:37.272622  5542 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:37.347002  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.279541  5547 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:37.345515  1725 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:37.288129  5375 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:37.195013  5489 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
W20260709 13:02:37.320869  5478 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 2.722s	user 0.003s	sys 0.004s
I20260709 13:02:37.278416  5168 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.325543  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.364775  1646 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
W20260709 13:02:37.365155  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.356745  5429 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:02:37.349099  1568 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:37.341006  1576 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:37.340065  5497 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.838s	user 0.008s	sys 0.000s
I20260709 13:02:37.352696  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:37.327538  1721 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:02:37.349143  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:37.357578  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:37.371155  1718 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:37.325235  5515 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.324286  1569 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:37.354668  5547 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:37.335311  5474 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.000s
I20260709 13:02:37.353219  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.379166  1574 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:37.340137  5496 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:02:37.366467  1570 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.000s	sys 0.015s
I20260709 13:02:37.373643  1568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:02:37.327262  5301 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.326927  5537 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.335772  5483 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:37.375576  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.356227  5375 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.338438  5065 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.328270  5495 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.318979  5513 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:02:37.381489  1568 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.402251  1568 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:37.382190  1570 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.000s	sys 0.015s
I20260709 13:02:37.363700  5542 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:37.386052  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.366019  5168 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:37.319624  5332 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:02:37.397051  1647 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:37.399364  1574 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:37.431422  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 450514us
I20260709 13:02:37.431870  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.432286  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2057) took 1100 ms. Trace:
I20260709 13:02:37.432687  1574 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:37.432724  1649 rpcz_store.cc:276] 0709 13:02:36.331437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.512614 (+181177us) service_pool.cc:224] Handling call
0709 13:02:37.432108 (+919494us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54001bb2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:37.432236 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577203584}
I20260709 13:02:37.433921  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:37.381875  5547 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:37.377801  5527 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.422974  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 425450us
I20260709 13:02:37.378407  5501 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.440142  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2128) took 1204 ms. Trace:
I20260709 13:02:37.442680  1568 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.434398  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:37.391604  5483 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.345139  5523 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.402602  5375 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.448848  1722 rpcz_store.cc:276] 0709 13:02:36.235840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.822172 (+586332us) service_pool.cc:224] Handling call
0709 13:02:36.822438 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.822662 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:37.200607 (+377945us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.439663 (+239056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.439725 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:37.439792 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.439887 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.440052 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":623509,"thread_start_us":592284,"threads_started":2}
I20260709 13:02:37.387837  5474 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.450129  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:37.477800  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2059) took 1145 ms. Trace:
W20260709 13:02:37.487855  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 206559us
W20260709 13:02:37.505146  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.473049  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2035) took 3203 ms. Trace:
I20260709 13:02:37.506994  1568 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.515859  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:37.519073  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69055.3us
W20260709 13:02:37.472187  5545 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:02:37.534086  5501 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:37.534205  5495 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:37.536284  5311 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:37.539924  5301 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:37.540421  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2124) took 1307 ms. Trace:
I20260709 13:02:37.545356  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2130) took 1309 ms. Trace:
I20260709 13:02:37.444475  5406 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:02:37.554838  5235 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.455890  5523 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.451829  5483 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.563498  1720 rpcz_store.cc:276] 0709 13:02:36.232515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.232674 (+   159us) service_pool.cc:224] Handling call
0709 13:02:37.540158 (+1307484us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54003ac354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:37.540351 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773125376}
I20260709 13:02:37.563659  1717 rpcz_store.cc:276] 0709 13:02:36.236438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.822798 (+586360us) service_pool.cc:224] Handling call
0709 13:02:36.823082 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:36.923954 (+100872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.923995 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:36.924618 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:02:37.142323 (+217705us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:37.142379 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.544803 (+402424us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003dccd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:37.544850 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.544894 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:37.544915 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.545287 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231338368,"thread_start_us":217175,"threads_started":1,"wal-append.queue_time_us":412158}
I20260709 13:02:37.473883  5518 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.495513  1647 rpcz_store.cc:276] 0709 13:02:36.332054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.636045 (+303991us) service_pool.cc:224] Handling call
0709 13:02:37.477610 (+841565us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54002e2054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:37.477738 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702517120}
I20260709 13:02:37.486507  5536 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:37.571362  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.575846  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.576210  1649 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:37.572539  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:37.520027  5510 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:37.574420  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.580864  1645 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:37.571225  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.582656  1651 raft_consensus.cc:2376] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9. Re-sending same reply.
I20260709 13:02:37.578002  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.578956  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.481113  5168 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.475564  5386 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.580247  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.562796  5378 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.557458  5558 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.588726  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.586043  1644 raft_consensus.cc:2376] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8. Re-sending same reply.
I20260709 13:02:37.562474  5311 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:37.588146  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 299007us
I20260709 13:02:37.590549  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:37.566102  5522 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:37.589377  1642 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:37.566042  5301 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:37.599167  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1706) took 1075 ms. Trace:
I20260709 13:02:37.599742  1570 rpcz_store.cc:276] 0709 13:02:36.524260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.539149 (+ 14889us) service_pool.cc:224] Handling call
0709 13:02:36.782672 (+243523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.782935 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:37.405996 (+623061us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.598797 (+192801us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:37.598845 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:37.598889 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.598951 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.599089 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":682120,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"thread_start_us":765241,"threads_started":2}
I20260709 13:02:37.600764  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.601677  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.602035  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.602783  1570 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:37.602460  1647 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:37.567986  5501 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:37.507073  1648 rpcz_store.cc:276] 0709 13:02:34.269397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.273161 (+  3764us) service_pool.cc:224] Handling call
0709 13:02:34.319129 (+ 45968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.319686 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.319725 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.324189 (+  4464us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.500071 (+175882us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:34.522829 (+ 22758us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.771551 (+248722us) spinlock_profiling.cc:243] Waited 936 us on lock 0x7b18004639c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:02:34.942162 (+170611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.942237 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.308921 (+2366684us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:37.472813 (+163892us) raft_consensus.cc:1685] finished
0709 13:02:37.472835 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.472987 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14874,"prepare.queue_time_us":34550,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":251778,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":8245,"spinlock_wait_cycles":25602304,"thread_start_us":345724,"threads_started":2,"wal-append.queue_time_us":265102}
I20260709 13:02:37.593284  1644 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:02:37.593834  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.451885  5527 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.620828  5522 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.469841  5542 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.622581  5474 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.625665  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:37.626317  5530 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.626859  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.627424  1572 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:37.628413  5530 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.591383  5536 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:37.629690  5230 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.629906  5536 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.632668  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.635123  5386 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.635988  5386 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.604408  5518 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:37.636845  5518 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.637509  5518 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.636901  5301 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.637427  5311 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.605297  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1689) took 3034 ms. Trace:
I20260709 13:02:37.638808  5311 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:37.532433  5508 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.516423  5331 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:02:37.638656  1567 rpcz_store.cc:276] 0709 13:02:34.571510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.601329 (+ 29819us) service_pool.cc:224] Handling call
0709 13:02:34.620439 (+ 19110us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.621231 (+   792us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:34.621266 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.634781 (+ 13515us) log.cc:844] Serialized 26 byte log entry
0709 13:02:36.652256 (+2017475us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b180022f060. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:02:36.980232 (+327976us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:36.980292 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.980395 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.471575 (+491180us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:37.605071 (+133496us) raft_consensus.cc:1685] finished
0709 13:02:37.605092 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.605242 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2092,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":35522,"spinlock_wait_cycles":47619584,"thread_start_us":327727,"threads_started":1,"wal-append.queue_time_us":2597835}
I20260709 13:02:37.588166  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:37.640574  1716 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:02:37.643965  1651 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
I20260709 13:02:37.628193  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:37.605487  1645 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:02:37.656133  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.667431  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.686628  5331 consensus_queue.cc:1243] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:37.695209  5495 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:37.701272  5495 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:37.704125  5495 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.705839  5495 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.636970  5204 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.722359  5168 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.677855  5386 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.726191  1571 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:02:37.726827  1571 pending_rounds.cc:85] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:37.727164  1571 pending_rounds.cc:107] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:02:37.727687  1571 raft_consensus.cc:2889] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.695689  5520 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:37.742842  5501 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.777228  5508 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.833539  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.834393  1645 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:37.623215  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.834560  5386 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.834893  1642 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.836721  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.837368  1648 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:37.838945  5515 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.840087  1724 raft_consensus.cc:606] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:37.840485  1724 raft_consensus.cc:614] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.842957  5301 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.606495  5378 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.853057  1646 raft_consensus.cc:2376] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9. Re-sending same reply.
I20260709 13:02:37.854336  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:37.855198  1649 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:37.855940  5515 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:37.857893  5514 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.829s	user 0.021s	sys 0.011s
W20260709 13:02:37.858392  5526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.000s	sys 0.020s
I20260709 13:02:37.858366  5514 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.864117  5518 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.851989  5483 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:37.845381  5477 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.021s
W20260709 13:02:37.859077  5526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.000s	sys 0.020s
I20260709 13:02:37.844117  5547 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:37.869779  5483 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.870543  5547 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.866897  5375 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.883347  5527 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.871975  5477 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:37.888653  5518 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.889537  5477 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:37.889715  5378 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.890857  5375 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:37.891741  5514 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:37.946795  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 358433us
W20260709 13:02:37.949476  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.952368  1568 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:37.955889  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.844108  1642 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:37.957932  1572 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:37.962143  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:37.963820  1649 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.965205  1650 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:37.966104  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.967763  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:37.967886  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:37.983506  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:37.997625  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1699) took 1562 ms (client timeout 1550 ms). Trace:
W20260709 13:02:37.998216  1573 rpcz_store.cc:269] 0709 13:02:36.434811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.436732 (+  1921us) service_pool.cc:224] Handling call
0709 13:02:37.997173 (+1560441us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b540028afd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:37.997387 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934738816}
I20260709 13:02:37.999867  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.004070  1575 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:02:38.004961  1649 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:38.006325  1570 raft_consensus.cc:2368] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11.
I20260709 13:02:38.007561  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:38.009195  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 418390us
I20260709 13:02:38.010664  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:38.012852  1474 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 115) took 36987 ms (client timeout 15000 ms). Trace:
I20260709 13:02:38.013290  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2134) took 1718 ms. Trace:
I20260709 13:02:38.010936  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:38.014132  1721 rpcz_store.cc:276] 0709 13:02:36.294372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.834124 (+539752us) service_pool.cc:224] Handling call
0709 13:02:36.834974 (+   850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.370890 (+535916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.012450 (+641560us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54001a7cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:38.012970 (+   520us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132266752}
W20260709 13:02:38.028707  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61150.8us
I20260709 13:02:38.029116  5520 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:38.030308  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:38.031658  5549 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.031790  5375 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.032188  5549 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:38.032801  5549 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.032935  5375 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.033567  5375 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.034832  1616 leader_election.cc:400] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11.
I20260709 13:02:38.035305  1616 leader_election.cc:403] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.040326  5518 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:38.028774  5495 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:38.043126  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2133) took 1786 ms. Trace:
I20260709 13:02:38.046628  5378 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.048722  5526 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.049548  5526 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.053217  1573 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.052770  5375 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.058588  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.060271  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.061363  1569 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:38.032817  5532 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:02:38.067205  5384 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.058650  5375 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.043761  1716 rpcz_store.cc:276] 0709 13:02:36.256942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.825149 (+568207us) service_pool.cc:224] Handling call
0709 13:02:36.847001 (+ 21852us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:37.677439 (+830438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.042291 (+364852us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b540010ead4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:38.042887 (+   596us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1824536320}
I20260709 13:02:38.077255   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:02:38.077287  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 728054us
I20260709 13:02:38.081452  5484 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.082013  5484 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.082417  5484 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.084611  5496 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.085353  5496 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.087630  5316 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:38.087890  5526 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.089391  5316 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.090343  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 449) took 1717 ms. Trace:
I20260709 13:02:38.093493  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.094309  1572 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:38.013352  1474 rpcz_store.cc:269] 0709 13:02:01.025109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.025373 (+   264us) service_pool.cc:224] Handling call
0709 13:02:01.406653 (+381280us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:02:38.012515 (+36605862us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:27.654306 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:27.655364 (+  1058us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:27.655433 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:02:27.655469 (+    36us) tablet.cc:662] Decoding operations
0709 13:02:27.797507 (+142038us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:27.797637 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:27.797684 (+    47us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:02:27.798645 (+   961us) tablet.cc:701] Row locks acquired
0709 13:02:27.798710 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:02:27.798952 (+   242us) write_op.cc:270] Start()
0709 13:02:27.799136 (+   184us) write_op.cc:276] Timestamp: P: 1783602147798871 usec, L: 0
0709 13:02:27.799163 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:02:27.851666 (+ 52503us) log.cc:844] Serialized 2766 byte log entry
0709 13:02:28.413052 (+561386us) op_driver.cc:464] REPLICATION: finished
0709 13:02:28.841960 (+428908us) write_op.cc:301] APPLY: starting
0709 13:02:28.858037 (+ 16077us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:28.924255 (+ 66218us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:28.988056 (+ 63801us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:29.805384 (+817328us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b5c0027fd8c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839f60331 00007fc839f60221 00007fc839f5af6c 00007fc8376bb30b 00007fc8376bafd3 00007fc8377b9e34 00007fc8377a597c 00007fc8377a9d61 00007fc8377a9d19 00007fc8377a9ca9 00007fc8377a9c71
0709 13:02:29.806114 (+   730us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:29.806491 (+   377us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:02:29.806546 (+    55us) write_op.cc:312] APPLY: finished
0709 13:02:29.806975 (+   429us) log.cc:844] Serialized 53 byte log entry
0709 13:02:29.967801 (+160826us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:29.973455 (+  5654us) write_op.cc:454] Released schema lock
0709 13:02:29.985170 (+ 11715us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:30.033633 (+ 48463us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:30.035299 (+  1666us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1963527,"reported_tablets":18,"spinlock_wait_cycles":29239040,"child_traces":[["op",{"apply.queue_time_us":368150,"apply.run_cpu_time_us":15931,"apply.run_wall_time_us":1201639,"num_ops":6,"prepare.queue_time_us":353764,"prepare.run_cpu_time_us":32328,"prepare.run_wall_time_us":358931,"raft.queue_time_us":205595,"raft.run_cpu_time_us":19982,"raft.run_wall_time_us":216776,"replication_time_us":613692,"spinlock_wait_cycles":7471872,"thread_start_us":848485,"threads_started":5,"wal-append.queue_time_us":425249,"wal-append.run_cpu_time_us":31210,"wal-append.run_wall_time_us":2361812}]]}
I20260709 13:02:38.098160  5496 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.103348   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:38.106887  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:38.119679  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 109038us
I20260709 13:02:38.047446  5378 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.120299  1570 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:38.122936  5378 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:38.109295  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.127529  5500 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:38.019395  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:38.119210  5496 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:38.128929  1572 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
W20260709 13:02:38.133160  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 634729us
W20260709 13:02:38.142771  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 126042us
I20260709 13:02:38.140324  5500 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:38.118207  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 408) took 1168 ms. Trace:
W20260709 13:02:38.117815  1647 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.514s	user 0.014s	sys 0.004s
I20260709 13:02:38.113703  2342 leader_election.cc:400] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8.
I20260709 13:02:38.157835  2342 leader_election.cc:403] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.157640  1575 rpcz_store.cc:276] 0709 13:02:36.950238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.167562 (+217324us) service_pool.cc:224] Handling call
0709 13:02:38.002854 (+835292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.004643 (+  1789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.117963 (+113320us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b540028afd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:38.118116 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764170112}
I20260709 13:02:38.159993  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.160732  1575 raft_consensus.cc:2376] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:38.161738  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.162927  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.163096  5549 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.164037  1647 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:38.167025  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.168121  1572 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:38.169462  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.170795  1572 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.164213  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.090831  2374 rpcz_store.cc:276] 0709 13:02:36.358538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.500528 (+141990us) service_pool.cc:224] Handling call
0709 13:02:36.500961 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.501250 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:36.969383 (+468133us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.268277 (+298894us) log.cc:844] Serialized 14 byte log entry
0709 13:02:37.468877 (+200600us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b180004b940. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:38.072745 (+603868us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b180004b940. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:02:38.074824 (+  2079us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.074989 (+   165us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:38.075400 (+   411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.075587 (+   187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:38.076081 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":812814,"prepare.queue_time_us":969898,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":820711,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":6684,"spinlock_wait_cycles":97622016,"thread_start_us":733357,"threads_started":2,"wal-append.queue_time_us":507075}
W20260709 13:02:38.189927  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.196542  1573 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:38.198575  1569 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:02:38.199102  1569 pending_rounds.cc:85] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:38.176182  5496 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:38.203788  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 90405.5us
I20260709 13:02:38.120265  5484 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.204873  1569 pending_rounds.cc:107] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:02:38.210444  1569 raft_consensus.cc:2889] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.215772  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:38.218091  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.219012  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.212273  1648 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:38.217105  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:38.224545  5522 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11.
I20260709 13:02:38.213714  5484 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:38.231527  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.187824  5532 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:02:38.241678  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.240869  5552 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:38.234087  5547 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 7. Current term is 8.
I20260709 13:02:38.236500  5204 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 13:02:38.243793  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.259343  5314 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:38.260892  5314 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:38.261314  5314 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:38.262401  5314 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:38.266299  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 50783.8us
I20260709 13:02:38.267354  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:38.233136  5518 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:02:38.288872  5168 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.289492  5168 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:38.289885  5168 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.296890  5522 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:38.297214  5483 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.297552  5522 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:38.297842  5483 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.299108  5406 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.300518  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:38.301771  5375 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:38.302228  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:38.296715  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90169.8us
I20260709 13:02:38.298760  5522 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.304571  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:38.305474  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55140.7us
I20260709 13:02:38.302961  5483 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:38.307772  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 117966us
I20260709 13:02:38.319449  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:38.320602  5311 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:38.314917  1575 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:38.325862  1570 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:02:38.329744  1572 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:38.331950  1573 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.349756  5514 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:38.354462  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.340018  5549 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.352617  5314 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.336043  5547 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:38.364497  5484 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.367574  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:38.385030  1576 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:38.386583  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.362563  5496 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.387490  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:38.389164  5386 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:38.389744  5386 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.391680  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:38.392604  5204 raft_consensus.cc:2764] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:02:38.388099  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.394464  5514 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:38.395677  5514 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:38.397397  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.375698  5311 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:38.380293  5484 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.413700  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.408717  5484 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.417219  1576 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:38.426000  5211 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:38.426868  1648 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:38.428912  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:38.432374  5501 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:38.433012  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:38.433013  5501 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:38.434712  5515 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.435328  5515 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.435637  5501 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.437071  5386 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:38.437512  5536 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:38.437644  5386 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:38.438117  5536 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:38.439301  5536 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.441283  1650 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:02:38.444572  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.444888  1573 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:38.446380  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.447957  5477 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.448217  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:38.448571  5477 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:38.450246  5587 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:02:38.451140  5477 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:38.450932  5500 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.451864  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:38.452281  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.453083  1721 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:38.448997  1716 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:38.453861  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:38.456750  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 117179us
I20260709 13:02:38.457157  5311 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.458652  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:38.459688  5348 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:38.460515  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.684s (SENT)
W20260709 13:02:38.461669  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 109881us
I20260709 13:02:38.463248  5348 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.463796  5348 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.465157  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:38.465972  1724 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:38.466735  5348 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:38.466965  5527 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:38.467453  5527 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:38.469800  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.469687  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:38.470582  1721 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:38.471518  1567 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:38.477326  1716 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:38.412714  5520 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:38.452153  5500 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:38.495069  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.517068  5522 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.520682  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:38.469748  5527 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:38.527469  1572 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:38.529278  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.531065  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.531925  1572 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:38.517763  5522 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.548872  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.547078  1567 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:38.536003  5483 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.532847  5204 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.561327  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.565297  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:38.564860  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 113088us
I20260709 13:02:38.550933  5384 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.569483  1716 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:38.540215  5501 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:02:38.562242  5483 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.588326  1721 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:38.588234  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.565358  5515 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:38.597941  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.572764  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.598121  1575 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:38.588912  5483 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.563879  5211 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 13:02:38.621357  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.573647  5384 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.620076  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:38.635486  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 114602us
W20260709 13:02:38.639068  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48495.6us
I20260709 13:02:38.645635  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:38.646751  1724 raft_consensus.cc:2410] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:38.650723  5316 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:38.653031  5235 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:02:38.654294  5384 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.662598  1576 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:38.631954  2376 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:38.661091  5204 raft_consensus.cc:2764] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:38.702316  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:38.622423  5484 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.716615  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:38.675102  5547 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:02:38.758244  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.760136  5522 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.764909  5525 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:38.765530  5525 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:38.766839  5525 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.768958  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:38.769793  5596 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:02:38.770252  5535 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:38.788897  5535 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:38.790323  5535 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:38.791474  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 108073us
I20260709 13:02:38.774905  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:38.792199  1568 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:38.792757  1576 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:38.778189  5484 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:38.781898  1647 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:02:38.784094  5602 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.081s	user 0.004s	sys 0.000s
W20260709 13:02:38.784614  5586 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:02:38.680281  5168 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.795607  5384 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.796846  5406 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:38.799965  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.801036  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.792853  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:38.803813  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.830555  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.831719  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:38.831939  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:38.832404  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.833377  1643 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.833976  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.834775  1645 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.836897  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.838903  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.813804  1648 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:38.839643  1644 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.848717  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:38.867707  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75438.5us
I20260709 13:02:38.869725  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.870584  1651 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.795063  5311 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.871598  1649 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:38.839512  5483 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.816691  5525 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.884415  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:38.886586  1647 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.828804  5406 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.826395  5384 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:38.814531  5548 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:38.898759  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.805399  5316 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:02:38.903663  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:38.902560  2376 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:38.910867  5603 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:38.911573  5603 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:38.911742  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.914954  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:38.916126  2375 raft_consensus.cc:2376] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9. Re-sending same reply.
I20260709 13:02:38.891448  5311 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:38.897758  5384 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.921772  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:38.923889  5552 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:38.924525  5552 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:38.932986  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.933302  5548 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:38.942499  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:38.945304  5211 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:38.945785  5211 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.946219  5211 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.949050  5489 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:38.949443  5483 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:38.954416  5608 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.201s	user 0.002s	sys 0.000s
I20260709 13:02:38.956691  5168 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:38.890704  5522 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:38.965837  5602 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:02:38.969301  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.972494  1724 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:38.976171  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:38.976753  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:38.977689  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 82711.8us
W20260709 13:02:38.977777  5588 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.604s	user 0.006s	sys 0.005s
W20260709 13:02:38.978900  5586 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:02:38.982908  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.916231  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:38.990700  1569 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:02:38.992551  1724 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:38.994895  1650 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:38.985532  1717 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:39.007593  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:38.986192  5603 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.008028  1721 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:39.008239  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:39.000126  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:39.013571  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:39.016615  1651 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:39.018504  2371 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:39.009114  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:39.022979  2375 raft_consensus.cc:2368] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:39.025184  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:39.030722  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.031821  1716 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:39.032927  5496 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:39.032810  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.033663  5496 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:39.033798  1724 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:39.035069  5496 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.035537  5523 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:39.037274  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2333) took 1062 ms. Trace:
W20260709 13:02:39.039027  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.037948  1571 rpcz_store.cc:276] 0709 13:02:37.974759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.979510 (+  4751us) service_pool.cc:224] Handling call
0709 13:02:37.980574 (+  1064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.981373 (+   799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.981408 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.981989 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:02:38.219802 (+237813us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:38.219843 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.227448 (+  7605us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.976663 (+749215us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:39.036728 (+ 60065us) raft_consensus.cc:1685] finished
0709 13:02:39.036754 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.037034 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2756,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":4013,"spinlock_wait_cycles":5054336,"thread_start_us":237232,"threads_started":1,"wal-append.queue_time_us":348924}
I20260709 13:02:38.995621  5168 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.040200  5168 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:39.041079  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.044030  1469 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:39.046723  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60766.5us
I20260709 13:02:39.045519  1720 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:39.048154  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:39.069738  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.081005  1644 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:39.089648  5552 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.090891  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:39.093433  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:39.094269  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:39.094964  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:39.096609  5508 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:39.097092  5460 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.097705  5552 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:39.097589  5460 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.099375  1571 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:02:39.096696  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:38.999050  5484 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:39.105765  5348 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:02:39.106055  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.805s (SENT)
I20260709 13:02:39.111325  5493 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:39.113145  5527 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.964987  5522 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.118199  1647 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:39.121130  1643 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:39.013969  5386 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.133210  1648 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:39.046629  5536 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.107880  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.147186  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.110785  1645 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:39.139756  1716 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:39.140991  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.142750  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.151110  1717 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.098502  5518 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:39.140009  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.108058  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:39.150245  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:39.153822  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.189479  1724 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:39.119891  5168 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.192857  1721 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:39.201341  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 66666.9us
I20260709 13:02:39.205389  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:39.145612  5527 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:39.187026  5518 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:39.213083  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:39.230662  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 133368us
I20260709 13:02:39.234728  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:39.260752  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.230490  5617 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.197s	user 0.005s	sys 0.000s
I20260709 13:02:39.217235  5168 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:39.263046  1721 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:39.216418  5532 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.233064  5518 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.227663  5527 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.192830  5386 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:39.271567  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78092.2us
I20260709 13:02:39.198674  5536 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.272833   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:39.208735  5484 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.268297  5348 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.297022  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:39.298187  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 72575.8us
I20260709 13:02:39.222616  5211 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:39.311102  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 160976us
I20260709 13:02:39.268517  5477 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:02:39.281001  5592 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:39.317389  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80593.2us
W20260709 13:02:39.320472  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.311566  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.323531  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.309343  5535 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:39.333648  1569 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:39.312929  5500 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:39.337404  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:39.342262  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 331424us
I20260709 13:02:39.344914  5301 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.328701  5547 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.297219  5168 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.352365  1721 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:39.333035  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.337518  5348 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.390738  5348 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.309448  5532 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:39.392839  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69650.3us
I20260709 13:02:39.311964  5406 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.394555  5168 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.273849  1469 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 8, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:39.396860  5406 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.338999  5592 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.398566  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.399857  5592 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:39.351506  5301 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.349673  5535 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:39.400703  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:39.374483  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53303.1us
W20260709 13:02:39.375586  5616 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.100s	user 0.001s	sys 0.001s
I20260709 13:02:39.378885  5484 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:39.382273  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:39.383142  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51831.5us
W20260709 13:02:39.384328  5620 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:02:39.385032  5613 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:02:39.386960  1720 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:39.388453  5615 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:39.389905  5522 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:39.345844  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.397745  5515 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:39.346509  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.403239  1725 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:39.397779  1572 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:39.414484  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.424960  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50748.9us
W20260709 13:02:39.415297  5614 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:02:39.415797  5576 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.231s	user 0.001s	sys 0.001s
I20260709 13:02:39.379454  5603 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.401940  5535 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.427527  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:39.414992  5508 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:39.448655  1717 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:39.455758  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53740.5us
I20260709 13:02:39.394590  5532 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.404711  5460 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:39.463162  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.464846  1722 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:39.397769  1469 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 6, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:39.473906  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:39.472803  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 102741us
I20260709 13:02:39.474090  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:39.477922  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:39.478021  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:39.477942  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.479846  1721 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.480579  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:39.483574  1575 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:02:39.487139  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.108s (SENT)
W20260709 13:02:39.487414  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 80752.1us
I20260709 13:02:39.489686  1571 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:39.474437  5384 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.003s
I20260709 13:02:39.495433  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.493476  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1746) took 1045 ms. Trace:
I20260709 13:02:39.508133  1570 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:39.477326  5615 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.482374  5460 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:39.485127  5348 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:39.537339  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:39.489972  5500 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:39.499230  1567 rpcz_store.cc:276] 0709 13:02:38.448196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.596231 (+148035us) service_pool.cc:224] Handling call
0709 13:02:38.610884 (+ 14653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.637871 (+ 26987us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:38.878918 (+241047us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.492328 (+613410us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b18000882a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:02:39.492751 (+   423us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.492949 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.493065 (+   116us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:39.493138 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.493232 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.493396 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81864,"prepare.queue_time_us":116801,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":9126,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":84726,"spinlock_wait_cycles":7987584,"thread_start_us":165728,"threads_started":1,"wal-append.queue_time_us":104309}
I20260709 13:02:39.547242  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1747) took 1068 ms. Trace:
I20260709 13:02:39.548746  1722 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:02:39.549691  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.558290  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65486.2us
W20260709 13:02:39.560808  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.527778  5460 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:39.563894  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76805.9us
I20260709 13:02:39.564081  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:39.564960  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 91197.3us
I20260709 13:02:39.565769  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.566639  5615 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.567236  5615 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:39.567530  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.567716  5615 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.561403  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.577438  5518 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.579807  5518 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.580989  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.580868  5615 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.582010  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:39.584757  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.498865  5384 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.547509  5547 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:39.586161  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.549950  5204 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.547039  5496 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:39.583236  5615 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.583878  5460 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.600165  1725 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:39.600234  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 113295us
I20260709 13:02:39.591174  5633 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.599992  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.951s (SENT)
I20260709 13:02:39.614347  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:39.589530  5518 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:39.618237  1725 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:39.627741  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.631528  1720 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:39.599226  5496 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:39.636444  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.637087  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:39.642556  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60604.5us
I20260709 13:02:39.619809  5518 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:39.645730  5518 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:39.649247  5636 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:39.650264  5636 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:39.630692  5552 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:39.607252  5630 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:39.661463  1721 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:39.619326  5633 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.621127  1469 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:39.670159  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.720630  1725 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.655584  5603 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.668375  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:39.683864  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.676903  5615 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.636726  5508 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:39.652926  5496 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.668627  5384 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:39.680872  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 85139.2us
W20260709 13:02:39.723157  5634 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:02:39.742062  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:39.723604  1469 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:39.726356  5527 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:39.672148  5525 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.777s	user 0.015s	sys 0.000s
I20260709 13:02:39.661490  5489 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:02:39.665347  5638 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.751555  5311 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:39.607442  5204 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.738390  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.737733  1721 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:39.675172  5211 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:39.736588  1723 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:39.741667  5644 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:39.746955  5489 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:39.748942  2375 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:39.749847  2371 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:02:39.750070  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:39.751655  2380 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:39.768914  1720 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:39.736132  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.751456  5641 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.679153  5500 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:39.774204  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 106018us
I20260709 13:02:39.777195  2372 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:39.780221  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.782156  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:39.782930  2378 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:02:39.793179  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:39.775156  5647 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.790416  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:39.778877  5489 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:39.802788  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.802984  1722 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:39.651603  5636 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.811558  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:39.813225  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.797578  5552 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.824154  1721 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
W20260709 13:02:39.815336  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.051s (SENT)
I20260709 13:02:39.824723  1721 pending_rounds.cc:85] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 6
I20260709 13:02:39.824971  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:39.844045  1721 pending_rounds.cc:107] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:39.844431  1721 raft_consensus.cc:2889] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.804171  5641 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:39.848882  1723 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:39.852231  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:39.871022  2374 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:39.784909  5644 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.857429  5641 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:39.844695  5552 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.885663  1723 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:39.864480  5622 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:39.869901  5484 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:39.892897  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77605.7us
I20260709 13:02:39.884418  5501 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:39.804762  5647 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.901289  5547 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:39.899341  5622 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.892549  1469 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 10, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:39.894507  5627 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.892133  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:39.964043  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92188.6us
W20260709 13:02:39.975646  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 164195us
W20260709 13:02:39.973143  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.191s (SENT)
I20260709 13:02:39.977162  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:39.907931  5301 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:39.982563  1719 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:39.985016  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:39.986838  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.988920  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.983464  5627 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:39.995895  5653 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:39.870059  5489 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:39.913198  5204 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:39.988827  5211 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:02:39.997313  5641 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.003762  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.006311  1722 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:02:40.005614  5648 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:02:39.991947  5493 raft_consensus.cc:2764] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:40.015872  2372 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:40.021643  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:39.995257  5644 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.031198  1725 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
W20260709 13:02:40.031615  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46752.2us
I20260709 13:02:40.034168  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:40.034269  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:02:40.043418  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.029109  5484 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.036540  5489 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.032441  5500 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:40.038388  5548 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:40.058416  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:02:40.059310  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 145100us
I20260709 13:02:40.044772  5644 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:40.061602  5493 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:02:40.052448  5211 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:40.069461  5622 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:40.068017  5508 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:40.070446  5508 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:40.071127  5647 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.069517  5493 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:40.072839  5552 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.065622  5644 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:40.080914  5644 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.071020  5508 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.088565  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.063898  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:40.072695  5493 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:40.086588  5624 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:02:40.100564  5230 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.101106  5493 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.101724  5624 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:40.102363  5624 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:40.102839  5493 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.077334  5633 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.103401  5624 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.046887  1469 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:40.105947  1469 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 9, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:40.104785  5624 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.135059  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2356) took 1364 ms. Trace:
I20260709 13:02:40.057564  5518 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:40.136206  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47795.7us
W20260709 13:02:40.136804  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 73048.7us
I20260709 13:02:40.092425  5552 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:02:40.112594  5406 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:02:40.142239  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:40.142036  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.099052  5622 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:40.136729  1576 rpcz_store.cc:276] 0709 13:02:38.770601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.893765 (+123164us) service_pool.cc:224] Handling call
0709 13:02:38.907054 (+ 13289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:38.910233 (+  3179us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:39.013233 (+103000us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.906896 (+893663us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18001780c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:02:40.028186 (+121290us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.030024 (+  1838us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.030181 (+   157us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.030220 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.030298 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.113553 (+ 83255us) raft_consensus.cc:1685] finished
0709 13:02:40.134652 (+ 21099us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.134948 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398645,"prepare.queue_time_us":29955,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":225,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":3462,"spinlock_wait_cycles":27452160,"thread_start_us":120837,"threads_started":1,"wal-append.queue_time_us":1021715}
I20260709 13:02:40.139751  5230 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:40.112790  5204 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:02:40.155373  5500 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:40.141300  5627 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.156999  5518 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:40.158672  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.008s	sys 0.015s
I20260709 13:02:40.163919  5633 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.181354  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.009s	sys 0.016s
I20260709 13:02:40.181535  5633 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:40.181775  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 988013us
I20260709 13:02:40.179253  5230 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:02:40.183091  5547 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:40.183611  5547 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:40.184288  5633 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.164439  5653 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.186923  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.188097  1725 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:40.165947  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.188851  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.189858  1720 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:40.189940  1649 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:40.169425  5631 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.154135  5641 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.174173  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:40.178890  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.195585  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1750) took 1686 ms. Trace:
I20260709 13:02:40.196396  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:40.175566  5500 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:40.205617  1576 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:40.205298  5500 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.206981  5500 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.207454  5314 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.208143  5314 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.211339  5314 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:40.214813  5536 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:40.216524  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.216645  5653 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:40.216730  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.217221  5653 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.217613  1569 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:40.217778  5653 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.218250  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.218878  1571 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:40.219920  5489 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.214854  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:40.221284  1567 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.221810  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.217162  5496 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.226066  5496 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:40.227833  2374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 457) took 1379 ms (client timeout 1550 ms). Trace:
I20260709 13:02:40.228310  2373 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:40.228413  2374 rpcz_store.cc:269] 0709 13:02:38.848043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.848265 (+   222us) service_pool.cc:224] Handling call
0709 13:02:40.227562 (+1379297us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54005fee54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:40.227709 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113724288}
I20260709 13:02:40.229692  5384 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.230387  5384 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:40.192199  5460 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.003s
I20260709 13:02:40.233844  5527 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:40.233662  5547 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.235419  5547 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.223810  1576 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:40.202575  1572 rpcz_store.cc:276] 0709 13:02:38.509353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.796662 (+287309us) service_pool.cc:224] Handling call
0709 13:02:38.797860 (+  1198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.429339 (+631479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.195275 (+765936us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54000271d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:40.195512 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1381088384}
W20260709 13:02:40.239383  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 97443us
W20260709 13:02:40.236532  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51629.6us
I20260709 13:02:40.245294  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:40.255282  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.939s (SENT)
I20260709 13:02:40.257769  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.264770  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.265661  1571 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:02:40.267661  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63409us
I20260709 13:02:40.112975  1469 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:40.274695  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.278584  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:40.259155  5631 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.276064  1642 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
I20260709 13:02:40.292980  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.300076  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.301239  1649 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:40.263926  5627 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.314687  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2238) took 1222 ms. Trace:
W20260709 13:02:40.276729  5626 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.715s	user 0.004s	sys 0.004s
W20260709 13:02:40.310660  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.348922  1716 rpcz_store.cc:276] 0709 13:02:39.090145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.096433 (+  6288us) service_pool.cc:224] Handling call
0709 13:02:40.312228 (+1215795us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b5400233554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:40.312453 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68426624}
W20260709 13:02:40.327924  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.250928  5460 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.301501  5386 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:40.274653  5641 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:02:40.254513  5536 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:02:40.352480  1569 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:40.345827  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:40.353152  5658 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:02:40.354781  1574 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:40.357159  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.356142  5644 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:40.403466  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 125065us
I20260709 13:02:40.379828  1718 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:40.381834  5460 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:40.313524  5631 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:40.418967  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 133580us
I20260709 13:02:40.313234  5483 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:02:40.368870  5518 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:40.387375  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.519424  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:40.521762  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.522064  1573 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:39.547650  1568 rpcz_store.cc:276] 0709 13:02:38.479035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.804638 (+325603us) service_pool.cc:224] Handling call
0709 13:02:38.881788 (+ 77150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.901317 (+ 19529us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:38.943229 (+ 41912us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.546411 (+603182us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.546654 (+   243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.546827 (+   173us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:39.546890 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.547039 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.547181 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":402429,"prepare.queue_time_us":395603,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":45267,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":18144,"thread_start_us":205731,"threads_started":1,"wal-append.queue_time_us":324850}
I20260709 13:02:40.367592  1572 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:02:40.385042  5592 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:40.458199  5230 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:40.458972  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.460899  5653 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.459183  5508 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:40.487324  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.490685  1567 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:40.369992  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:40.380159  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.363797  1469 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:40.368739  1570 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:40.401563  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.400152  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.382773   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:40.493139  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.542966  1717 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.493830  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.494562  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.509366  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:40.514818  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.544806  1648 raft_consensus.cc:2368] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10.
I20260709 13:02:40.545782  1646 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:40.525498  5592 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:40.549404  5592 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.550817  5592 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.525223  5384 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.518898  5632 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.553642  5632 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:40.556489  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.557282  5515 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:40.557771  5515 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.558167  5515 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.560499  5514 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:40.561928  5489 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:40.521643  5518 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:40.562345  5489 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:40.562996  5230 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:40.563171  5518 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.564599  5518 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.564177  5230 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.565806  5508 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:40.535454  5483 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:40.566468  5483 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:40.567315  1720 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:40.515344  1647 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:02:40.543838  1723 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:40.517031  1716 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:40.515224  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 200666us
I20260709 13:02:40.523077  1645 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:40.572599  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:40.525790  1719 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:40.530171  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:40.573643  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64410.3us
I20260709 13:02:40.522742  1572 pending_rounds.cc:85] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:40.532145  1642 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:02:40.575006  1642 pending_rounds.cc:85] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 8
I20260709 13:02:40.575277  1642 pending_rounds.cc:107] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:02:40.575646  1642 raft_consensus.cc:2889] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.540899  1575 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:02:40.516855  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.576442  1650 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:40.576812  1569 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:40.575100  1572 pending_rounds.cc:107] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:02:40.578168  1572 raft_consensus.cc:2889] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.578126  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.579515  1569 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:40.574579  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:40.580397  1643 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:40.538101  1576 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:40.580260  1721 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:40.571130  2371 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:40.582774  5631 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:40.584792  5522 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:40.586109  5633 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:40.526242  5168 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.584956  5631 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:40.588874  1574 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:40.591566  1717 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:40.586547  5633 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:40.648257  5633 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.648802  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.649730  1642 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.592834  5460 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:40.600963  5644 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:40.649963  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:40.650295  5460 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:40.650763  5460 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.650945  5527 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:40.651888  5527 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:40.652973  5527 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.601284  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:40.656769  2371 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:40.602854  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:40.605093  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.658272  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:40.658713  1572 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:40.649866  5633 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.662235  5653 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.663920  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.664893  1643 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:40.606779  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:40.563310  5489 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:40.610208  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.669874  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.610880  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.567467  5483 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:40.588699  5632 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:40.612780  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.622545  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.570027  1462 maintenance_manager.cc:643] P d61e0c3f41244926bf144dca0f9acf61: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.355s	user 0.204s	sys 0.123s Metrics: {"bytes_written":29720,"cfile_init":1,"dirs.queue_time_us":195538,"dirs.run_cpu_time_us":571,"dirs.run_wall_time_us":3265,"drs_written":1,"lbm_read_time_us":413,"lbm_reads_lt_1ms":4,"lbm_write_time_us":374893,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":7,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":19,"peak_mem_usage":0,"rows_written":63,"thread_start_us":162035,"threads_started":2,"wal-append.queue_time_us":149400}
I20260709 13:02:40.630240  5508 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:40.630465  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57998.6us
I20260709 13:02:40.636103  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:40.637262  5316 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:40.637857  5484 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:40.639437  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:40.643961  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.645218  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70779.1us
W20260709 13:02:40.709775  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 70431.1us
I20260709 13:02:40.710105  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:40.645557  5314 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:40.658645  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:40.713331  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55446.2us
I20260709 13:02:40.670904  1717 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:40.672449  1650 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:02:40.674336  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.683880  1642 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:40.684264  1643 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:02:40.688905  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56705.6us
W20260709 13:02:40.688594  1539 connection.cc:256] RPC call timeout handler was delayed by -2.302234111999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 13:02:40.683452  5624 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.009s
I20260709 13:02:40.690528  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.690774  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.692662  1645 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:40.717012  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.693454  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:40.694293  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.695721  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.695797  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.697494  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.701452  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:40.717885  1576 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:40.718876  1720 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:40.716920  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.733772  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.734462  1649 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:40.735396  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0448341119999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 13:02:40.735880  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9485341119999475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 13:02:40.716251  5632 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:40.737035  1651 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.737018  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:40.719126  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.741778  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:40.742650  2374 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:40.752633  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.755050  1643 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:40.782774  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.744048  5632 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.803474  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:40.803668  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:40.804261  1717 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:40.803040  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.811702  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:40.816509  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.817867  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 127307us
W20260709 13:02:40.813215  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58828.2us
I20260709 13:02:40.824324  1648 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.737720  5477 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:40.731832  5636 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.746098  5484 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.716331  5525 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:02:40.739073  5508 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:40.841408  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.807065  5314 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:40.837155  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.798579  5644 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.756575  5489 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:40.809623  5632 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.820910  5384 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.842120  2373 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:02:40.859542  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.901728  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.862684  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:40.870864  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.878342  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.756112  5548 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:02:40.802821  5627 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:02:40.835741  5477 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.838522  5653 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:02:40.888916  1724 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.886775  1649 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:40.812753  5211 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 13:02:40.932325  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.910182  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:40.916116  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.833773  5633 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:02:40.843528  5624 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:40.881916  5631 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:40.890480  5484 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.944404  1651 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:40.927806  5635 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.957568  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47521.6us
I20260709 13:02:40.953418  1463 maintenance_manager.cc:419] P d61e0c3f41244926bf144dca0f9acf61: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7766 bytes on disk
I20260709 13:02:40.961218  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:40.962138  1651 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11.
I20260709 13:02:40.931865  5477 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:40.937317  5644 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.982664  1642 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:02:41.009836  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50350.1us
I20260709 13:02:40.967953  5211 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:40.826737  1469 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:41.027019  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 290125us
I20260709 13:02:40.972060  5496 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:40.960660  5635 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.036067  1462 maintenance_manager.cc:643] P d61e0c3f41244926bf144dca0f9acf61: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.061s	user 0.002s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":500,"lbm_reads_lt_1ms":4}
I20260709 13:02:41.041034  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:41.053526  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:40.964808  5484 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:41.031715  5460 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:41.068178  5664 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:02:41.069065  5460 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:41.035542  5211 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:41.086381  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:41.077611  1720 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:41.037387  5631 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:02:41.061159  5653 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:41.087780  5638 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:41.088907  5635 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:41.002391  5508 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.038748  5314 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:41.155725  5518 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.003s
I20260709 13:02:41.137370  5638 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:41.171715  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 339612us
W20260709 13:02:41.178821  1688 connection.cc:256] RPC call timeout handler was delayed by -1.8116016289999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 13:02:41.168152  5230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.097s	sys 0.157s
W20260709 13:02:41.186283  5230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.098s	sys 0.157s
W20260709 13:02:41.083418  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45928.3us
I20260709 13:02:41.194259  1718 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:41.197010  1648 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:41.207355  1643 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:41.177707  5638 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.209642  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:41.233443  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:41.235170  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.217181  5527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.005s	sys 0.022s
I20260709 13:02:41.236826  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:41.200901  5518 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.239835  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:41.242147  1648 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:41.239496  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.006s	sys 0.035s
W20260709 13:02:41.246966  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.007s	sys 0.035s
W20260709 13:02:41.253763  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.016s	sys 0.040s
W20260709 13:02:41.254339  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.016s	sys 0.041s
I20260709 13:02:41.241253  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:41.246603  5527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.006s	sys 0.022s
W20260709 13:02:41.362180  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 121023us
I20260709 13:02:41.363019  2373 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:02:41.284080  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.284492  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.239775  5631 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:41.225281  5635 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:02:41.381569  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:41.179131  5211 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:41.385519  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.375501  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.405902  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 141949us
W20260709 13:02:41.427407  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.263218  5230 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.245587  5653 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:02:41.439271  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:41.250204  5496 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.169s
W20260709 13:02:41.456660  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.445654  1647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.008s	sys 0.008s
W20260709 13:02:41.459645  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.499239  1647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.008s	sys 0.013s
W20260709 13:02:41.503913  2380 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.600s	user 0.004s	sys 0.006s
I20260709 13:02:41.505347  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:41.500144  1644 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.008s	sys 0.012s
W20260709 13:02:41.518615  1644 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.009s	sys 0.012s
I20260709 13:02:41.519438  2380 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:41.518814  1643 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:41.446193  5518 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.566308  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:41.517316  1576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.000s	sys 0.023s
W20260709 13:02:41.570197  1576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.000s	sys 0.024s
W20260709 13:02:41.521757  1724 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.001s
I20260709 13:02:41.437716  5644 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:02:41.449757  5548 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:41.552383  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:41.581243  1643 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:41.513988  2344 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.001s	sys 0.022s
I20260709 13:02:41.497082  5527 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.574081  1650 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.778s	user 0.014s	sys 0.001s
I20260709 13:02:41.574211  1572 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:41.584718  2344 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.002s	sys 0.022s
I20260709 13:02:41.572131  1724 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:41.525243  5484 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:41.593750  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 695700us
I20260709 13:02:41.591048  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:41.591898  1650 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:41.575500  5518 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:41.586287  5552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.193s	user 0.029s	sys 0.038s
I20260709 13:02:41.449986  5230 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:41.588609  5636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.011s	sys 0.036s
W20260709 13:02:41.639726  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.787s (SENT)
W20260709 13:02:41.647929  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.022s	sys 0.058s
I20260709 13:02:41.689390  1650 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 13:02:41.692956  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.022s	sys 0.058s
W20260709 13:02:41.691256  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.688359  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 306958us
I20260709 13:02:41.694820  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:41.697299  1643 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:41.699378  1650 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:02:41.648741  5552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.255s	user 0.030s	sys 0.040s
I20260709 13:02:41.712848  5631 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:02:41.721329  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 287244us
I20260709 13:02:41.639743  5635 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:41.692590  5636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.011s	sys 0.037s
W20260709 13:02:41.742183  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 655974us
I20260709 13:02:41.589689  5548 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:41.747670  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:41.743750  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:41.703161  5653 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:41.749614  1725 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 7: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:41.755719  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 122943us
W20260709 13:02:41.757092  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.984s (SENT)
I20260709 13:02:41.757623  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:41.717464  5642 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.754232  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:41.766337  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:41.772991  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1778) took 1185 ms. Trace:
W20260709 13:02:41.749212  5515 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.185s	user 0.017s	sys 0.002s
I20260709 13:02:41.782272  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:41.783113  1718 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:41.441514  1469 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.597618  5527 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.755062  5636 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.799449  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:41.748234  5633 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:41.820467  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:41.815513  1572 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:41.778347  1569 rpcz_store.cc:276] 0709 13:02:40.587376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.872052 (+284676us) service_pool.cc:224] Handling call
0709 13:02:40.872524 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.872864 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:41.209264 (+336400us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.703543 (+494279us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.772231 (+ 68688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.772435 (+   204us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:41.772525 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.772655 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.772872 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30004,"prepare.queue_time_us":108108,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":156,"thread_start_us":823696,"threads_started":4}
W20260709 13:02:41.823096  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 105063us
I20260709 13:02:41.761749  5653 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:41.840868  1719 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:41.832232  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:41.847884  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:41.857671  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:41.859177  1717 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:41.859856  1572 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 13:02:41.860239  1572 pending_rounds.cc:85] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 3
I20260709 13:02:41.860495  1572 pending_rounds.cc:107] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 13:02:41.861164  1572 raft_consensus.cc:2889] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:41.863016  1721 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:41.889379  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1780) took 1255 ms. Trace:
I20260709 13:02:41.890064  1572 rpcz_store.cc:276] 0709 13:02:40.634174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.847382 (+1213208us) service_pool.cc:224] Handling call
0709 13:02:41.847774 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.888926 (+ 41152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.889269 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.912457  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:41.886140  5668 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.445s	user 0.000s	sys 0.002s
I20260709 13:02:41.775425  5642 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:41.845695  5703 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:41.799223  5515 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:41.928232  5552 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.929104  5696 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:02:41.929064  5552 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:41.931514  5676 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
W20260709 13:02:41.932723  5679 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.405s	user 0.000s	sys 0.002s
W20260709 13:02:41.933717  5681 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:02:41.934091  5518 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:41.935639  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2378) took 1560 ms. Trace:
W20260709 13:02:41.924998  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 100927us
I20260709 13:02:41.938571  1719 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:41.941520  5515 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:41.936169  1571 rpcz_store.cc:276] 0709 13:02:40.375685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.537699 (+162014us) service_pool.cc:224] Handling call
0709 13:02:40.538069 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.538356 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.538388 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.538995 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.247837 (+708842us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:41.247910 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.247994 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.935289 (+687295us) raft_consensus.cc:1685] finished
0709 13:02:41.935313 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.935538 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":591773,"threads_started":1,"wal-append.queue_time_us":932980}
W20260709 13:02:41.950080  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.940321  5680 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
W20260709 13:02:41.937876  5672 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.541s	user 0.002s	sys 0.000s
I20260709 13:02:41.975557  5230 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:41.938154  5697 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:41.976624  5694 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.916441  5702 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:41.977207  5548 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:41.980623  5384 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.072s	user 0.017s	sys 0.001s
I20260709 13:02:41.981238  5384 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:41.952855  5518 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:41.982018  5518 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.953563  5690 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:41.984779  5690 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.986179  5642 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:41.985567  1469 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 8, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:41.987643  5690 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:41.954280  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1775) took 1384 ms. Trace:
I20260709 13:02:41.989785  5527 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:41.954444  5706 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:41.955528  5691 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.955744  5703 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.943338  5707 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:41.925645  5631 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.831583  5636 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.980813  5653 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:41.988511  1576 rpcz_store.cc:276] 0709 13:02:40.570002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.738309 (+168307us) service_pool.cc:224] Handling call
0709 13:02:40.738707 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.824282 (+ 85575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.824318 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.857554 (+ 33236us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.571040 (+713486us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:41.571102 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.571231 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.953991 (+382760us) raft_consensus.cc:1685] finished
0709 13:02:41.954015 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.954205 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10592,"raft.queue_time_us":33417,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":12851,"thread_start_us":651189,"threads_started":1,"wal-append.queue_time_us":874769}
I20260709 13:02:41.954682  5700 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.951023  1718 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:02:41.991423  5632 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.088s	user 0.012s	sys 0.002s
I20260709 13:02:41.991765  5690 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.958237  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.032301  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.994621  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.995343  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:41.936612  5592 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.385s	user 0.018s	sys 0.000s
W20260709 13:02:41.991312  5489 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.088s	user 0.011s	sys 0.004s
W20260709 13:02:41.941246  5683 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:02:42.002648  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 172139us
W20260709 13:02:42.003747  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.007277  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:42.046365  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.047102  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.039356  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.034641  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:42.044900  1576 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:41.990880  5518 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:41.999241  5697 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:42.058569  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.010s	sys 0.008s
W20260709 13:02:42.064546  1649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.000s	sys 0.022s
W20260709 13:02:42.065176  1649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.000s	sys 0.022s
I20260709 13:02:42.067071  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:42.068428  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 98813.5us
I20260709 13:02:42.069988  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.041146  5552 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.079097  1646 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:42.079738  1614 connection.cc:256] RPC call timeout handler was delayed by -1.4696363010000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
I20260709 13:02:42.083887  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 566) took 1350 ms. Trace:
I20260709 13:02:42.045431  5527 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:42.093987  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 336922us
I20260709 13:02:42.031375  5631 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:42.106321  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 47547.3us
I20260709 13:02:41.977208  5694 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.038444  5632 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.126055  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.126989  1573 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.108738  5552 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:42.128888  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:42.129088  5552 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.114743  5697 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.046065  5707 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.075831  5698 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:42.122396  5689 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:02:42.124771  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.001488  5691 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.043506  5690 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:42.113680  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.043s (SENT)
I20260709 13:02:41.988180  1469 catalog_manager.cc:5697] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 1 to 8. New cstate: current_term: 8 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:42.145406  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:42.151702  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1781) took 1493 ms. Trace:
I20260709 13:02:42.038575  5700 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.124276  5631 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.143657  5518 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:42.167285  1650 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:42.143360  5711 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:42.146173  5688 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.165278  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.199486  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1776) took 1629 ms. Trace:
W20260709 13:02:42.186331  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 116515us
W20260709 13:02:42.201380  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87769.4us
W20260709 13:02:42.197006  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 124591us
W20260709 13:02:42.201370  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 69640.3us
I20260709 13:02:42.207167  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1589) took 1347 ms. Trace:
I20260709 13:02:42.173440  5484 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.212553  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.171191  1716 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:42.207690  1645 rpcz_store.cc:276] 0709 13:02:40.860093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.923396 (+ 63303us) service_pool.cc:224] Handling call
0709 13:02:40.987482 (+ 64086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.009087 (+ 21605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.009136 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.040636 (+ 31500us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.498067 (+457431us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:41.745984 (+247917us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.746941 (+   957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.747071 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.206911 (+459840us) raft_consensus.cc:1685] finished
0709 13:02:42.206935 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.207096 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20909,"prepare.queue_time_us":249493,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":113020,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":31462,"thread_start_us":594060,"threads_started":2,"wal-append.queue_time_us":938455}
I20260709 13:02:42.162274  5712 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:42.210126  1575 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:42.236912  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.202994  1574 rpcz_store.cc:276] 0709 13:02:40.570300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.869259 (+298959us) service_pool.cc:224] Handling call
0709 13:02:40.906365 (+ 37106us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.942433 (+ 36068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.942473 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.979771 (+ 37298us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.524043 (+544272us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:41.524092 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.524174 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.199012 (+674838us) raft_consensus.cc:1685] finished
0709 13:02:42.199036 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.199394 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13433,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":15289,"thread_start_us":477587,"threads_started":1,"wal-append.queue_time_us":668835}
I20260709 13:02:42.194538  5698 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:42.243851  2380 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:42.204659  5631 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.254793  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.255383  5719 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:42.255522  5631 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.256153  5719 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.257270  5703 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.246397  5690 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.258952  5719 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.259418  5631 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.260831  5552 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.260977  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1782) took 1544 ms. Trace:
I20260709 13:02:42.234382  5694 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.235772  5484 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.261579  1571 rpcz_store.cc:276] 0709 13:02:40.716819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.994376 (+1277557us) service_pool.cc:224] Handling call
0709 13:02:41.994862 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.001773 (+  6911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.001821 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.002512 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.205196 (+202684us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:42.205421 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.205742 (+   321us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.260723 (+ 54981us) raft_consensus.cc:1685] finished
0709 13:02:42.260747 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.260905 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6343,"raft.queue_time_us":14560,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":13068,"thread_start_us":144990,"threads_started":1,"wal-append.queue_time_us":213336}
I20260709 13:02:42.263655  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2387) took 1669 ms. Trace:
I20260709 13:02:42.264611  5484 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.266686  5700 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.268720  5632 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.238937  5712 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.269853  5712 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.236158  5708 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.248956  5691 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.199970  5518 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:42.247431  5684 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:02:42.297883  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2078) took 1625 ms. Trace:
I20260709 13:02:42.298306  1647 rpcz_store.cc:276] 0709 13:02:40.672087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.672239 (+   152us) service_pool.cc:224] Handling call
0709 13:02:40.689912 (+ 17673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.713756 (+ 23844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.713792 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.714372 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.499919 (+785547us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:41.499982 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.500049 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.297596 (+797547us) raft_consensus.cc:1685] finished
0709 13:02:42.297624 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.297784 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52015,"raft.queue_time_us":26894,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":33598,"thread_start_us":755223,"threads_started":1,"wal-append.queue_time_us":1304022}
I20260709 13:02:42.299933  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.214828  5548 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.260746  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:42.279778  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.315332  2373 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:42.280860  5536 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:42.282290  5518 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.282727  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1588) took 1423 ms. Trace:
I20260709 13:02:42.211687  5711 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.322039  5508 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:42.309697  5638 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.076s	user 0.016s	sys 0.000s
I20260709 13:02:42.323343  5638 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.308038  5483 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.320865  1642 rpcz_store.cc:276] 0709 13:02:40.859596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.859744 (+   148us) service_pool.cc:224] Handling call
0709 13:02:40.898893 (+ 39149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.983372 (+ 84479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.282413 (+1299041us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54000f4bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:42.282660 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191455616}
I20260709 13:02:42.284129  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:42.284960  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1600) took 1113 ms. Trace:
W20260709 13:02:42.241614  5477 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.217s	user 0.008s	sys 0.010s
I20260709 13:02:42.275936  5527 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:42.276520  1469 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 6 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:42.291249  5692 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:42.292062  5708 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:42.335218  5708 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.333884  1469 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.336673  5477 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.336985  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:42.338061  2380 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:42.326005  5536 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:42.292706  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 89254us
I20260709 13:02:42.341033  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.342025  5508 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:42.342470  5508 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:42.324718  5698 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.343423  5508 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.344940  5508 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.344808  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.294697  5636 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.264077  1570 rpcz_store.cc:276] 0709 13:02:40.594504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.727372 (+1132868us) service_pool.cc:224] Handling call
0709 13:02:41.727926 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.848966 (+121040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.849118 (+   152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.967013 (+117895us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.167928 (+200915us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:42.167985 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.168085 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.263391 (+ 95306us) raft_consensus.cc:1685] finished
0709 13:02:42.263416 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.263587 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":225768,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"thread_start_us":256148,"threads_started":2,"wal-append.queue_time_us":254187}
I20260709 13:02:42.347258  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.314399  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:42.314494  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 73122.5us
I20260709 13:02:42.348980  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.349418  5711 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.349386  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.260095  5707 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.326563  1643 rpcz_store.cc:276] 0709 13:02:41.171764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.698397 (+526633us) service_pool.cc:224] Handling call
0709 13:02:41.698779 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.712727 (+ 13948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.712766 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.713840 (+  1074us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.055994 (+342154us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.056041 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.056130 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.284698 (+228568us) raft_consensus.cc:1685] finished
0709 13:02:42.284722 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.284894 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7992,"raft.queue_time_us":79307,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":19269,"thread_start_us":310716,"threads_started":1,"wal-append.queue_time_us":568709}
W20260709 13:02:42.295439  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.347419  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.312700  1573 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:42.340001  5477 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.355012  1688 leader_election.cc:400] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:42.351367  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.361105  1688 leader_election.cc:403] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:42.341118  5536 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.355469  5527 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.355805  5624 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.356935  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1590) took 1496 ms. Trace:
I20260709 13:02:42.356299  1469 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.327493  1576 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:42.352985  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.328820  5552 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:42.362859  5714 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.326822  5548 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.371840  5638 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.374482  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2153) took 1764 ms. Trace:
I20260709 13:02:42.375488  5707 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:42.363092  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:42.377424  5493 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:42.380519  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.387658  1719 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:42.382730  1723 rpcz_store.cc:276] 0709 13:02:40.610393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.622144 (+ 11751us) service_pool.cc:224] Handling call
0709 13:02:42.374171 (+1752027us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400213754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:42.374395 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3236279040}
W20260709 13:02:42.396656  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.910s (SENT)
I20260709 13:02:42.397392  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:42.363317  1649 rpcz_store.cc:276] 0709 13:02:40.863053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.942178 (+ 79125us) service_pool.cc:224] Handling call
0709 13:02:40.963873 (+ 21695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.964314 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:41.451094 (+486780us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.194680 (+743586us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.194811 (+   131us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.232662 (+ 37851us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.238434 (+  5772us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:42.243524 (+  5090us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.355606 (+112082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.356506 (+   900us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.356558 (+    52us) raft_consensus.cc:1685] finished
0709 13:02:42.356575 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.356851 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90021,"prepare.queue_time_us":759624,"prepare.run_cpu_time_us":257,"prepare.run_wall_time_us":20576,"raft.queue_time_us":116042,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":199,"thread_start_us":1021890,"threads_started":3,"wal-append.queue_time_us":689023}
I20260709 13:02:42.351742  5711 raft_consensus.cc:2764] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:42.394532  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.408604  5638 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.414861  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.406759  5712 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.421829  5525 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.421700  5638 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.423234  1469 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:42.425771  1469 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:42.384327  5548 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:42.429219  5677 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.724s	user 0.004s	sys 0.000s
I20260709 13:02:42.429282  5638 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:42.432905  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81291.6us
I20260709 13:02:42.433672  5711 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:42.433925  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.434357  5711 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.392560  5493 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:42.436267  5493 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:42.437335  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.437546  5493 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:42.440493  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.441337  5666 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.442375  5666 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.443058  5666 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.085326  1644 rpcz_store.cc:276] 0709 13:02:40.713739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.734799 (+ 21060us) service_pool.cc:224] Handling call
0709 13:02:40.735199 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.735443 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:41.519629 (+784186us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.060386 (+540757us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.063624 (+  3238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.063691 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:42.063753 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.063830 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.063981 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2930,"prepare.queue_time_us":896446,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4453,"raft.queue_time_us":11432,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":29134,"spinlock_wait_cycles":23772928,"thread_start_us":1230560,"threads_started":2}
I20260709 13:02:42.450240  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.450819  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.452543  1719 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.452272  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.455539  1575 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:42.457746  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.464138  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.464255  2373 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:42.392752  5235 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 8. Current term is 9.
I20260709 13:02:42.537663  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:42.533125  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 82436.6us
I20260709 13:02:42.534519  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.539901  1573 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:42.541564  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.403596  5714 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:42.574575  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.544932  2380 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:42.545631  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.577258  1575 raft_consensus.cc:2368] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
W20260709 13:02:42.551281  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.555706  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.381659  5552 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.585481  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.568838  1573 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:02:42.577983  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 118353us
W20260709 13:02:42.579082  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.591117  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49642.6us
I20260709 13:02:42.593014  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.591251  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.590876  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.594090  2374 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:42.414951  5728 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.152s
I20260709 13:02:42.595503  1573 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:42.595853  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:42.597959  2373 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:42.597492  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.773s (SENT)
I20260709 13:02:42.598632  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.552038  5622 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:42.600610  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2157) took 1923 ms (client timeout 1764 ms). Trace:
W20260709 13:02:42.602274  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2155) took 1932 ms (client timeout 1697 ms). Trace:
W20260709 13:02:42.602718  1716 rpcz_store.cc:269] 0709 13:02:40.669546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.669709 (+   163us) service_pool.cc:224] Handling call
0709 13:02:42.602099 (+1932390us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001702d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:42.602199 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3072688000}
I20260709 13:02:42.603281  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.444132  5707 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.606668  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.604712  1718 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:02:42.605168  1719 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:02:42.601102  1724 rpcz_store.cc:269] 0709 13:02:40.677666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.701041 (+ 23375us) service_pool.cc:224] Handling call
0709 13:02:42.600392 (+1899351us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400294ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:42.600532 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354476288}
I20260709 13:02:42.608958  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:42.617630  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.618163  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:42.620195  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.621590  1568 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:42.624734  1724 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:02:42.624842  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.630201  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.632159  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.633977  5699 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:42.635324  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.636830  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.595554  5698 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.641014  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.642645  1576 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:02:42.592744  5711 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.622679  5514 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:02:42.646628  5712 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.647353  5477 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:42.649282  5477 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:42.649881  5477 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:42.650969  5477 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.652643  5666 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:42.654284  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:42.655004  2374 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.655170  5508 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.626183  5690 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:42.658619  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.643774  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.660997  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.662375  1724 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:42.663072  1576 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:42.427470  1469 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 2 to 5, term changed from 3 to 7, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196), da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:42.596606  5638 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:42.658280  5703 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:42.607319  5644 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:42.651877  5707 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.669700  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:42.675884  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.698879  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.699426  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.700496  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.688501  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.701262  2378 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:42.684773  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.691949  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.702517  1724 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:42.597209  5714 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:42.675652  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.703847  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.717162  2380 raft_consensus.cc:2376] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10. Re-sending same reply.
I20260709 13:02:42.712479  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1779) took 2078 ms. Trace:
I20260709 13:02:42.717742  1568 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:02:42.686123  5638 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.685745  5708 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.705942  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:42.693289  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:42.632885  5535 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:42.656531  5690 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.729240  5211 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.675493  5699 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.675302  5726 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:42.709642  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 85946.3us
I20260709 13:02:42.668318  5592 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:42.706537  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.675299  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.676071  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.743682  2372 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:42.689941  2374 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:42.701126  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:42.689164  5703 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:42.712245  1645 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:42.746835  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:42.710021  1643 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:42.714257  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.713984  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.748659  1570 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:42.748970  1647 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:42.701855  2373 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:42.716766  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:42.716956  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.748138  1642 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
W20260709 13:02:42.751266  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50234.3us
I20260709 13:02:42.751487  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.752588  1642 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 6.
I20260709 13:02:42.754153  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:42.754607  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.718997  1569 rpcz_store.cc:276] 0709 13:02:40.633761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.844810 (+1211049us) service_pool.cc:224] Handling call
0709 13:02:41.845220 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.845486 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:41.896430 (+ 50944us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.440119 (+543689us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b1800402be0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:02:42.711253 (+271134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.711362 (+   109us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:42.711466 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.711663 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.712201 (+   538us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118861,"prepare.queue_time_us":50697,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4105,"raft.queue_time_us":86912,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":20842368,"thread_start_us":183960,"threads_started":1,"wal-append.queue_time_us":810931}
I20260709 13:02:42.684626  1573 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:42.757776  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:42.758486  1649 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:42.758579  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2289) took 1007 ms. Trace:
I20260709 13:02:42.730042  1575 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:42.760123  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:42.759603  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.763191  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.189949  1572 rpcz_store.cc:276] 0709 13:02:40.658456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.891258 (+1232802us) service_pool.cc:224] Handling call
0709 13:02:41.891811 (+   553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.892188 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:42.043858 (+151670us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.151062 (+107204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.151142 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.151235 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.151366 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.151591 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23424,"prepare.queue_time_us":82934,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":10504,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":15683,"thread_start_us":160622,"threads_started":2}
I20260709 13:02:42.688060  5552 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.745402  1576 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.715832  5483 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.776628  5713 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:42.787428  1469 catalog_manager.cc:5697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.789913  1469 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:02:42.791776  5670 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.553s	user 0.004s	sys 0.000s
W20260709 13:02:42.801069  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47049us
I20260709 13:02:42.735019  5627 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.830199  1642 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:02:42.771845  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55214.1us
I20260709 13:02:42.773854  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.764153  1569 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:42.759276  1725 rpcz_store.cc:276] 0709 13:02:41.751014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.383777 (+632763us) service_pool.cc:224] Handling call
0709 13:02:42.385698 (+  1921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.386125 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.386155 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.392274 (+  6119us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.565863 (+173589us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.588027 (+ 22164us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.654928 (+ 66901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.655014 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.758299 (+103285us) raft_consensus.cc:1685] finished
0709 13:02:42.758321 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.758505 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102556,"prepare.queue_time_us":21921,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":77110,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":10975,"thread_start_us":121140,"threads_started":1,"wal-append.queue_time_us":274488}
I20260709 13:02:42.740243  5710 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:42.786844  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 569) took 1542 ms. Trace:
I20260709 13:02:42.769068  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.837836  1573 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:42.769035  1539 leader_election.cc:400] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10.
I20260709 13:02:42.838764  1539 leader_election.cc:403] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:42.839741  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70924.5us
I20260709 13:02:42.796552  5642 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.778182  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:42.777113  1647 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:42.841847  1649 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:42.750710  5592 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:42.801612  5735 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.384s	user 0.002s	sys 0.000s
I20260709 13:02:42.809612  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.800637  5705 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:42.769271  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.779676  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.855074  1568 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:42.801676  5493 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:42.780148  5653 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:42.782706  5701 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:42.803565  5708 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:42.810286  5703 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.770190  5712 raft_consensus.cc:2764] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:42.799413  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:42.873626  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.874122  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.758353  5622 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:42.876706  5622 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.877462  5622 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:42.879300  5592 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.832127  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:42.881163  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49158.1us
I20260709 13:02:42.883153  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:42.740093  5547 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:42.833720  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:42.819461  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:42.825770  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.891194  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:42.833525  1645 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:42.836283  1651 rpcz_store.cc:276] 0709 13:02:41.244033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.273163 (+1029130us) service_pool.cc:224] Handling call
0709 13:02:42.273529 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.324107 (+ 50578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.324139 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.324744 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.593899 (+269155us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:42.593942 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.594015 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.786520 (+192505us) raft_consensus.cc:1685] finished
0709 13:02:42.786546 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.786756 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62895,"raft.queue_time_us":51337,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":4650,"thread_start_us":268732,"threads_started":1,"wal-append.queue_time_us":421566}
I20260709 13:02:42.847568  1539 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:42.847553  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:42.892750  1539 leader_election.cc:403] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:02:42.848852  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:42.851389  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2389) took 1427 ms. Trace:
I20260709 13:02:42.800972  5690 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:42.854803  1716 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:42.857888  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2161) took 2168 ms. Trace:
I20260709 13:02:42.858670  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2079) took 2177 ms. Trace:
I20260709 13:02:42.813930  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.858260  5705 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:42.808331  5518 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.869182  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.870561  5701 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:42.884932  1688 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11.
I20260709 13:02:42.885277  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.888295  1576 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:42.841756  1572 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.791647  1469 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.893898  1573 rpcz_store.cc:276] 0709 13:02:41.424216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.850083 (+1425867us) service_pool.cc:224] Handling call
0709 13:02:42.850567 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.850858 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.850901 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.850972 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:42.851041 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.851128 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.851293 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:42.898788  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.906075  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2391) took 1481 ms. Trace:
I20260709 13:02:42.940285  1574 rpcz_store.cc:276] 0709 13:02:41.424917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.862600 (+1437683us) service_pool.cc:224] Handling call
0709 13:02:42.905113 (+ 42513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.905390 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.905432 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:42.905499 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.905545 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.905622 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.905818 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:42.853266  5713 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10.
I20260709 13:02:42.933776  1688 leader_election.cc:403] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:42.939638  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 512) took 2538 ms. Trace:
I20260709 13:02:42.911015  1720 rpcz_store.cc:276] 0709 13:02:40.680947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.763222 (+ 82275us) service_pool.cc:224] Handling call
0709 13:02:40.764015 (+   793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.088291 (+324276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.848630 (+1760339us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54003dccd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:42.848904 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712832896}
W20260709 13:02:42.943397  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58618.5us
I20260709 13:02:42.946475  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:42.933965  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92981us
I20260709 13:02:42.912513  1646 rpcz_store.cc:276] 0709 13:02:40.672615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.672750 (+   135us) service_pool.cc:224] Handling call
0709 13:02:40.689487 (+ 16737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.086024 (+1396537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.849537 (+763513us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540038a9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:42.849776 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3194748160}
I20260709 13:02:42.949642  1572 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:42.951568  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:42.953701  1573 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:42.953352  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.955369  1721 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:42.958402  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:42.959491  1716 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:42.963987  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:42.854040  5711 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:02:42.968524  1725 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:42.942816  1722 rpcz_store.cc:276] 0709 13:02:40.401429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.401560 (+   131us) service_pool.cc:224] Handling call
0709 13:02:40.447226 (+ 45666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.549634 (+102408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.549675 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.611551 (+ 61876us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b2c000590a8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:02:40.612092 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.016568 (+404476us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:41.016614 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.016697 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.790572 (+1773875us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:42.939345 (+148773us) raft_consensus.cc:1685] finished
0709 13:02:42.939371 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.939561 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48881,"raft.queue_time_us":58849,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":120,"spinlock_wait_cycles":59387520,"thread_start_us":403964,"threads_started":1,"wal-append.queue_time_us":563591}
I20260709 13:02:42.965821  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:42.972577  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:42.975291  1567 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:42.778705  5714 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.982918  1721 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:02:42.889015  5547 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:42.895695  5527 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:42.915551  5642 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:02:43.007861  5642 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:43.009677  1725 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:42.986173  1572 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:42.987748  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.997774  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.996825  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.011453  1648 raft_consensus.cc:2376] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:02:43.011507  1642 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
W20260709 13:02:43.012141  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1603) took 1772 ms (client timeout 1889 ms). Trace:
I20260709 13:02:43.013307  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.848448  5710 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:42.893896  5688 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.949165  5708 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:43.003392  1716 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:43.001850  1719 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:43.004455  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 113335us
I20260709 13:02:43.025300  1574 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:42.992450  5712 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:43.005385  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 157956us
I20260709 13:02:42.985467  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:42.975577  5314 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:43.017616  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.039697  1647 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:43.020017  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.041172  1645 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:43.025923  1574 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:43.012805  1648 rpcz_store.cc:269] 0709 13:02:41.240378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.952895 (+1712517us) service_pool.cc:224] Handling call
0709 13:02:43.012067 (+ 59172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:42.975890  5211 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11.
I20260709 13:02:42.958446  5641 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:43.021976  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.045497  1642 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:42.975390  5592 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.004771  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:42.989362  5514 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:43.019292  5726 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.085840  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:43.031688  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.032032  5493 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:43.033701  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48360.3us
I20260709 13:02:43.037469  5707 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:43.037909  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:43.046772  1649 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:43.048564  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:43.027284  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:43.051559  2373 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:43.053145  1722 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:43.043756  5644 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.056809  5728 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.057662  5518 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:43.029589  5699 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:43.058569  5547 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.059508  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1602) took 1842 ms. Trace:
I20260709 13:02:43.030097  5701 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.028669  5636 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:02:43.016556  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:43.044108  5638 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.126875  1720 raft_consensus.cc:606] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:43.068873  1469 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 9, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:43.070950  5712 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:43.073330  5641 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:43.073746  1567 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:43.142843  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1786) took 1022 ms. Trace:
I20260709 13:02:43.143234  1567 rpcz_store.cc:276] 0709 13:02:42.120004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.072613 (+952609us) service_pool.cc:224] Handling call
0709 13:02:43.073302 (+   689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.142480 (+ 69178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.142770 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:43.092134  5726 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:43.087721  1643 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:43.020769  5642 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:43.154651  1572 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:43.047231  1574 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.093823  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:43.112866  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75094.5us
W20260709 13:02:43.156001  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 66410.1us
I20260709 13:02:43.044950  5698 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.117700  1650 rpcz_store.cc:276] 0709 13:02:41.217541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.718577 (+501036us) service_pool.cc:224] Handling call
0709 13:02:41.718950 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.196472 (+477522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.059173 (+862701us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b5400090bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:43.059432 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030310272}
I20260709 13:02:43.049945  5508 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.029263  5460 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
W20260709 13:02:43.089952  5697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.040s	sys 0.054s
W20260709 13:02:43.189985  5697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.040s	sys 0.054s
I20260709 13:02:43.166998  5708 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:43.191291  5708 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:43.192791  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.192713  5708 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.193849  1647 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:02:43.118880  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.339s (SENT) [suppressed 18 similar messages]
I20260709 13:02:43.196903  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2290) took 1445 ms. Trace:
I20260709 13:02:43.197392  1723 rpcz_store.cc:276] 0709 13:02:41.751826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.395779 (+643953us) service_pool.cc:224] Handling call
0709 13:02:42.396130 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.410477 (+ 14347us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:42.589100 (+178623us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.745008 (+155908us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.842375 (+ 97367us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.842610 (+   235us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:42.843022 (+   412us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.843274 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.843382 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.874565 (+ 31183us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b1800401f20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:43.129352 (+254787us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b1800401f20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:43.196677 (+ 67325us) raft_consensus.cc:1685] finished
0709 13:02:43.196701 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.196838 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131430,"prepare.queue_time_us":136541,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":81,"raft.queue_time_us":4739,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":29448,"spinlock_wait_cycles":19883520,"thread_start_us":133525,"threads_started":2,"wal-append.queue_time_us":232600}
I20260709 13:02:43.129288  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:43.199955  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70764.1us
I20260709 13:02:43.201128  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.202044  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.138967  5701 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.208712  5211 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.209442  5211 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.209959  5211 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.132593  5666 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:02:43.212723  5477 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:43.215333  5666 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:43.217219  1567 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:02:43.222584  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:43.225714  5642 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.227375  5642 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.227572  5235 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:43.228497  5235 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:43.192090  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.144739  5547 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.234380  1648 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:43.151041  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:43.153621  1570 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:43.131352  5536 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:43.156798  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:43.251067  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94367.3us
I20260709 13:02:43.253177  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.161464  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1787) took 1041 ms. Trace:
I20260709 13:02:43.254035  1576 rpcz_store.cc:276] 0709 13:02:42.120306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.160259 (+1039953us) service_pool.cc:224] Handling call
0709 13:02:43.160778 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.161013 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.161050 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:43.161106 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:43.161152 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.161228 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.161381 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:43.163482  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2277) took 1432 ms. Trace:
I20260709 13:02:43.119589  5707 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:43.116281  5644 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.142138  5641 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:43.130465  5633 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.172796  5698 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.181424  1574 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.300148  5633 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:43.300717  5633 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.183799  1575 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.186841  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.187669  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1783) took 1068 ms. Trace:
I20260709 13:02:43.238250  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2398) took 1031 ms. Trace:
W20260709 13:02:43.303509  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 146765us
I20260709 13:02:43.303876  1570 rpcz_store.cc:276] 0709 13:02:42.206961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.152994 (+946033us) service_pool.cc:224] Handling call
0709 13:02:43.153412 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.237765 (+ 84353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.238140 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:43.303769  1572 rpcz_store.cc:276] 0709 13:02:42.119037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.014404 (+895367us) service_pool.cc:224] Handling call
0709 13:02:43.154454 (+140050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.155245 (+   791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.187439 (+ 32194us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54002a6cd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:43.187595 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321068928}
I20260709 13:02:43.156839  1642 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:43.239670  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 88730.9us
I20260709 13:02:43.249354  5624 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:43.155157  5726 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.167868  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1788) took 1047 ms. Trace:
I20260709 13:02:43.255463  1718 rpcz_store.cc:276] 0709 13:02:41.731017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.056025 (+325008us) service_pool.cc:224] Handling call
0709 13:02:42.604150 (+548125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.607720 (+  3570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.163252 (+555532us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400294ad0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:43.163417 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259748224}
I20260709 13:02:43.215833  5477 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:43.261730  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:43.120270  5728 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.051344  5705 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.275460  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:43.242764  5666 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.277376  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.133049  5711 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:02:43.237908  5508 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.230021  5235 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.279040  5710 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.139024  5493 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.261570  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.217244  5712 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.258991  5699 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.285405  5707 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.286474  1571 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:43.298106  5713 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:43.266554  5691 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:43.250229  5729 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:02:43.306594  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.311089  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:43.307556  1573 rpcz_store.cc:276] 0709 13:02:42.120632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.166790 (+1046158us) service_pool.cc:224] Handling call
0709 13:02:43.167170 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.167391 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.167431 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:43.167493 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:43.167541 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.167615 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.167787 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:43.321647  1469 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:43.317564  5477 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:43.267923  5636 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.332593  1570 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:43.339347  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64231.3us
I20260709 13:02:43.302201  5698 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:43.349560  5622 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:43.353704  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.404620  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.341069  5624 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:02:43.356153  5592 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:43.357336  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95845.4us
I20260709 13:02:43.334497  5666 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.375578  1646 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 6.
W20260709 13:02:43.380918  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.393505  1573 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.395903  1570 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
W20260709 13:02:43.399537  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.460240  1570 pending_rounds.cc:85] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 3
I20260709 13:02:43.460978  1570 pending_rounds.cc:107] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:43.461367  1570 raft_consensus.cc:2889] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:43.461815  1570 pending_rounds.cc:107] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:02:43.462108  1570 raft_consensus.cc:2889] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:43.397254  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:43.463392  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 66402us
I20260709 13:02:43.324153  5690 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:02:43.453845  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:43.400609  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 89663us
I20260709 13:02:43.467289  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:43.473009  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:43.488024  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.394435  5636 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.488802  5552 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.489420  5552 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.490839  5636 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:43.493296  5675 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.493923  5675 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.494306  5675 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.402842  5460 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.495751  5460 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.498019  5728 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.498967  5728 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.499456  5728 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.499940  5728 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.502269  5622 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.503373  1651 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:43.457719  5698 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:43.504887  5644 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.401858  5711 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.505820  5711 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.508226  1723 raft_consensus.cc:606] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:43.508782  1723 raft_consensus.cc:614] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:43.509600  5483 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:43.510057  5483 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:43.511155  5483 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.512375  5638 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.513042  5638 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.513453  5638 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.513759  1571 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:02:43.514220  1571 pending_rounds.cc:85] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:43.514562  1571 pending_rounds.cc:107] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:43.514853  1571 raft_consensus.cc:2889] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:43.516860  5644 consensus_queue.cc:1175] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f737473248e4c059fa74af3b8e1f77e" responder_term: 9 status { last_received { term: 8 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:43.520931  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.521101  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.528877  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:43.535321  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.378302  5518 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:43.392675  5713 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:43.522833  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.391767  5508 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.391268  5691 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.508265  5711 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.539091  5552 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.549125  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:43.498378  5460 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:43.550733  5483 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.552615  5690 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.558768  5726 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.458956  5641 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.398823  5699 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.472170  5592 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.564986  5644 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.393761  5477 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:43.578933  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:43.584038  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.633872  1646 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:43.349565  5211 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:43.594655  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 475805us
I20260709 13:02:43.643251  1576 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.615285  5698 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.623922  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 401486us
I20260709 13:02:43.624595  1642 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 13:02:43.625428  1651 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:43.393828  1469 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:43.609078  5518 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.627132  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:43.620983  1571 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:43.650699  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 406563us
I20260709 13:02:43.646317  1649 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:43.647480  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:43.618239  5749 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.638281  5715 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.654420  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:43.645709  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 339689us
I20260709 13:02:43.665190  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:43.625133  5690 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:43.683740  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:43.660902  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:43.687282  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:43.687757  5690 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.670368  5518 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.714550  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:43.717592  2373 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:02:43.731170  1724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.022s	sys 0.000s
W20260709 13:02:43.732450  1724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.022s	sys 0.000s
I20260709 13:02:43.685380  5697 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:43.681036  5708 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:43.705076  5688 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.711822  5636 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:43.756135  2374 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:43.643033  5644 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.720374  5624 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:43.719404  5496 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.012s	sys 0.009s
I20260709 13:02:43.719730  5627 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:43.768687  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:43.737306  5711 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:43.705677  5622 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:43.684578  5698 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.685246  5633 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.785228  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:43.761494  5688 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.764552  5636 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:43.794369  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.542s (SENT)
I20260709 13:02:43.768253  5547 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.768788  5697 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:43.736891  5514 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:43.823050  1646 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:43.816026  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:43.796044  5691 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:43.819844  1648 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:02:43.830353  1648 pending_rounds.cc:85] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 7
I20260709 13:02:43.830845  1648 pending_rounds.cc:107] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:02:43.831300  1648 raft_consensus.cc:2889] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:43.780095  5496 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.012s	sys 0.009s
I20260709 13:02:43.804024  5688 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.779749  5711 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:43.845973  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 323221us
I20260709 13:02:43.778651  5622 consensus_queue.cc:1243] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:43.848893  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2397) took 1794 ms. Trace:
I20260709 13:02:43.810887  5705 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:43.864904  5751 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:02:43.866590  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 526) took 1196 ms. Trace:
I20260709 13:02:43.867323  5675 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:43.868852  5750 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:02:43.870208  5697 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.872068  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2393) took 2446 ms. Trace:
W20260709 13:02:43.872974  5759 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:43.872589  1569 rpcz_store.cc:276] 0709 13:02:41.425565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.913473 (+1487908us) service_pool.cc:224] Handling call
0709 13:02:42.914046 (+   573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.961058 (+ 47012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.961103 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.968815 (+  7712us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.070029 (+101214us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b180015be60. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:02:43.460240 (+390211us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:43.460293 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.460396 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.871715 (+411319us) raft_consensus.cc:1685] finished
0709 13:02:43.871741 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.871983 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20757,"raft.queue_time_us":21720,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":32961,"spinlock_wait_cycles":15489408,"thread_start_us":389977,"threads_started":1,"wal-append.queue_time_us":631251}
I20260709 13:02:43.877799  5633 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:43.706960  5749 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.881479  5715 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:43.859531  5711 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.882005  5715 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.882499  5749 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.830514  5707 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:43.884696  5707 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:43.885769  5699 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.885718  5707 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.879673  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1800) took 1465 ms. Trace:
I20260709 13:02:43.888590  1576 rpcz_store.cc:276] 0709 13:02:42.414569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.281015 (+866446us) service_pool.cc:224] Handling call
0709 13:02:43.281413 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.643867 (+362454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.879406 (+235539us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54001711d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:43.879610 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831581568}
I20260709 13:02:43.857988  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:43.843932  5764 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.854861  5755 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:02:43.866990  1721 rpcz_store.cc:276] 0709 13:02:42.670161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.018860 (+348699us) service_pool.cc:224] Handling call
0709 13:02:43.019372 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.042178 (+ 22806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.042243 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.094045 (+ 51802us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.598328 (+504283us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:43.598401 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.598924 (+   523us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.866361 (+267437us) raft_consensus.cc:1685] finished
0709 13:02:43.866383 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.866528 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":324,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":12339,"thread_start_us":445324,"threads_started":1,"wal-append.queue_time_us":485507}
I20260709 13:02:43.818478  5636 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.861495  5666 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.774112  5708 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:43.855654  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:43.895416  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 212296us
W20260709 13:02:43.902621  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.856455  1574 rpcz_store.cc:276] 0709 13:02:42.056384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.994867 (+938483us) service_pool.cc:224] Handling call
0709 13:02:43.025131 (+ 30264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.848060 (+822929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.848449 (+   389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:43.848512 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:43.848556 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.848649 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.848816 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209353,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":19833,"thread_start_us":270491,"threads_started":1}
I20260709 13:02:43.861497  5641 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.886881  5698 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.885805  5644 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:43.864909  5211 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:43.893427  5536 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:43.934497  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.852253  5314 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:02:43.720892  5713 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:43.930763  5514 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:02:43.930968  5235 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:02:43.860245  1469 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 2 to 3, term changed from 7 to 8, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to <none>, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193), VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194), VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195), VOTER da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 8 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:43.965904  1469 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:43.974684  1723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:43.990173  2373 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:44.005918  5666 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:44.007429  5748 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.615s	user 0.003s	sys 0.001s
I20260709 13:02:44.008749  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:44.016420  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 107052us
I20260709 13:02:44.021770  2380 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:44.025703  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2390) took 2601 ms. Trace:
I20260709 13:02:44.026724  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.027011  1724 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:43.940864  5592 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.026108  1568 rpcz_store.cc:276] 0709 13:02:41.424600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.855880 (+1431280us) service_pool.cc:224] Handling call
0709 13:02:42.856233 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.882119 (+ 25886us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:42.882158 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.916823 (+ 34665us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.262516 (+345693us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:43.262563 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.262643 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.006830 (+744187us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:44.010050 (+  3220us) raft_consensus.cc:1685] finished
0709 13:02:44.010070 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.025642 (+ 15572us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9803,"raft.queue_time_us":23945,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":25974,"spinlock_wait_cycles":26310528,"thread_start_us":261335,"threads_started":1,"wal-append.queue_time_us":423771}
I20260709 13:02:44.028223  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.034479  1717 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:44.035658  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.028534  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.036856  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:44.030529  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:44.023758  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 301657us
I20260709 13:02:44.037160  1724 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:44.029614  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:44.040230  1718 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:43.943704  5726 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:44.064972  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:44.069777  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.070458  1719 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:44.070598  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.071864  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.075060  1716 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:44.076309  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.077317  1721 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:02:44.091400  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:44.092123  2380 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.956270  5622 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 13:02:44.095186  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:44.097750  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.121455  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.100725  5758 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:02:44.100586  5547 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.110878  5477 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.113324  5627 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:44.020481  5638 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.074157  5749 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.114573  5772 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.115569  5691 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:44.073127  5536 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:44.119585  5675 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:44.124969  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44809.2us
I20260709 13:02:44.025117  5666 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:02:44.114089  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1797) took 1723 ms. Trace:
W20260709 13:02:44.133344  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 446157us
I20260709 13:02:44.071719  5644 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:02:44.114612  5708 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:44.153505  1723 raft_consensus.cc:606] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:44.140724  5627 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:44.166618  1567 rpcz_store.cc:276] 0709 13:02:42.390331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.219312 (+828981us) service_pool.cc:224] Handling call
0709 13:02:43.219667 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.240611 (+ 20944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.240662 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.241417 (+   755us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.628756 (+387339us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:43.628804 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.628891 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.113888 (+484997us) raft_consensus.cc:1685] finished
0709 13:02:44.113909 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.114029 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20391,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":9986,"thread_start_us":351450,"threads_started":1,"wal-append.queue_time_us":798318}
I20260709 13:02:44.174726  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.181191  1723 raft_consensus.cc:614] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:44.142637  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 47527us
I20260709 13:02:44.197263  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.203641  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.130738  5547 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.209290  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.214800  2380 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:44.094098  5527 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 13:02:44.154253  5765 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.329s	user 0.002s	sys 0.000s
I20260709 13:02:44.185242  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:44.237528  1646 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:44.240262  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:44.247334  1649 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:44.153383  5477 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:44.248565  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 214783us
I20260709 13:02:44.248925  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.250068  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.264529  1576 pending_rounds.cc:85] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:44.150794  5642 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.146723  5235 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.201479  5743 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:44.146963  5698 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.265038  1576 pending_rounds.cc:107] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:02:44.277209  1576 raft_consensus.cc:2889] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:44.253731  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68574.5us
W20260709 13:02:44.254455  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 398894us
I20260709 13:02:44.256170  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:44.257948  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:44.141639  5726 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:02:44.141652  5536 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:44.281766  5536 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.264604  1567 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:44.151029  5653 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:44.283885  5653 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.284142  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 578) took 1097 ms. Trace:
I20260709 13:02:44.286139  5653 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.131680  5314 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:02:44.280066  1649 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:44.284703  1650 rpcz_store.cc:276] 0709 13:02:43.186666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.187495 (+   829us) service_pool.cc:224] Handling call
0709 13:02:43.187854 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.302693 (+114839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.302725 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.303284 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.686824 (+383540us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:43.686891 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.907777 (+220886us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000b9d54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:43.907839 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.153664 (+245825us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:44.283890 (+130226us) raft_consensus.cc:1685] finished
0709 13:02:44.283920 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.284075 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8492,"raft.queue_time_us":7108,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":2293,"spinlock_wait_cycles":274102400,"thread_start_us":337961,"threads_started":1,"wal-append.queue_time_us":500908}
I20260709 13:02:44.288844  1651 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:44.178972  5749 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:44.290134  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:44.240811  5714 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.297453  1650 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.151429  5211 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 13:02:44.248726  5493 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.277199  1573 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:02:44.141263  5690 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.300858  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:44.292634  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.296716  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:44.304327  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:44.178814  5755 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:02:44.309988  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:44.333814  1649 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.214977  5633 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:02:44.335388  1651 raft_consensus.cc:2376] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:02:44.338703  2380 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:44.311945  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.018s
W20260709 13:02:44.343022  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.019s	sys 0.019s
I20260709 13:02:44.345329  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:44.346818  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54267.5us
I20260709 13:02:44.159754  5772 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.298339  5777 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:44.339854  1650 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
W20260709 13:02:44.368585  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.372578  2380 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:44.290634  5642 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.378893  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.380354  1649 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:44.348737  5766 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:02:44.291715  5688 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:44.390447  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45212us
I20260709 13:02:44.302073  5749 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.290524  5235 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.364177  5624 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.397502  1725 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:44.396025  1568 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:44.333313  5493 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.353119  5786 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:44.410252  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:02:44.402274  5705 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 13:02:44.312037  5714 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.316470  5764 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
W20260709 13:02:44.414060  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 268058us
I20260709 13:02:44.370489  5777 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.353354  5666 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:02:44.426734  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:44.423835  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 245031us
I20260709 13:02:44.413419  5774 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:44.413841  5483 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.785s	user 0.009s	sys 0.003s
W20260709 13:02:44.424518  5712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.007s	sys 0.014s
W20260709 13:02:44.422101  5518 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.001s
I20260709 13:02:44.430138  5653 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:44.475709  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52953.7us
I20260709 13:02:44.437206  5777 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.449868  5786 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.469990  5712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.008s	sys 0.015s
I20260709 13:02:44.472819  5638 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:44.474740  5728 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.911s	user 0.013s	sys 0.000s
I20260709 13:02:44.473811  5622 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.524070  5514 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:44.524557  5714 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.524921  5666 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:44.449146  5755 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 13:02:44.500851  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74212us
I20260709 13:02:44.523085  5690 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.523381  5728 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.502475  5496 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.491173  5705 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:02:44.524835  5514 consensus_queue.cc:1243] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:44.525550  5784 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 13:02:44.523417  5781 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:44.475991  5518 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.511592  5786 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:44.545409  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.549198  5547 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.554842  5547 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.553047  5493 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.556232  5749 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.559437  5627 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.560423  5235 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:44.554984  5636 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.003s
I20260709 13:02:44.560858  5772 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.562429  5211 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:44.562330  5636 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.562849  5211 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:44.564396  5636 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:44.565011  5789 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:44.561620  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.566216  5772 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:44.566584  5772 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:44.568459  5772 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.569247  5786 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:44.571151  5728 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.571554  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2409) took 1736 ms. Trace:
I20260709 13:02:44.571905  1575 rpcz_store.cc:276] 0709 13:02:42.835146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.376449 (+541303us) service_pool.cc:224] Handling call
0709 13:02:43.451865 (+ 75416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.662310 (+210445us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:43.900809 (+238499us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.263655 (+362846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.263709 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:44.263758 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.571326 (+307568us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:44.571374 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.571499 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22618,"prepare.queue_time_us":442243,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":22761,"spinlock_wait_cycles":491749632,"thread_start_us":526130,"threads_started":2,"wal-append.queue_time_us":476007}
W20260709 13:02:44.574126  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.549868  5787 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 13:02:44.574767  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99099.3us
W20260709 13:02:44.583864  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56375.2us
I20260709 13:02:44.592749  5705 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:44.563781  5211 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.622442  5627 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.622716  5518 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.623474  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:44.623788  5788 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:02:44.622514  5769 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.531s	user 0.000s	sys 0.004s
I20260709 13:02:44.624723  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.625344  1574 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.628818  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:44.628006  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54787.2us
I20260709 13:02:44.629542  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:44.632644  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.632977  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.634888  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.596891  5314 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:44.616492  5642 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:44.645030  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.643733  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.653373  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 477) took 1390 ms. Trace:
I20260709 13:02:44.653625  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:44.654295  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.018s	sys 0.008s
W20260709 13:02:44.656116  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.018s	sys 0.008s
I20260709 13:02:44.663614  1574 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:44.643536  5518 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:44.655656  2371 rpcz_store.cc:276] 0709 13:02:43.262995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.263163 (+   168us) service_pool.cc:224] Handling call
0709 13:02:43.268792 (+  5629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.695559 (+426767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.695608 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:43.705020 (+  9412us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.156557 (+451537us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:44.156592 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.156658 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:44.653151 (+496493us) raft_consensus.cc:1685] finished
0709 13:02:44.653172 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.653309 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50781,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7233,"thread_start_us":827386,"threads_started":2,"wal-append.queue_time_us":634706}
I20260709 13:02:44.681579  5798 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:44.682638  5666 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:44.649559  5314 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:44.684808  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.687634  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.643874  5690 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:44.692057  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:44.700866  1724 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:44.701354  1717 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:44.656133  5815 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:44.665555  5211 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.628696  5705 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:44.702097  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58447.1us
I20260709 13:02:44.705564  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.710687  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:44.717232  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.670337  5712 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.699868  5749 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.721117  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.725953  1720 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:44.721711  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:44.722757  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:44.719897  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.723924  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2401) took 2119 ms. Trace:
I20260709 13:02:44.691764  5518 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:44.699767  5766 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.728231  1572 rpcz_store.cc:276] 0709 13:02:42.604769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.315887 (+711118us) service_pool.cc:224] Handling call
0709 13:02:43.317291 (+  1404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.631421 (+314130us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:44.015721 (+384300us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.366042 (+350321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.366109 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:44.366590 (+   481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.712153 (+345563us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b1800067a20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419
0709 13:02:44.723614 (+ 11461us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540027d154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:44.723676 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:44.723852 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4370,"prepare.queue_time_us":472912,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":39,"raft.queue_time_us":119,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12039,"spinlock_wait_cycles":727684736,"thread_start_us":719709,"threads_started":2,"wal-append.queue_time_us":524922}
I20260709 13:02:44.703871  5815 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.738078  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:44.739303  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.771106  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.725369  5493 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.722954  5690 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:44.773011  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.724996  5638 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:44.783924  1572 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:44.794416  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.768496  5786 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:44.777124  5726 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:44.790738  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.703540  5666 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.743515  5518 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.803393  5638 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.820477  5638 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:44.820534  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.821717  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.822471  1570 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:02:44.823385  1570 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:02:44.825392  1567 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:44.795914  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:44.768095  5802 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.797379  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:44.772960  5815 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:44.805335  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:44.863049  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57780us
W20260709 13:02:44.865932  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 199543us
I20260709 13:02:44.867108  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:44.805528  5786 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:44.807736  5536 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:44.874442  5810 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:02:44.874691  1576 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:02:44.874709  5786 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.875577  5496 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:44.876106  5496 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:44.877014  5496 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.809492  5705 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.782177  5314 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.878966  5641 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:44.879053  5314 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.879895  5641 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:44.859911  5715 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.944s	user 0.014s	sys 0.000s
I20260709 13:02:44.810828  5690 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.736217  5749 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.880762  5774 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:02:44.813383  5800 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:44.814716  5666 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:44.826843  1575 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.776104  5712 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.860926  1572 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:44.700220  5798 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.871549  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:44.939193  5798 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:44.804715  5812 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.805176  5726 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.939946  5726 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.939960  5812 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.943219  5812 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:44.826592  5713 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:44.882254  5641 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.812834  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:44.944504  5713 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:02:44.911309  5711 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.029s	user 0.011s	sys 0.000s
I20260709 13:02:44.944955  5713 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.945210  5711 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.898092  5749 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:44.880628  5715 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.916435  5819 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.944963  1720 raft_consensus.cc:606] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:44.948199  5819 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.948928  1720 raft_consensus.cc:614] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:44.950513  5819 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.788085  5766 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:44.950923  5715 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:44.952566  5766 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:44.871721  5707 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:44.953205  5707 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:44.953655  5707 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.903504  5518 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:44.956876  1570 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:44.957926  5518 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:44.926015  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.928524  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.928217  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57460.4us
W20260709 13:02:44.962909  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 4.567s (SENT)
W20260709 13:02:44.986497  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 120278us
W20260709 13:02:44.987706  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.990942  1575 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:02:44.991154  5748 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:45.009837  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.010581  2371 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:44.990866  5715 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:45.015595  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.015861  2373 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:45.014659  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.021999  1717 raft_consensus.cc:2368] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:45.022786  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.023343  1717 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:45.024109  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.025890  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.026407  1720 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:45.028661  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.029742  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:45.030321  2371 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.031644  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.032574  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.034487  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.035897  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:45.036551  2374 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.042241  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:45.043504  2374 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:45.044538  2373 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.047158  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:45.048451  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.048986  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.054653  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.005657  5705 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:02:45.060217  2373 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:45.061968  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.062546  2374 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:45.062722  2373 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:45.065855  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 103049us
I20260709 13:02:45.066763  1725 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:45.012375  5772 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.071831  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.088900  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.092242  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.093411  2372 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:45.095916  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.096827  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.098152  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.099154  1575 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:45.060760  5715 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:45.100394  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.055884  5786 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:45.102792  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.106316  1572 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.082967  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.097409  5638 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:45.141712  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.103461  5715 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.063786  5712 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:45.103942  5518 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:45.161882  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.096877  5690 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:02:45.150573  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.151971  5819 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:45.154462  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.102885  5774 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:02:45.156715  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.156955  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:45.158109  5825 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:02:45.158802  5831 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:45.159461  5726 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.150179  5715 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.163553  5772 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.164690  5712 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.169291  5688 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:45.171187  5514 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:45.148608  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.135138  5702 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:45.114135  5713 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.014810  5777 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.194141  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.192379  5710 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.176743  5820 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:02:45.199730  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:45.200798  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.227689  5702 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.228147  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.228412  5710 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.229718  1723 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:45.230476  5710 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.231004  5707 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.228216  5702 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.234004  5702 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.234052  5793 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:45.234613  5793 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:45.245445  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:45.178684  5536 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:02:45.178656  5798 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:02:45.219141  5712 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.249744  1573 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:45.235805  5211 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:02:45.257964  1718 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:45.243906  5666 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.272107  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.273028  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.275839  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:45.280634  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 183864us
I20260709 13:02:45.283628  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.284899  1645 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:45.256568  5712 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.289436  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.298031  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.268893  5802 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.296013  1724 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:45.306540  2371 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:45.308455  1646 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 6: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:45.305392  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.304831  1539 leader_election.cc:400] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:45.321118  1539 leader_election.cc:403] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:45.316702  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:45.322127  1574 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:45.304888  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.325524  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.326459  1643 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:45.323556  1642 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:45.325021  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:45.336906  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.339090  1650 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:45.341204  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.341952  1650 raft_consensus.cc:2368] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:45.342664  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.343186  1650 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:45.343947  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.345749  5755 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:45.348351  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.349013  1649 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:45.349912  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.351321  1649 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:45.344862  5749 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:45.352898  5749 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.355203  5749 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.331521  5653 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.356146  5653 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.357757  1650 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:45.361061  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:45.364606  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.364720  1722 raft_consensus.cc:2410] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:45.330240  5624 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.364722  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 224515us
I20260709 13:02:45.367420  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.371846  1642 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:45.287839  5666 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.327795  5496 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.375330  1646 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:45.362489  1574 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:45.377462  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2417) took 1721 ms. Trace:
I20260709 13:02:45.378017  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.379217  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.378049  1571 rpcz_store.cc:276] 0709 13:02:43.655825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.835778 (+179953us) service_pool.cc:224] Handling call
0709 13:02:43.836195 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.429707 (+593512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.429735 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:44.429775 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:44.429824 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.377261 (+947437us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54000cba54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:45.377302 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.377413 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21468,"raft.queue_time_us":8087,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":15406,"spinlock_wait_cycles":1346727680}
I20260709 13:02:45.383383  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.363340  5786 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:45.377351  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.392084  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:45.365286  5777 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.368448  5749 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:45.401999  1643 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:45.400947  5774 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:45.402884  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.330367  5802 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.367741  5638 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:45.433081  5836 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.434020  5836 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.434849  5836 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.435372  5836 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.437552  5844 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:45.405992  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 212982us
I20260709 13:02:45.305665  5793 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.439545  5793 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:45.439818  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.408752  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.438827  1642 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:45.374866  5624 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:45.458323  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 132884us
I20260709 13:02:45.407315  5726 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.406910  5837 raft_consensus.cc:2764] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:45.493417  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.387861  5496 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.497958  1568 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:45.445266  5702 consensus_queue.cc:1175] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f737473248e4c059fa74af3b8e1f77e" responder_term: 6 status { last_received { term: 6 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 6 index: 6 } }
W20260709 13:02:45.499711  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59965.7us
I20260709 13:02:45.502233  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:45.504806  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.505993  2371 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.408609  5749 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.506021  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:45.492463  5713 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.500200  5780 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:02:45.495055  5803 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:45.512792  5314 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.523839  2371 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:45.511195  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.504689  5726 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.527711  1645 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:45.503877  5707 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:45.514614  5755 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:45.536070  5755 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.536561  5755 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.526634  5536 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:45.526358  5852 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:45.542412  5836 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:45.542608  5641 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.549690  5496 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.551455  5749 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:45.553167  5815 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.000s
I20260709 13:02:45.553603  5815 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.555805  5711 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.556542  5726 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.531729  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.559847  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.533119  5708 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:45.533445  5802 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:45.535005  5624 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:45.560642  1646 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.561317  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.574733  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.565850  1642 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:45.508873  5786 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.531590  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:45.570538  2374 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:02:45.578900  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:45.576212  5852 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:45.508594  5688 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:02:45.598119  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.598749  1569 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:45.599606  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:45.601078  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.601754  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:45.602118  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.602873  1570 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:45.603539  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:45.578302  2371 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:45.603121  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.580773  2373 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:45.547737  5715 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.582338  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.581777  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:45.585191  5839 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:45.586656  5712 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.536616  5638 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.529495  5803 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.596792  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1625) took 1361 ms. Trace:
I20260709 13:02:45.628926  1648 rpcz_store.cc:276] 0709 13:02:44.235763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.236455 (+   692us) service_pool.cc:224] Handling call
0709 13:02:44.275121 (+ 38666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.432993 (+157872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.433023 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.449174 (+ 16151us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.698276 (+249102us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:44.698971 (+   695us) log.cc:844] Serialized 14 byte log entry
0709 13:02:44.699177 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.596423 (+897246us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54000910d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:45.596470 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.596514 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:45.596616 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.596726 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7672,"prepare.queue_time_us":211,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":35457,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":8640,"spinlock_wait_cycles":362143616,"thread_start_us":192674,"threads_started":1,"wal-append.queue_time_us":435914}
I20260709 13:02:45.596903  5666 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:45.569043  5653 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:45.605062  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.634732  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.606235  2378 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:02:45.565652  5514 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:45.638859  1569 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:45.527410  5314 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.607170  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.640156  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.614202  5701 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:45.616868  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 85388.2us
I20260709 13:02:45.645871  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.617849  5726 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.618000  5852 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.659677  5772 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:45.642765  5711 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.580508  5749 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.621292  1646 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:45.621779  1574 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:45.604727  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.566495  5708 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:45.632464  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.632493  5688 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:45.610559  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.648120  2372 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:45.672582  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.673733  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.674865  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.675612  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.656675  5802 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.643527  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.639081  5514 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:45.638293  1647 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:45.670197  2380 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:45.671473  1572 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:45.649188  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.652839  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.708761  2378 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:45.678290  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.685336  2371 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:45.663228  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:45.642735  5314 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.679893  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.712801  2375 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:45.723537  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2440) took 1091 ms. Trace:
I20260709 13:02:45.724234  1573 rpcz_store.cc:276] 0709 13:02:44.632023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.987932 (+355909us) service_pool.cc:224] Handling call
0709 13:02:44.989570 (+  1638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.256786 (+267216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.723235 (+466449us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540034fb54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:45.723460 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597790080}
I20260709 13:02:45.732988  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.733440  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:45.733935  2380 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:45.734653  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59123.7us
I20260709 13:02:45.735663  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.764074  5815 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:45.660326  5772 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:45.730794  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.783499  1646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:45.789357  1646 raft_consensus.cc:606] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:45.789842  1646 raft_consensus.cc:614] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:45.785873  1568 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:02:45.786199  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.771046  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.668322  5749 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:45.791695  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 110363us
I20260709 13:02:45.798712  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:45.792068  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 112304us
I20260709 13:02:45.789384  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:02:45.801816  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71118.9us
I20260709 13:02:45.677448  5849 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.810283  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.811439  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.814208  1539 leader_election.cc:400] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
W20260709 13:02:45.812052  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 76491.6us
I20260709 13:02:45.815217  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:45.815927  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:45.818542  1645 raft_consensus.cc:2410] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:02:45.822706  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:45.788411  5837 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:45.792464  5666 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.830549  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.792261  5688 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:45.793691  5460 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:45.829941  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:45.873533  1568 raft_consensus.cc:2410] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:02:45.857241  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.804327  5840 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:45.805747  5749 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:45.842150  5799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.000s	sys 0.025s
I20260709 13:02:45.791652  5713 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:45.868952  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.902482  1642 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:45.730593  5802 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.684737  5708 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.813146  5815 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:45.808445  5780 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:45.843711  5847 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.855059  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:45.893113  1717 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:45.819089  5536 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:45.878798  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:45.858937  5211 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:45.921962  5211 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.922407  5211 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:45.857558  5766 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:45.925280  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70334.1us
I20260709 13:02:45.925410  5766 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.925637  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:45.926014  5766 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.926093  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.881809  1649 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:45.928407  5815 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:45.857589  5786 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:45.929347  5815 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.930037  5786 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:45.896464  5799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.000s	sys 0.026s
I20260709 13:02:45.918473  5803 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:45.934237  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 201360us
I20260709 13:02:45.911460  1574 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:45.818379  5772 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.908864  5653 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:45.910198  5460 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:45.910614  5819 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:45.903218  5624 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:45.818604  5849 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.843282  5843 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:02:45.913874  5842 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:45.963518  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 177396us
I20260709 13:02:45.914378  5844 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:45.933765  5708 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.871634  5837 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.871201  5514 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:45.964254  5653 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.988510  5840 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:45.990698  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:45.984876  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 184689us
I20260709 13:02:45.977788  5772 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:45.979703  5713 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.972537  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:45.982697  5715 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:45.931944  5847 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.001742  5857 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:02:46.005215  5858 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:46.005903  5857 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:45.984596  5844 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.006290  5857 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:46.006438  5844 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:46.007208  5857 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.008942  5871 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:46.009512  5871 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.010409  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:46.012058  1570 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:46.011826  5726 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:46.012512  1570 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:46.013100  5726 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:46.013563  5726 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:46.006125  5847 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.006498  5858 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:46.014421  5726 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.015138  5858 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:46.013224  1570 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.015688  5726 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:46.015976  5858 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.016526  1570 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:46.017192  5858 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.017916  5849 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:45.989796  5843 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:45.996325  5854 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 13:02:46.000365  5836 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:46.000744  5514 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.036899  5836 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.037503  5836 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.999219  5713 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:46.039417  5713 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:46.046396  5832 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.887s	user 0.000s	sys 0.009s
I20260709 13:02:46.047652  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2433) took 1436 ms. Trace:
W20260709 13:02:46.048059  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.048004  1576 rpcz_store.cc:276] 0709 13:02:44.610793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.875587 (+264794us) service_pool.cc:224] Handling call
0709 13:02:44.875895 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.876092 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.876115 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.881904 (+  5789us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.101200 (+219296us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:45.101240 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.101306 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.045782 (+944476us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:46.047351 (+  1569us) raft_consensus.cc:1685] finished
0709 13:02:46.047372 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.047590 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47353216,"thread_start_us":183824,"threads_started":1,"wal-append.queue_time_us":242149}
W20260709 13:02:46.049562  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.050819  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2183) took 1032 ms. Trace:
I20260709 13:02:46.051247  1717 rpcz_store.cc:276] 0709 13:02:45.018349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.023896 (+  5547us) service_pool.cc:224] Handling call
0709 13:02:46.050632 (+1026736us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54003dc2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:46.050757 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623155456}
I20260709 13:02:46.054724  5786 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:46.055619  5855 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 13:02:46.057885  5837 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.011677  5871 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:46.061328  5793 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.000s
I20260709 13:02:45.980196  5624 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.062218  5812 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:46.062795  5812 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:46.022284  5875 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.065688  5875 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.067868  5875 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:46.068030  5864 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:02:45.994602  5653 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.024552  5712 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.026436  1570 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:02:46.027084  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.990095  5688 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:45.991422  5842 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:46.022588  5802 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.081344  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.084213  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.120551  1724 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:46.120894  1570 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:46.122071  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.123169  1570 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:46.124037  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.087787  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.125941  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:46.102960  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 130522us
I20260709 13:02:46.063905  5624 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.167837  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e69630519d3c48eca350b04183abca32"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:46.168776  1720 raft_consensus.cc:606] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:46.169127  1720 raft_consensus.cc:614] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:46.168430  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49073.1us
I20260709 13:02:46.137874  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.170418  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.171612  1722 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:46.132728  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.129356  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.143016  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.173211  1717 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:46.139432  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.137596  5641 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.068648  5857 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:46.122299  5755 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.001s
I20260709 13:02:46.172643  1724 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:02:46.124764  5835 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:02:46.176573  1576 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:46.182879  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.183503  1723 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
I20260709 13:02:46.179957  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.184616  1724 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.139781  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.137276  5536 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:46.170799  5858 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:46.199088  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 262884us
I20260709 13:02:46.187430  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.063966  5812 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.199750  1716 raft_consensus.cc:2410] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:46.124559  5849 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:46.121353  5715 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:46.174059  5871 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:02:46.201220  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.203593  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.140117  5798 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.208050  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.160380  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.212327  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.220549  1717 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:46.174549  5712 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:02:46.231441  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71224.2us
I20260709 13:02:46.172031  5211 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.232832  1724 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:46.239226  5701 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.239737  5701 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.240247  5211 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.205925  5536 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.242830  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:46.264909  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.179873  5641 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:46.270594  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 447992us
W20260709 13:02:46.248781  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 418313us
I20260709 13:02:46.246945  1716 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:46.271162  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.273986  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.174386  5840 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.214214  5812 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.279057  2376 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:46.279872  2373 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:46.280485  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:46.127674  5314 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.184926  5755 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.232986  5842 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.244571  5638 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.208745  5849 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.203783  5858 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.287496  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:46.177469  5836 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:46.294430  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.295058  5211 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:46.230105  5766 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.247076  5798 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:46.300122  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.302095  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:46.304250  5211 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:46.316351  5211 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:46.286614  5872 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.331557  1651 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:02:46.286566  5870 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:46.311594  2375 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:02:46.313313  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75108.9us
I20260709 13:02:46.339733  1645 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:46.296880  5624 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:46.317828  5879 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:46.321631  5641 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.326756  5849 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.332270  1646 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:46.349349  1649 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:46.351445  5211 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.346514  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:46.353127  5211 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:46.356019  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.357203  1642 raft_consensus.cc:2410] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:46.363461  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:46.308578  5749 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.369544  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:46.372664  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.379721  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.380262  1647 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:46.383029  1647 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:46.383285  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:46.384119  2373 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:46.387035  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:46.387864  2380 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.390956  1650 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:46.389976  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.397121  1647 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:46.391777  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.355219  5460 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:46.343781  5883 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:46.316761  5713 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.403688  2372 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:02:46.368026  5711 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.704s	user 0.005s	sys 0.007s
W20260709 13:02:46.404923  5518 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.009s	sys 0.015s
I20260709 13:02:46.406003  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.374151  5849 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:46.341634  5872 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.377560  5705 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:46.410326  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.369561  5882 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:46.426102  5518 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.265s	user 0.010s	sys 0.015s
I20260709 13:02:46.435222  5858 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.435683  5518 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:46.354454  5847 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:02:46.378059  5712 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:46.355662  5514 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:02:46.432495  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.444561  1646 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:46.432745  2373 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:46.359710  5624 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.428197  5701 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.421985  5869 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:46.359330  5844 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.401712  5841 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:46.399444  5772 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:46.423663  5711 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.373996  5715 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:02:46.443136  1574 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:46.343906  5870 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.435016  5882 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:46.466032  5878 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:02:46.465813  5518 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:46.466679  5881 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:02:46.447744  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.417014  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:46.426856  5849 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.440534  2380 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:46.462294  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:46.472641  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43969.8us
I20260709 13:02:46.474310  1648 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:46.442107  5837 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.479940  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:46.481967  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.483011  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.489758  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:46.492463  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.498663  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.458053  5892 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.502228  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.503547  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.469866  5314 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.479588  5849 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:46.512840  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.506341  5766 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.496091  5799 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:46.521621  5786 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:46.523885  5891 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.525269  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:46.528165  1648 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.509266  5879 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:46.508772  5836 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.484827  5653 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.567869  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.568780  1568 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:02:46.569664  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.569978  5653 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.570309  1574 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:46.571034  1724 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:46.571467  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.497210  5755 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.528529  5817 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:46.530306  5893 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:02:46.498026  5837 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.470019  5841 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.530678  5874 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.530582  5897 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:46.531155  5895 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:46.532573  5887 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:46.532691  5870 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:46.533335  5885 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:02:46.522374  5708 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.007s
I20260709 13:02:46.493340  5477 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:46.526700  5876 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:46.543016  5624 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.501011  5892 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.494235  5514 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:46.550657  5883 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:46.623246  5883 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:46.509584  5712 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.624605  5892 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.625741  5514 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:46.625151  5712 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.626202  5514 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:46.627442  1720 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:02:46.627135  5514 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.552371  5844 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.524075  5690 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.553102  5798 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.524550  5688 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.565132  5713 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.566514  5872 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.578081  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.586697  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:46.639160  1568 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:46.639547  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 52994.8us
I20260709 13:02:46.589275  2376 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:46.640125  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.588615  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:46.641358  1570 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.526134  5799 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:46.592566  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66392.7us
I20260709 13:02:46.593956  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.650410  1716 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.595824  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:46.598212  1724 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:02:46.603839  1648 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:02:46.606415  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.616364  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.577476  5817 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:46.592239  5895 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.653636  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:46.653918  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:46.628743  5514 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.655622  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:46.657799  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.666296  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:46.666993  1576 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.604609  5707 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:02:46.663570  1568 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:46.672849  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.664438  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.669166  1720 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.5-10.5]
W20260709 13:02:46.677083  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.628268  5712 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.667819  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:46.686388  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:46.689687  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.690989  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:46.667824  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:46.693049  1570 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:46.701674  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:46.704210  1716 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:46.706194  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:46.656199  5799 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:46.712983  5211 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:46.715873  1570 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:02:46.717823  1572 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:46.707168  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:46.724856  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.727674  1570 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:46.654999  5836 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:46.663167  5844 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:46.624658  5883 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.662858  5872 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:46.733518  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:46.729128  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:46.735638  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:46.655917  5895 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.746246  5211 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:46.637904  5690 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.672459  5624 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:46.640861  5688 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.736776  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.752204  1576 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:02:46.752435  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 257797us
I20260709 13:02:46.766822  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:46.767443  1648 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.754398  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:46.759392  2376 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:46.711160  5876 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.692598  5713 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:02:46.737051  5772 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:46.737324  5715 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:46.752199  1716 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:46.733502  5879 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:02:46.764961  5710 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.961556  5710 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.960165  5835 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.964169  5710 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:46.965802  5902 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
I20260709 13:02:46.964706  5835 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.959604  5690 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:46.875197  5812 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.007s
I20260709 13:02:46.969223  1725 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:46.970487  5835 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.756740  5817 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.972124  5817 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.972777  5841 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:46.973109  5710 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:46.756527  5819 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.223s
I20260709 13:02:46.975914  5688 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:46.767634  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:46.966768  5901 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:46.978794  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:46.819505  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:46.939857  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1639) took 1669 ms. Trace:
W20260709 13:02:46.877487  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.940606  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 441) took 1869 ms. Trace:
I20260709 13:02:46.985966  5211 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:46.986454  1569 rpcz_store.cc:276] 0709 13:02:45.071367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.606905 (+535538us) service_pool.cc:224] Handling call
0709 13:02:46.940417 (+1333512us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b54000262d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:46.940548 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7222016}
W20260709 13:02:46.864512  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.782974  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:46.989406  2372 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.983846  1648 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:46.732371  5886 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:46.768620  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:46.757699  5726 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.009s
W20260709 13:02:46.895763  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.813s (SENT)
I20260709 13:02:46.998059  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:46.939517  5705 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:02:46.998963  1648 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
W20260709 13:02:46.933218  5860 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.950s	user 0.003s	sys 0.000s
W20260709 13:02:47.000743  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 266137us
I20260709 13:02:46.993258  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:47.008841  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 439) took 1938 ms (client timeout 1813 ms). Trace:
W20260709 13:02:47.008986  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 240507us
W20260709 13:02:47.009275  1575 rpcz_store.cc:269] 0709 13:02:45.070511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.142661 (+ 72150us) service_pool.cc:224] Handling call
0709 13:02:47.008634 (+1865973us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540027d654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:47.008767 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463050112}
I20260709 13:02:47.007128  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:46.998701  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:47.001618  1570 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:46.978294  5857 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.798s	user 0.017s	sys 0.000s
I20260709 13:02:47.011585  5871 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:47.011788  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:47.012504  5894 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:02:47.012946  1576 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:47.014038  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 118345us
I20260709 13:02:47.017876  5644 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.018743  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:46.819984  5698 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:46.990089  5883 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:47.019356  2380 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:47.019840  5904 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:47.020383  5904 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:47.021837  5904 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:46.761116  5836 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.025486  5777 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:46.916234  5872 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:47.026592  1646 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:47.008778  5766 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.762463  5624 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:47.029311  5624 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:47.029202  5766 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.026028  5836 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.032701  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:47.018395  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:47.035094  2379 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:47.018436  5644 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.029837  5749 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:47.038260  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:47.039013  1648 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.028469  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.047290  2372 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:46.983649  1644 rpcz_store.cc:276] 0709 13:02:45.270165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.271701 (+  1536us) service_pool.cc:224] Handling call
0709 13:02:45.299965 (+ 28264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.838599 (+538634us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:45.882171 (+ 43572us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.187506 (+305335us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.188089 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.188300 (+   211us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:46.188373 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.939326 (+750953us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540005c3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:46.939390 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.939451 (+    61us) raft_consensus.cc:1685] finished
0709 13:02:46.939572 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.939773 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66058,"prepare.queue_time_us":31038,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":41,"raft.queue_time_us":136335,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":40837,"spinlock_wait_cycles":365350784,"thread_start_us":595421,"threads_started":2,"wal-append.queue_time_us":399004}
I20260709 13:02:47.004814  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:46.759347  5460 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.250s
W20260709 13:02:47.036863  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 303440us
I20260709 13:02:46.854928  5641 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.189s
I20260709 13:02:47.002835  5772 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:46.998791  5715 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:47.010751  5705 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:47.070465  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.072315  1721 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:47.088120  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:47.098439  1648 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:47.102027  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:47.106878  1721 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.108063  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:47.097649  5886 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:47.109046  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 579) took 1468 ms. Trace:
I20260709 13:02:47.109313  5886 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.109447  2374 rpcz_store.cc:276] 0709 13:02:45.641011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.641430 (+   419us) service_pool.cc:224] Handling call
0709 13:02:45.641862 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.642293 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.642321 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.649164 (+  6843us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.794953 (+145789us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:45.794999 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.795078 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.932492 (+1137414us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:47.108786 (+176294us) raft_consensus.cc:1685] finished
0709 13:02:47.108806 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.108968 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1212,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":19551,"spinlock_wait_cycles":13975424,"thread_start_us":145279,"threads_started":1,"wal-append.queue_time_us":257913}
W20260709 13:02:47.111613  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 435) took 2057 ms (client timeout 2214 ms). Trace:
W20260709 13:02:47.112006  1567 rpcz_store.cc:269] 0709 13:02:45.053957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.289177 (+235220us) service_pool.cc:224] Handling call
0709 13:02:47.111444 (+1822267us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54000c1f54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:47.111561 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907937664}
W20260709 13:02:47.117455  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 109297us
W20260709 13:02:47.120782  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.122256  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:47.122993  1725 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:47.051730  5819 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.126236  1568 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:47.012157  5871 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.129766  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.102080  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.136299   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:47.137128  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:47.137655  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.138062   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:02:47.133090  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 148146us
I20260709 13:02:47.138612  1717 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:47.138396  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:47.140123  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.135615  1646 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 7.
I20260709 13:02:47.141283  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:47.141436  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.142306  1721 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.116604  5883 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:47.133569  5901 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:02:47.026028  5777 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.063154  5799 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.152652  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:47.156905  1725 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:02:47.157482  1646 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.159796  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.163733  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.165903  1725 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:47.144696  5701 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:47.172577  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:47.146380  5835 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:47.151875  5883 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:47.130504  5872 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:47.130314  5688 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:02:47.177047  5914 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.106966  5772 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.174448  5477 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:47.191557  1572 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.184748  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.194523  1647 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:47.207016  2379 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:47.207240  1724 raft_consensus.cc:2368] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:47.184245  5777 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.219333  1723 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:47.199186  5705 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.214274  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:47.218770  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80440.4us
I20260709 13:02:47.187096  5871 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:47.189316  5910 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:47.226960  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.229399  1646 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.191143  5638 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:47.222525  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.230825  5916 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:47.231391  5916 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:47.220860  5514 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.005s
I20260709 13:02:47.220423  5836 raft_consensus.cc:2764] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:02:47.276029  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.895s (SENT)
I20260709 13:02:47.276557  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:47.275317  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.279333  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 126795us
I20260709 13:02:47.284847  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:47.288555  1644 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: [6.4-9.6]
I20260709 13:02:47.290952  1644 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [10.4-11.5]
I20260709 13:02:47.296969  1719 raft_consensus.cc:2410] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:47.211692  5701 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.225099  5920 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.185104  5715 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.212493  5835 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:47.277236  5874 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.657s	user 0.014s	sys 0.004s
I20260709 13:02:47.299192  5847 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.351418  1722 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:47.357645  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 349709us
I20260709 13:02:47.228253  5705 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.312305  5793 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:47.316249  5644 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.255133  5638 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.356751  5874 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:47.380895  5835 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:47.385458  5895 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.003s
I20260709 13:02:47.386219  5895 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.385434  5705 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:47.387740  1645 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:47.388789  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:47.389797  5895 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.393595  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:47.402316  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.358155  5928 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:47.407521  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:47.411058  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 104024us
I20260709 13:02:47.413976  1648 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.413363  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:47.416302  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:47.417243  2379 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:47.359916  5847 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.423432  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:47.317296  5920 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.444868  5920 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.425470  5793 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:47.426815  5928 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:47.446276  5928 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:47.429678  5923 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:02:47.433885  1717 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:47.428974  5921 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:02:47.436755  5705 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:47.448774  5705 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.449383  5624 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:47.451268  5911 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.522s	user 0.000s	sys 0.005s
I20260709 13:02:47.451648  5705 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.452379  5928 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:47.425990  5849 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.903s	user 0.003s	sys 0.010s
I20260709 13:02:47.438561  5701 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.440253  5841 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:47.441941  5847 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.446348  5793 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:47.425853  5799 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.437036  5638 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:47.475957  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:47.498135  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.490559  5841 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:47.487908  5710 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:02:47.521137  5698 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:47.523820  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.495864  5849 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.533614  5698 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.542532  1643 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:47.519975  5837 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:47.545825  1720 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:47.542886  5793 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.546686  5841 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.597313  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1653) took 1225 ms. Trace:
I20260709 13:02:47.602200  1649 rpcz_store.cc:276] 0709 13:02:46.372142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.372303 (+   161us) service_pool.cc:224] Handling call
0709 13:02:46.399696 (+ 27393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.468836 (+ 69140us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:46.493438 (+ 24602us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.765696 (+272258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.765750 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:46.765793 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.597101 (+831308us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b5400411ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:47.597141 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.597258 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68924,"prepare.queue_time_us":28886,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":106811,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":53173,"spinlock_wait_cycles":158494976,"thread_start_us":253724,"threads_started":1,"wal-append.queue_time_us":483417}
I20260709 13:02:47.589335  5849 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.594940  5841 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:47.608431  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1652) took 1242 ms. Trace:
I20260709 13:02:47.613116  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:47.612262  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:47.613299  1642 rpcz_store.cc:276] 0709 13:02:46.366394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.366542 (+   148us) service_pool.cc:224] Handling call
0709 13:02:46.400017 (+ 33475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.494018 (+ 94001us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:46.513601 (+ 19583us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.708142 (+194541us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.708195 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:46.708242 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.608163 (+899921us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b540031bad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:47.608212 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.608371 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101688,"prepare.queue_time_us":50277,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":27892,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14315,"spinlock_wait_cycles":215668480,"thread_start_us":191486,"threads_started":1,"wal-append.queue_time_us":243390}
I20260709 13:02:47.616998  2376 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:47.611372  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:47.618109  1644 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.618965  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:47.589800  5698 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.601508  5624 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:47.621878  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.642028  1642 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:47.622187  5211 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:47.622648  5935 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:02:47.623091  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.630807  1644 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:47.618109  5798 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.643427  5914 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:47.643473  5817 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:47.635757  5928 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:47.656298  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.657172  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:47.663969  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:47.665607  5211 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:47.666054  5211 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:47.666474  5211 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.666368  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:47.660468  5847 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:47.669399  5872 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:47.693193  2377 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:47.693749  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 545) took 1610 ms. Trace:
I20260709 13:02:47.695240  2379 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:02:47.683015  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:47.697116  2376 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:47.669267  5925 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:47.666352  5641 raft_consensus.cc:2764] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:47.669658  5477 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:47.674055  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:47.704428  5460 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:47.685279  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.685362  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.705473  1651 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:47.706462  5799 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:47.706892  5799 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.704828  5460 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:47.707356  5799 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.708040  5460 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.686563  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:47.710242  5710 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:47.694157  1718 rpcz_store.cc:276] 0709 13:02:46.083403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.083550 (+   147us) service_pool.cc:224] Handling call
0709 13:02:46.137869 (+ 54319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.282175 (+144306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.282208 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.282251 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:46.282283 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.693489 (+1411206us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400086e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:47.693543 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.693689 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331465600}
I20260709 13:02:47.698807  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.703311  5872 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:47.712313  5698 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:47.712710  5872 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.663389  5798 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.715828  5798 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.716181  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.663615  5817 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.718406  5817 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.662108  5914 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.715864  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:47.722036  2377 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:47.723918  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.725402  5211 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.729668  2377 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:47.731537  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.733067  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:47.735419  1643 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:47.737970  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2467) took 1169 ms. Trace:
I20260709 13:02:47.738231  5702 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:47.738401  1574 rpcz_store.cc:276] 0709 13:02:46.568325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.585607 (+ 17282us) service_pool.cc:224] Handling call
0709 13:02:46.585960 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.879365 (+293405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.879413 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.879477 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:46.879523 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.737761 (+858238us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54002b0054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:47.737805 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.737907 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673763200}
I20260709 13:02:47.739315  5710 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:47.739732  5710 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:47.694542  5847 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.740073  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:47.747699  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.748667  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.749706  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.750125  1575 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:47.750684  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:47.753082  1723 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:47.753796  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 342022us
I20260709 13:02:47.754308  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.755817  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:47.756013  2376 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:47.740152  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:47.757110  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:47.757678  1569 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:47.741088  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 606) took 1164 ms. Trace:
I20260709 13:02:47.758725  1567 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:47.758904  1645 rpcz_store.cc:276] 0709 13:02:46.576570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.667534 (+ 90964us) service_pool.cc:224] Handling call
0709 13:02:47.740851 (+1073317us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54001bc1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:47.741016 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614114304}
I20260709 13:02:47.742238  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2475) took 1087 ms. Trace:
I20260709 13:02:47.760401  1568 rpcz_store.cc:276] 0709 13:02:46.655134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.672532 (+ 17398us) service_pool.cc:224] Handling call
0709 13:02:47.742096 (+1069564us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540030a054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:47.742187 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040902912}
I20260709 13:02:47.744607  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:47.762888  5211 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.740777  5710 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.776270  5710 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.740432  5514 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:47.740729  5924 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:02:47.780889  5514 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:47.782555  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.782996  5514 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.744381  5847 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.783535  5688 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:47.783821  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.741683  5925 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:47.784854  5925 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:47.784171  5688 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:47.785825  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.785667  5925 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:47.787597  1723 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:47.787133  5817 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:47.786613  5688 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:47.760356  2377 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:47.788817  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:47.760864  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.794193  1723 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:47.795147  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.795982  1723 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:47.796911  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.791574  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:47.790347  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:47.799567  1723 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:47.742720  5914 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:47.790885  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:47.942063  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 151267us
I20260709 13:02:47.809602  1721 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:02:47.945565  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:47.947096  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:47.899140  5948 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:02:47.792668  5711 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:47.898375  5688 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:47.789938  5798 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:47.795754  5925 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.909461  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:47.834124  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.947935  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:47.977301  1723 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:47.978680  1721 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:02:47.983342  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.983549  5914 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.001740  5914 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.005951  1721 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:02:48.006690  1721 pending_rounds.cc:85] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 8
I20260709 13:02:48.007062  1721 pending_rounds.cc:107] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:48.007488  1721 raft_consensus.cc:2889] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:48.009855  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 319344us
I20260709 13:02:47.944051  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:48.020853  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 76927.3us
I20260709 13:02:48.022485  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:47.974100  5698 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:48.070113  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:47.982241  5927 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:02:48.095062  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.169649  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 147301us
I20260709 13:02:48.045797  5711 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.173089  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:48.062048  5772 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.211199  5817 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.994849  5798 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.213397  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:48.179039  5841 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.554s	user 0.000s	sys 0.009s
I20260709 13:02:48.068555  5924 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:48.222692  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.177292  5496 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:48.238139  5947 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.449s	user 0.001s	sys 0.000s
I20260709 13:02:48.170979  5927 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.256744  5914 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:48.262697  5798 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:48.102514  5708 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 13:02:48.222651  5856 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:48.264586  5698 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.277952  5701 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:48.342160  5496 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.356745  5496 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:48.267153  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53834.4us
I20260709 13:02:48.178185  5712 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 13:02:48.356065  5701 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.392943  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1655) took 1992 ms. Trace:
I20260709 13:02:48.393697  1650 rpcz_store.cc:276] 0709 13:02:46.400142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.400363 (+   221us) service_pool.cc:224] Handling call
0709 13:02:46.400927 (+   564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.472277 (+ 71350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.472309 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.472369 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:46.472403 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.392458 (+1920055us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:48.392554 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.392836 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164267008}
I20260709 13:02:48.228085  5514 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:48.363443  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:48.289057  5841 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.351086  5710 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.002s
I20260709 13:02:48.265379  5817 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.415037  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2346) took 1290 ms. Trace:
I20260709 13:02:48.419045  1719 rpcz_store.cc:276] 0709 13:02:47.125062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.125189 (+   127us) service_pool.cc:224] Handling call
0709 13:02:48.414612 (+1289423us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400372b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:48.414918 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312582016}
I20260709 13:02:48.343007  5698 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:48.360318  5496 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.232131  5772 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.285035  5924 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.461694  5916 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:48.463734  5924 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.400318  5701 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:48.438824  5708 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:48.417948  5837 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:48.439268  5698 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.371541  5856 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.422024  5841 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.467288  5882 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.470326  5708 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:48.409042  5514 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.477754  5837 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.544029  5817 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:48.612370  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.613363  1572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.019s
W20260709 13:02:48.705353  1572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.001s	sys 0.020s
I20260709 13:02:48.706207  5882 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.706789  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2483) took 1052 ms. Trace:
I20260709 13:02:48.614082   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:48.614377  1571 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 442) took 3233 ms (client timeout 1895 ms). Trace:
I20260709 13:02:48.706036  5708 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:48.631969  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.005s	sys 0.005s
W20260709 13:02:48.634413  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.637463  2343 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.013s	sys 0.012s
W20260709 13:02:48.637516  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.013s	sys 0.022s
W20260709 13:02:48.641987  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.013s	sys 0.041s
I20260709 13:02:48.566133  5856 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:48.547078  5927 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:48.716167  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.013s	sys 0.041s
W20260709 13:02:48.559965  5956 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:02:48.551702  5962 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:48.718465  5962 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:48.720521  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.014s	sys 0.022s
W20260709 13:02:48.721549  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 667422us
I20260709 13:02:48.720834  5962 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.663328  5514 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:48.518052  5536 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:48.680874  5950 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:48.682753  5841 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.684230  5802 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:48.690900  5951 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.159s	user 0.002s	sys 0.000s
W20260709 13:02:48.692152  5970 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.117s	user 0.002s	sys 0.000s
I20260709 13:02:48.685912  5837 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.744277  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 585) took 1726 ms. Trace:
I20260709 13:02:48.700551  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2484) took 1045 ms. Trace:
W20260709 13:02:48.708137  1571 rpcz_store.cc:269] 0709 13:02:45.380859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.383065 (+  2206us) service_pool.cc:224] Handling call
0709 13:02:48.613542 (+3230477us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002a6cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:48.614061 (+   519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3168090496}
W20260709 13:02:48.628286  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 264814us
I20260709 13:02:48.745302  1573 rpcz_store.cc:276] 0709 13:02:47.654558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.743592 (+ 89034us) service_pool.cc:224] Handling call
0709 13:02:47.743877 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.744049 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:47.906159 (+162110us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.654672 (+748513us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.670188 (+ 15516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.671902 (+  1714us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:48.699909 (+ 28007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.700013 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.700271 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6757,"prepare.queue_time_us":343807,"prepare.run_cpu_time_us":163,"prepare.run_wall_time_us":518,"raft.queue_time_us":172172,"raft.run_cpu_time_us":717,"raft.run_wall_time_us":38710,"spinlock_wait_cycles":30336,"thread_start_us":905711,"threads_started":3,"wal-append.queue_time_us":584459}
I20260709 13:02:48.712226  5772 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:48.714994  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.006s	sys 0.005s
I20260709 13:02:48.652417  5967 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.724670  2343 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.013s	sys 0.013s
I20260709 13:02:48.727240  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1659) took 2296 ms. Trace:
W20260709 13:02:48.727907  5799 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.020s	user 0.015s	sys 0.015s
W20260709 13:02:48.738255  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.738432  5916 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.755452  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 679886us
I20260709 13:02:48.756284  5916 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.755455  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:48.758167  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:48.759336  1614 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:48.759698  5916 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:48.707775  5882 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.756606  5772 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.752470  1648 rpcz_store.cc:276] 0709 13:02:46.430359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.132062 (+701703us) service_pool.cc:224] Handling call
0709 13:02:47.132445 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.414420 (+281975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.726085 (+1311665us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54001bbcd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:48.726780 (+   695us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646873472}
I20260709 13:02:48.707373  1572 rpcz_store.cc:276] 0709 13:02:47.654275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.743584 (+ 89309us) service_pool.cc:224] Handling call
0709 13:02:47.743877 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.744046 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:48.097020 (+352974us) log.cc:844] Serialized 14 byte log entry
0709 13:02:48.706122 (+609102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.706221 (+    99us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:48.706384 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.706505 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.706727 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11882,"prepare.queue_time_us":530956,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":66,"raft.queue_time_us":6397,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":229,"spinlock_wait_cycles":41344,"thread_start_us":917922,"threads_started":2,"wal-append.queue_time_us":413354}
I20260709 13:02:48.768044  5962 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:48.769693  5802 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:48.776903  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.776696  5837 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:48.781488  5972 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:48.781756  5772 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.784443  5972 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:48.779196  5802 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.765918  5841 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:48.789016  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:48.792984  5950 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.793269  5817 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:48.763187  5799 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.791880  5802 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:48.795017  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:48.807571  5849 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.809020  5817 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.744971  2378 rpcz_store.cc:276] 0709 13:02:47.017104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.017231 (+   127us) service_pool.cc:224] Handling call
0709 13:02:47.017548 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.032946 (+ 15398us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:47.033532 (+   586us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.147889 (+114357us) log.cc:844] Serialized 14 byte log entry
0709 13:02:47.550874 (+402985us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b180040e5e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:47.700300 (+149426us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b180040e5e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287
0709 13:02:47.700468 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.700544 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:47.700607 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.743376 (+1042769us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b540011f8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:48.743467 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.743802 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8531,"prepare.queue_time_us":356,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":103184,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":3047,"spinlock_wait_cycles":73025536,"thread_start_us":206272,"threads_started":2,"wal-append.queue_time_us":192503,"wal-append.run_cpu_time_us":2544,"wal-append.run_wall_time_us":1216766}
I20260709 13:02:48.810032  5913 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:48.820218  5913 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:48.822628  5766 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:48.821676  5841 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:48.824076  5927 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:48.824941  5841 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.809794  5849 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.791402  5967 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.825132  5927 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.817165  5882 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.832258  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2209) took 1902 ms. Trace:
I20260709 13:02:48.831637  5972 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.833644  5708 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:48.833453  5934 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.865280  5934 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.829938  5837 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:48.869310  5837 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.873884  5961 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.875969  5961 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.840302  5914 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.878979  5914 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.882004  5914 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:48.840462  5536 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.839397  5883 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:48.841919  5849 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:48.842322  5967 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:48.843750  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.843909  5817 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:48.845340  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87509.4us
I20260709 13:02:48.897718  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:48.898829  5967 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:48.899408  5967 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:48.900934  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.901386  5967 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.846287  5772 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.847288  5799 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:48.847688  5925 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.852s	user 0.004s	sys 0.019s
I20260709 13:02:48.921623  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:48.922901  1572 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:48.926944  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:48.860721  1722 rpcz_store.cc:276] 0709 13:02:46.929579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.929717 (+   138us) service_pool.cc:224] Handling call
0709 13:02:46.977304 (+ 47587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.355605 (+378301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.831506 (+1475901us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:48.831975 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859953536}
I20260709 13:02:48.861871  5815 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:48.873659  5928 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:48.869853  5972 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.837095  5950 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:48.890069  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2206) took 2208 ms (client timeout 1568 ms). Trace:
I20260709 13:02:48.891584  5883 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.892716  5837 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.892586  5536 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:48.894546  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:48.961830  1720 rpcz_store.cc:269] 0709 13:02:46.681358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.681556 (+   198us) service_pool.cc:224] Handling call
0709 13:02:48.889275 (+2207719us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b5400087854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:48.889711 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790493568}
I20260709 13:02:48.967548  2378 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:48.962862  5815 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:49.007158  5701 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.007s
I20260709 13:02:49.012394  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.013990  2379 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:02:49.016145  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:48.986179  5962 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.995630  5841 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.018967  5536 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.007911  5701 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.026247  5815 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.026865  5962 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.001784  5934 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:48.998216  5837 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:49.029908  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.032899  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:49.031829  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.033902  2378 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:49.000708  5802 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.042217  5961 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:49.052827  2376 raft_consensus.cc:2410] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:49.059304  5701 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:49.077286  5962 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:49.085759  5688 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.086807  5688 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.089679  5925 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.090545  5925 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.091240  5688 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.092594  5514 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.093333  5514 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.084448  5802 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.098232  5925 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.100093  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:49.101702  5711 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.886s	user 0.020s	sys 0.006s
I20260709 13:02:49.102324  5711 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:49.103190  5536 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:49.103761  5536 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.104307  5536 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.104522  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.105610  1720 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:49.106828  1722 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:49.108460  5925 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:49.112277  5925 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.113169  5925 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.117063  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.118266  1722 raft_consensus.cc:2410] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:02:49.108484  5847 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.260s	user 0.014s	sys 0.012s
I20260709 13:02:49.120316  5847 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.122202  1716 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:49.123863  5847 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.126329  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2219) took 1378 ms. Trace:
I20260709 13:02:49.131150  1716 rpcz_store.cc:276] 0709 13:02:47.748174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.748720 (+   546us) service_pool.cc:224] Handling call
0709 13:02:47.748962 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.122849 (+1373887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.125937 (+  3088us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003ab454. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:49.126252 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3157325568}
W20260709 13:02:49.127802  5698 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.562s	user 0.005s	sys 0.015s
I20260709 13:02:49.134083  5698 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:49.137460  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.140215  5698 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:49.155489  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 80167.5us
I20260709 13:02:49.157213  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:49.159211  5961 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:49.160153  5961 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:49.163483  5780 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:49.164541  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.164251  5780 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:49.170830  5780 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.171283  5928 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:49.161736  5961 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.176499  5934 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:49.177225  5780 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.181098  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.182320  5925 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.192721  1716 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:49.175469  5536 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.194432  5925 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.198429  5536 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.209930  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:49.210280  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:49.210958  2376 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.211083  2377 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.209998  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:49.213323  2379 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:49.219578  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 454698us
I20260709 13:02:49.234462  2377 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:49.235591  5927 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.249562  5536 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.250013  2379 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:49.250604  5536 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.252102  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:49.254944  5925 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:49.256150  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.257721  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.258920  1720 raft_consensus.cc:2376] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11. Re-sending same reply.
I20260709 13:02:49.259654  5925 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:49.256775  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:49.260637  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.261108  5883 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:49.261672  5883 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:49.262034  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.262867  5883 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.265120  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.264792  5913 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.266080  1719 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:49.266199  5913 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.267669  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.268873  1721 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:49.253865  5536 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.269474  5925 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.272053  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:49.275182  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.276288  1721 raft_consensus.cc:2376] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9. Re-sending same reply.
I20260709 13:02:49.249931  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.279402  2376 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:49.279739  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.280886  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.279680  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.284432  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.293078  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.293483  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:49.299249  1573 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:49.299249  1722 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:49.299932  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:49.301208  2375 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:49.302235  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.303715  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.304612  1718 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:02:49.306214  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.299329  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.307832  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.308775  2376 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:49.314324  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2353) took 2174 ms (client timeout 1668 ms). Trace:
W20260709 13:02:49.314922  1717 rpcz_store.cc:269] 0709 13:02:47.139642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.139884 (+   242us) service_pool.cc:224] Handling call
0709 13:02:49.313819 (+2173935us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400087354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:49.314072 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673952640}
I20260709 13:02:49.316118  5927 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.316886  5927 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.320583  5536 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:49.321370  5927 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.324431  1724 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:02:49.329252  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.716s (SENT) [suppressed 16 similar messages]
I20260709 13:02:49.336028  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:49.338219  1721 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:49.337455  5514 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:49.344002  1614 connection.cc:256] RPC call timeout handler was delayed by -1.480532366000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
I20260709 13:02:49.338759  5975 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:49.345808  1614 connection.cc:256] RPC call timeout handler was delayed by -1.3968323660000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
I20260709 13:02:49.325588  5536 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:49.354154  5975 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:49.357523  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:49.380095  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:49.368173  5815 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:49.380264  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:49.379865  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 50676.3us
I20260709 13:02:49.384567  1721 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:49.385318  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.381733  5983 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:49.400164  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:49.401062  2376 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:49.357755  5975 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:49.407601  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.428550  5711 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:49.429216  5711 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:49.432664  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63324.1us
I20260709 13:02:49.454864  5460 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:49.460817  1650 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:49.442544  5536 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.435595  5711 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.409874  5815 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.521772  5780 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:49.522624  5780 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.530097  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:49.533568  5802 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:49.535336  5913 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.536499  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:49.535797  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 614) took 1740 ms. Trace:
I20260709 13:02:49.562448  1724 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:49.570591  5802 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:49.577852  5780 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.583070  1567 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:49.586244  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.589372  1567 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:49.593256  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:49.596635  5913 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.591951  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.596213  1570 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:49.538348  5883 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.541162  1642 rpcz_store.cc:276] 0709 13:02:47.795019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.804110 (+  9091us) service_pool.cc:224] Handling call
0709 13:02:47.898884 (+ 94774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:47.982797 (+ 83913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.982840 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.004853 (+ 22013us) log.cc:844] Serialized 26 byte log entry
0709 13:02:48.630676 (+625823us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:48.630846 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.535062 (+904216us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400306e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:49.535162 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.535217 (+    55us) raft_consensus.cc:1685] finished
0709 13:02:49.535236 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.535571 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1583,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":49613,"spinlock_wait_cycles":191733504,"thread_start_us":462874,"threads_started":1,"wal-append.queue_time_us":493479}
I20260709 13:02:49.613729  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.538491  5514 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.618829  1721 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:49.626437  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:49.600274  5913 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.632818  1572 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:02:49.641861  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.642077  1568 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:49.643527  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:49.645705  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 109933us
I20260709 13:02:49.616605  5883 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.649009  1570 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:49.646647  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:49.659211  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.664924  1576 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:49.665354  1574 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:49.665073  5712 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:49.666740  5712 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.669770  5712 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:49.670449  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.673718  5802 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:49.674935  5802 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:49.677419  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.644438  5514 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.681638  1576 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:49.682559  5514 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.683090  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.683894  1576 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:49.687455  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.684702  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.684031  5815 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:49.687202  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.693351  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.694217  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.694736  1570 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:49.695590  1569 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:49.696374  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.696841  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:49.699712  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.700139  1568 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:49.700487  1570 raft_consensus.cc:2410] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:49.700791  5712 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:49.701642  5712 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:49.700521  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.712185  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.712575  1568 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:49.713222  1574 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:49.713670  1574 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:49.703852  5802 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.714745  1574 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:49.715488  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68972.5us
I20260709 13:02:49.716823  5883 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.716699  5712 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.713261  1576 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:02:49.720037  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.005s	sys 0.017s
I20260709 13:02:49.715802  1574 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:49.724263  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:49.724864  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.006s	sys 0.017s
I20260709 13:02:49.725378  1575 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:49.730181  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 814779us
I20260709 13:02:49.717535  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:49.739538  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.749893  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:49.750895  1570 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.754684  5815 raft_consensus.cc:2764] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:49.762144  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.705559  5967 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.765205  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.774771  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:49.775277  1574 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 13:02:49.775889  1574 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:49.773505  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:49.775626  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:49.776296  1574 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:02:49.789994  1574 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.795666  1568 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:49.796450  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.799165  1572 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:49.805431  5815 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:49.806352  5815 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.796579  1568 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:49.809584  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.810171  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.811542  1567 raft_consensus.cc:2410] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:49.815692  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.818570  1572 raft_consensus.cc:2410] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:49.833235  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.818558  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:49.839504  1576 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
W20260709 13:02:49.841903  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 124514us
I20260709 13:02:49.843179  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:49.848935  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46821.3us
W20260709 13:02:49.848254  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94955.7us
I20260709 13:02:49.838833  1575 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:49.844846  5712 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.872854  1644 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:49.873963  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:49.872875  1568 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:49.877525  5993 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.127s	user 0.001s	sys 0.001s
I20260709 13:02:49.878587  1568 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.872932  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:49.869059  5712 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:49.872951  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50561.7us
W20260709 13:02:49.886286  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.887742  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:49.890688  1575 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:49.891541  1575 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:49.893436  1575 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.898064  5688 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:49.898643  5780 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:49.899381  5688 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.901230  5999 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.900903  5688 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.902026  5999 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:49.905390  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:49.808495  5967 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:49.909566  5688 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:49.911347  5688 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:49.912125  5688 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.912760  5688 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.866187  5701 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:49.910313  1575 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.918620  1568 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:02:49.939257  1568 pending_rounds.cc:85] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:49.947492  1568 pending_rounds.cc:107] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:02:49.947894  1568 raft_consensus.cc:2889] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.913914  5967 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:49.993534  1570 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:49.899925  5780 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:50.009270  5711 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.005s
I20260709 13:02:50.009975  5711 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.012020  5780 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.028940  6000 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.029834  6000 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.017558  5999 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:50.038352  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 194956us
I20260709 13:02:50.041829  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:50.044198  6000 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.062414  6006 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:50.063282  6006 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.071933  6006 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.081065  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:50.090116  1575 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:02:50.091109  1575 pending_rounds.cc:85] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:02:50.091506  1575 pending_rounds.cc:107] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.8
W20260709 13:02:50.103363  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.104135  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.091858  1575 raft_consensus.cc:2889] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.105875  5698 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.106437  5698 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:50.106880  5698 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.125761  5701 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:50.124517  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:50.128211  5701 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:50.129868  5701 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.131882  5701 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.144173  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:50.145282  5997 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.344s	user 0.003s	sys 0.003s
I20260709 13:02:50.147990  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:50.150260  6006 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:50.151237  6008 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:50.156461  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 122458us
I20260709 13:02:50.163626  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.151957  6008 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:50.175098  6008 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.164846  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.184545  5514 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:50.187193  5698 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:50.188514  5994 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.474s	user 0.003s	sys 0.003s
I20260709 13:02:50.187992  5701 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.191988  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.188943  6000 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:50.192790  5698 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:50.195402  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.212927  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 49458.4us
W20260709 13:02:50.222429  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.217325  6006 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.226189  5514 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:50.227584  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 72919.9us
I20260709 13:02:50.226908  5514 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.229044  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:50.231434  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.237193  1614 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:50.233280  5514 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.242559  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.243536  6010 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:50.262015  6010 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.263398  6010 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.256898  6006 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.262567  5975 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.272972  1648 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:50.277634  5964 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.279066  5964 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.282196  5964 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.246836  5701 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.261695  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.289811  5536 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:50.290508  5536 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.291055  5536 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.244848  6008 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.282569  5477 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:50.292881  5477 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.293388  5477 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:50.282838  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.290753  5780 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:50.295785  6008 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.297405  5836 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:50.247599  6000 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.250497  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:50.315864  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.316983  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.317728  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.308283  5815 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.309106  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.310127  5698 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:50.323089  5698 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.310184  6000 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:50.323750  6000 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.324183  6000 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.324422  5698 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:50.310222  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73173.9us
I20260709 13:02:50.327667  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:50.328539  5514 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.329377  5701 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:02:50.309959  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 61433.5us
I20260709 13:02:50.333534  5701 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.336285  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:50.340205  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:50.340471  5701 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.342475  5477 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.344672  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.344956  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.345598  6014 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:50.345853  1716 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:50.345885  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.346141  6014 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.346688  5477 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.346814  6014 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:50.347473  5477 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.348590  5477 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.349644  5477 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.349536  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.353739  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:50.360594  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:50.360710  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:50.361554  1721 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.373297  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.377125  6003 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.357934  5999 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:02:50.343001  5964 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:02:50.394508  1721 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:02:50.399019  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60800.2us
W20260709 13:02:50.399636  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53830.4us
I20260709 13:02:50.402915  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:50.406828  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.405903  6015 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.409734  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.425937  1720 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:50.396566  5975 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:02:50.428840  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.416755  5999 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:02:50.431803  5514 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.440927  5514 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:50.441876  5514 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.442167  6006 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:50.442636  5514 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.443077  5536 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.447229  5536 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.409580  6010 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.436652  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.437330  5477 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.463011  5477 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.463620  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.463888  5712 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.421124  6015 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.432093  5999 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:50.464802  6015 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.447016  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.467154  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.468199  6006 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.469137  6006 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.448207  6014 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.438365  6000 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.464833  5999 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:50.482471  1720 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:50.483807  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:50.484519  1717 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.485208  5975 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:50.485700  5975 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.486109  5975 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.486539  6014 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.460189  6010 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.489897  6000 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.477145  1650 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:50.482178  6003 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:50.483448  5999 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.454646  5780 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.505201  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:50.506023  2378 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:50.508790  6003 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:50.510206  6003 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.510447  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:50.514385  5999 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.521437  1539 leader_election.cc:400] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:50.522025  1539 leader_election.cc:403] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:50.522794  6000 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:02:50.524489  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.533182  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:50.535863  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:50.531672  6008 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.545719  5802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.027s	sys 0.045s
I20260709 13:02:50.534494  6010 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:50.531683  6001 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.546826  6006 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:50.551157  5701 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:50.573057  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:50.558291  5802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.028s	sys 0.045s
W20260709 13:02:50.571736  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 170677us
W20260709 13:02:49.993489  5460 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.006s
I20260709 13:02:50.587394  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:50.588037  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:50.591298  1573 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:50.584959  5514 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.577046  6000 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.606379  1719 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:50.599686  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:50.606631  6020 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.607888  5711 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.590354  6010 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.564914  5815 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:02:50.628638  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55697us
W20260709 13:02:50.631724  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64918.5us
I20260709 13:02:50.634155  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:50.637537  1717 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:50.637662  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:50.638300  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:50.639971  1576 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:50.627367  5847 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.574493  6008 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.654639  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:50.655472  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74023.3us
I20260709 13:02:50.645967  5802 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:50.650890  6009 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:02:50.666687  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 257067us
I20260709 13:02:50.649122  5701 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.675283  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.677026  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.677183  6008 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.678668  6008 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:50.680644  5999 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:50.681661  5999 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:50.682106  5999 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.682590  5999 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.680725  5802 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:50.685899  1719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2372) took 1407 ms (client timeout 1700 ms). Trace:
W20260709 13:02:50.686393  1719 rpcz_store.cc:269] 0709 13:02:49.278406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.280492 (+  2086us) service_pool.cc:224] Handling call
0709 13:02:50.685600 (+1405108us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003ac354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:50.685761 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3004972288}
I20260709 13:02:50.687556  5802 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:50.693199  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.667764  6021 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:50.690142  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.695921  5514 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.696054  6003 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.690266  6015 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.699164  1721 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:50.699684  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:50.654357  5847 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.662372  5815 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:50.718092  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.699250  5711 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:50.656944  6020 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:50.726590  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95051.7us
I20260709 13:02:50.723866  5701 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.756474  6020 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.742477  6015 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.726053  5514 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:50.749140  5837 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:50.752090  6000 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:50.774470  6000 consensus_queue.cc:1243] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:50.771167  5514 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.775357  6000 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:50.776840  6025 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:50.777459  5514 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.694794  6021 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.780591  6010 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:50.772900  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54924.1us
I20260709 13:02:50.775813  6000 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:50.727771  5711 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.783599  6000 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.785416  5711 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:50.773240  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.777598  6025 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.717059  6003 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:50.793036  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 119455us
W20260709 13:02:50.788022  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73655.5us
I20260709 13:02:50.795002  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:50.798147  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.800359  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:50.786302  6021 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.793110  6003 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.793596  6025 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:50.813508  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:50.841179  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.847430  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.807139  6006 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:50.868613  1721 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:50.869341  1571 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:50.813664  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 152973us
I20260709 13:02:50.873350  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:50.874522  1575 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:50.876937  1717 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:50.837366  5925 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.877895  6006 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.879156  1720 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:50.879637  1573 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.813483  6000 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.880642  1719 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:50.881543  5712 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:50.882236  5712 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:50.883021  6001 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.884452  6001 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.886873  6001 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.890467  5975 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.876771  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:50.878822  5925 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:50.896821  1573 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:02:50.897878  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.900851  1576 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:50.853911  6003 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.836848  6015 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.907001  5712 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:50.820276  5711 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:50.911831  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 113783us
I20260709 13:02:50.918219  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.867231  5883 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:50.937392  2379 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:50.937330  2371 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:50.937798  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:50.943746  2378 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:50.939510  5925 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.939695  5847 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:50.940122  6013 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:50.964916  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:50.940476  5975 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:50.940837  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:50.965670  1570 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.966089  1567 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.941246  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:50.966969  1568 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:50.942147  1575 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:02:50.932314  1719 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:50.945900  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.968688  1721 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:50.947412  1573 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:02:50.953507  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.953539  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.958312  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.958775  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.959780  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:50.972170  1576 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:50.962445  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:50.978549  1718 raft_consensus.cc:2368] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:50.967633  1575 pending_rounds.cc:85] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:02:50.983454  1575 pending_rounds.cc:107] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:02:50.983918  1575 raft_consensus.cc:2889] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.917205  5964 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:50.993876  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:50.987572  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:50.996487  1722 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.006625  2378 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:50.904213  5999 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.000007  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:51.009441  1644 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:50.958281  5460 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:51.004304  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.016304  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.017467  1719 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:51.019735  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.023173  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:51.012379  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:51.000635  6014 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.954334  6015 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.040723  1718 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:51.038112  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.044821  2376 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:50.955988  5836 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:02:51.061741  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.062502  1721 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.057981  1722 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:51.011088  5925 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:51.025808  1672 debug-util.cc:398] Leaking SignalData structure 0x7b080039b200 after lost signal to thread 434
W20260709 13:02:51.076764  1672 debug-util.cc:398] Leaking SignalData structure 0x7b08000f6780 after lost signal to thread 1508
W20260709 13:02:51.078727  1672 debug-util.cc:398] Leaking SignalData structure 0x7b08002eeb20 after lost signal to thread 1600
I20260709 13:02:51.080194  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.063903  1570 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:02:51.084276  1672 debug-util.cc:398] Leaking SignalData structure 0x7b0800308900 after lost signal to thread 1601
I20260709 13:02:51.002450  5847 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:02:51.090982  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:50.999668  5883 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.089566  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.096159  5815 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.097062  5815 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.097889  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.098670  1722 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:51.099650  5815 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.099864  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.082679  5925 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.100688  5925 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.100814  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.060581  6014 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:51.102123  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97957.4us
I20260709 13:02:51.102876  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.100814  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:51.089964  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.074705  5698 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.092157  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.037667  6030 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:51.082470  5837 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.119742  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.119505  5837 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.119942  1722 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:51.121253  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.122596  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.096206  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.123378  1568 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:51.123616  1725 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:51.089730  6001 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.124815  6001 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.125362  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:51.108530  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 85494us
I20260709 13:02:51.126622  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:51.111310  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.014008  5964 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:51.115036  1672 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6680 after lost signal to thread 1675
W20260709 13:02:51.132823  1672 debug-util.cc:398] Leaking SignalData structure 0x7b080030dbe0 after lost signal to thread 1676
W20260709 13:02:51.134722  1672 debug-util.cc:398] Leaking SignalData structure 0x7b08001b0600 after lost signal to thread 1749
I20260709 13:02:51.115855  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:51.117480  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 105450us
I20260709 13:02:51.135390  1721 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:51.122731  5837 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:51.066407  5802 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.112107  1574 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:51.105645  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:51.137564  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.129437  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.131556  1719 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:51.138636  2371 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 7.
I20260709 13:02:51.139473  1568 raft_consensus.cc:2410] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:02:51.127524  1573 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:51.092315  5514 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:51.142436  1672 debug-util.cc:398] Leaking SignalData structure 0x7b08002efc00 after lost signal to thread 1750
W20260709 13:02:51.144057  1672 debug-util.cc:398] Leaking SignalData structure 0x7b080033d440 after lost signal to thread 2404
W20260709 13:02:51.144768  1672 debug-util.cc:398] Leaking SignalData structure 0x7b08002b7880 after lost signal to thread 2405
I20260709 13:02:51.150442  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:51.150902  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45353.5us
I20260709 13:02:51.094688  5780 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.168192  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.170230  1569 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:51.125468  5883 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:51.107227  5712 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.109058  6014 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.106738  5815 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.126960  6006 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.178408  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.180104  1570 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:51.180213  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.188978  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.130477  6030 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:51.176064  5688 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.191s	user 0.016s	sys 0.006s
I20260709 13:02:51.169831  5780 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.207685  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.174928  6013 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.183347  6000 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.128041  5698 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.230326  2371 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:51.183943  5460 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:51.174876  5975 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:51.227059  1571 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:02:51.171003  6043 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:02:51.228808  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.248023  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.248977  1648 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:51.218104  5780 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.201927  5701 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:51.211558  5688 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.261405  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.261298  5460 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.262282  2375 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:02:51.247294  1644 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.259896  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.250551  6031 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:51.250774  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.261420  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.267625  1722 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:51.254609  1725 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:51.256526  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.257414  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:51.269531  2378 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.179549  5712 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.212868  6020 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.181785  5802 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.214110  6013 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.177806  6009 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:02:51.273207  1716 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:51.227486  5815 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.287352  2371 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:02:51.267197  5837 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:51.277842  6031 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:51.278295  1616 connection.cc:256] RPC call timeout handler was delayed by -1.4046378090000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 13:02:51.278460  1723 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:51.300508  1616 connection.cc:256] RPC call timeout handler was delayed by -1.23453780899996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 13:02:51.300906  1616 connection.cc:256] RPC call timeout handler was delayed by -1.1696313550000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3068s for a response
I20260709 13:02:51.239626  6000 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.280097  5964 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.303471  1539 leader_election.cc:400] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:51.303850  1539 leader_election.cc:403] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:51.303996  1722 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:51.283593  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.217202  6042 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:02:51.268150  1649 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:02:51.301178  1616 connection.cc:256] RPC call timeout handler was delayed by -1.1463378089999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
W20260709 13:02:51.288779  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 169549us
W20260709 13:02:51.295250  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.298416  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.210276  6028 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:02:51.302991  2376 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:51.276546  5999 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.286819  6013 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.315596  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.316102  1717 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:51.320271  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 142533us
I20260709 13:02:51.321135  1648 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:02:51.329366  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.373405  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.325958  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.287464  6020 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.325568  6006 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:51.381428  6006 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:51.382325  1648 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:51.383540  6043 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:51.384008  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.385685  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:51.386444  1649 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.389154  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:51.400254  6046 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:02:51.401607  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:51.402441  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.402573  1721 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:51.307840  5975 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.331702  5712 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.403985  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.335539  5802 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.406044  2378 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:02:51.334136  6000 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.325568  6031 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.409159  6049 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:02:51.415247  6014 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.416822  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.418186  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:51.366153  5999 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:51.425141  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.367548  6015 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:51.470633  1722 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:51.336138  5698 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.444008  1649 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.473300  1722 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:51.309504  5837 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.447587  2371 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:02:51.456187  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 165139us
I20260709 13:02:51.460530  1644 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 8: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:51.375895  5514 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:51.465372  5999 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.483696  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.487495  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.465854  6052 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.401484  5688 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:51.494488  6032 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:02:51.469826  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.338258  6030 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.499586  5999 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.502135  5975 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.436128  5780 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.506862  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.509977  1718 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:51.458113  6001 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:51.461074  6000 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:51.525183  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:51.526594  1688 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.619s (SENT)
W20260709 13:02:51.526724  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47200us
W20260709 13:02:51.519268  1567 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.002s
I20260709 13:02:51.492116  6008 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.536937  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.529528  1567 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:51.504719  5837 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:51.382400  6006 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.548982  5815 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.549256  6006 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.550076  6058 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:02:51.511241  5999 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.509374  5925 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.552605  5999 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.553123  5999 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.553794  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.554885  5925 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:51.555267  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.556130  6001 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.556650  6001 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.557057  6001 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:51.557436  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.716s (SENT)
I20260709 13:02:51.557777  6014 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:51.559302  1576 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:51.559716  1576 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:51.454610  6003 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.485620  6048 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:02:51.561729  1720 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:51.553153  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.568415  1574 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:51.565356  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.569420  1567 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:51.573015  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.574044  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.575011  1571 raft_consensus.cc:2410] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:51.567791  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.580312  1568 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:51.580999  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.588747  1644 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:51.577956  1576 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.541316  6008 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.596262  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.599061  1568 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:51.541827  5712 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:02:51.596901  1569 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:51.598145  1572 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:51.596936  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:51.600921  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.597990  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.603899  1571 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.605129  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.605844  1570 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.600837  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.608928  1574 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.616688  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.590044  5837 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:51.541548  5802 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 13:02:51.601284  6025 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.713s	user 0.005s	sys 0.010s
I20260709 13:02:51.622308  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.604559  6008 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.569049  5711 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 13:02:51.566623  5701 consensus_queue.cc:1175] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f737473248e4c059fa74af3b8e1f77e" responder_term: 13 status { last_received { term: 13 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 13 index: 7 } }
I20260709 13:02:51.523808  6052 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.653288  5711 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:51.653774  5711 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:51.566999  6000 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.589088  6032 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:02:51.647010  6029 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:51.648855  6024 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:51.649039  6049 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:51.649359  6055 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:51.657127  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:51.645108  1567 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:02:51.667691  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 69776.5us
I20260709 13:02:51.597577  5975 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:51.625411  5837 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.687589  1574 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:51.611110  6058 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:02:51.603075  6021 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:02:51.573793  5514 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.694720  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.764560  1568 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.696902  1569 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:02:51.765179  1569 pending_rounds.cc:85] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:02:51.765452  1569 pending_rounds.cc:107] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:51.765679  1569 raft_consensus.cc:2889] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.701720  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:51.705063  1722 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:51.655318  6052 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:51.709515  1575 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.720202  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.721218  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.745746  1570 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:51.768566  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66976.1us
I20260709 13:02:51.694115  6049 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.749848  1571 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:51.687055  6029 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:51.767554  2371 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.767520  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.767928  1716 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:51.714287  5837 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:51.743317  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134572us
I20260709 13:02:51.781400  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.787789  1716 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:51.781510  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 124499us
I20260709 13:02:51.769623  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:51.782841  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:51.789096  1569 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:02:51.791874  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.792302  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.792883  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.792398  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.656929  6025 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.761333  5975 consensus_queue.cc:1243] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:51.655397  5711 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.793954  2376 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.796835  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:51.802032  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:51.805433  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:51.809304  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.810034  1571 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.777101  5961 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.783072  6049 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:51.825465  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:51.830114  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:51.830592  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.831591  1716 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:51.833947  1720 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:51.835050  6003 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.835853  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.837235  1649 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:51.840332  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:51.836969  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.843981  1575 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:51.845623  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:51.810199  5802 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.803715  6020 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:51.809716  5780 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:51.794301  6057 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:51.852424  1651 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:51.850803  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.853361  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:51.881328  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.855719  5460 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:51.856494  6035 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.856701  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.893702  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.856966  6022 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:51.857208  6054 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.219s
I20260709 13:02:51.855747  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.858192  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.858582  6013 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:51.859153  6033 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:02:51.859287  6060 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:02:51.928720  6033 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:51.865329  6061 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:02:51.865643  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:51.866465  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:51.874143  1568 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:51.875320  1571 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:51.903596  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.889531  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.894222  1647 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:02:51.934549  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 146928us
I20260709 13:02:51.931563  6033 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:51.817301  6051 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.252s
W20260709 13:02:51.950565  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 105038us
I20260709 13:02:51.807467  6052 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:02:51.869081  5802 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:51.952965  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63530.6us
I20260709 13:02:51.951249  1643 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:51.951983  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:51.959249  1644 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:51.877892  6057 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:51.968905  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 91873.2us
I20260709 13:02:51.893108  6014 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:51.964859  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.925603  6020 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.943873  6003 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:51.978308  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:52.001358  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.963868  5847 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:51.971057  5802 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.966815  5712 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:52.005393  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53511.1us
I20260709 13:02:51.981559  6014 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.026049  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:52.037307  1717 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:51.980890  6057 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:52.033569  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 236817us
I20260709 13:02:52.009087  6021 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.049654  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:51.985780  5460 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:51.972885  6048 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:52.002650  5711 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.097865  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:52.098515  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53415.4us
I20260709 13:02:52.004516  5780 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:52.007215  6022 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:51.960466  6029 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.115980  1720 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:52.153846  6025 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:52.216640  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 202157us
I20260709 13:02:52.156559  6063 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:52.354660  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:52.043959  6014 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.097725  6000 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:52.142992  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 117137us
W20260709 13:02:52.363170  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 265406us
I20260709 13:02:52.364310  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.360913  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:51.944948   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:52.152601  6073 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:52.155929  6066 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:52.374800  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:52.037056  6043 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:52.228430  6079 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.141618  5514 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:51.992471  6001 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.152201  6070 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:52.145017  5712 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:52.252128  6010 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:52.213645  6058 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:52.229642  6069 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:02:52.234771  5999 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.005s
W20260709 13:02:52.220011  6062 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.408s	user 0.001s	sys 0.001s
I20260709 13:02:52.299993  6030 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:52.300669  6013 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:52.244145  6008 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.001s
I20260709 13:02:52.324379  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 630) took 1423 ms. Trace:
I20260709 13:02:52.151927  6021 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.441113  6087 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:52.434130  5712 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:52.144320  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 342406us
I20260709 13:02:52.444578  5712 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.161571  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2243) took 1062 ms. Trace:
I20260709 13:02:52.448616  1724 rpcz_store.cc:276] 0709 13:02:51.098985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.100920 (+  1935us) service_pool.cc:224] Handling call
0709 13:02:51.101261 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.167063 (+ 65802us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:51.177494 (+ 10431us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.515103 (+337609us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b1800410200. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:02:51.609309 (+ 94206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:51.609370 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:51.609430 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.161214 (+551784us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5400223654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:52.161312 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.161495 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14003,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":63,"raft.queue_time_us":2010,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":7753,"spinlock_wait_cycles":59839104,"thread_start_us":93902,"threads_started":1,"wal-append.queue_time_us":485191}
I20260709 13:02:52.451465  6014 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.342593  1719 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.416780  6070 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:52.350183  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:52.452811  6070 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:52.453239  6084 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:52.453927  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 103866us
I20260709 13:02:52.455506  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:52.312541  5847 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.424064  6024 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:52.457181  5847 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.457633  6024 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.459723  6009 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.329749  6048 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.456094  6014 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:52.415330  6079 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.461122  6048 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.462854  6048 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.463691  6079 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.371079  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:52.465288  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 94295.1us
I20260709 13:02:52.466667  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:52.338871  5460 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.469373  5460 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.470963  6085 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:52.355890  5964 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:52.377187  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.473244  5964 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:52.473292  6008 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.473948  5964 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:52.474013  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96944.7us
I20260709 13:02:52.475528  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:52.349895  6022 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:52.476572  6054 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.381685  6071 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:02:52.477893  6022 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.478937  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.479655  6071 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:52.432302  6058 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.480105  5712 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:52.376688  6052 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:52.480715  6013 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:52.386173  6059 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.593s	user 0.004s	sys 0.009s
W20260709 13:02:52.376047  5837 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.013s
I20260709 13:02:52.435966  5999 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.482496  5837 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.483456  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 521) took 1024 ms. Trace:
W20260709 13:02:52.404830  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.392318  5802 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.484431  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:52.378207  6029 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.415537  5514 consensus_queue.cc:1243] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:52.342389  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2241) took 1244 ms. Trace:
I20260709 13:02:52.489974  1647 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.484942  5837 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.477043  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 575) took 1207 ms. Trace:
I20260709 13:02:52.480690  5712 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:52.381937  6000 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.477977  6035 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:52.481482  6052 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.359019  5780 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.425328  6010 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:52.397248  6006 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.847s	user 0.015s	sys 0.000s
I20260709 13:02:52.354445  6063 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:52.485792  1570 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:52.507148  6063 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.508610  1642 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:02:52.510423  6081 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:52.483879  2379 rpcz_store.cc:276] 0709 13:02:51.458900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.459071 (+   171us) service_pool.cc:224] Handling call
0709 13:02:51.459504 (+   433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:51.467904 (+  8400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.467939 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:51.468720 (+   781us) log.cc:844] Serialized 52 byte log entry
0709 13:02:51.607258 (+138538us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:51.607303 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.607382 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.385567 (+778185us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b240007a8e8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830b079c4 00007fc830b05acb 00007fc830cc0dc9 00007fc830cc0a55 00007fc830cc009d 00007fc8338f210e 00007fc83390988c 00007fc833908bb6 00007fc8339087d8 00007fc833912221
0709 13:02:52.483116 (+ 97549us) raft_consensus.cc:1685] finished
0709 13:02:52.483141 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.483375 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11146,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":28142848,"thread_start_us":138018,"threads_started":1,"wal-append.queue_time_us":229567}
I20260709 13:02:52.510912  6078 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:52.491758  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.490317  1718 rpcz_store.cc:276] 0709 13:02:51.098230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.099538 (+  1308us) service_pool.cc:224] Handling call
0709 13:02:52.342092 (+1242554us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540010ead4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:52.342311 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940897664}
I20260709 13:02:52.435793  1650 rpcz_store.cc:276] 0709 13:02:50.901185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.901893 (+   708us) service_pool.cc:224] Handling call
0709 13:02:50.902302 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:50.941858 (+ 39556us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:51.160774 (+218916us) log.cc:844] Serialized 14 byte log entry
0709 13:02:51.665046 (+504272us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b1800404500. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:02:51.932822 (+267776us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.954446 (+ 21624us) log.cc:844] Serialized 26 byte log entry
0709 13:02:51.956843 (+  2397us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:51.956899 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.957001 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.304238 (+347237us) spinlock_profiling.cc:243] Waited 9.26 ms on lock 0x7b1800404500. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:52.324053 (+ 19815us) raft_consensus.cc:1685] finished
0709 13:02:52.324078 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.324278 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129221,"prepare.queue_time_us":226959,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":128200,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":18923,"spinlock_wait_cycles":26022400,"thread_start_us":411207,"threads_started":3,"wal-append.queue_time_us":641704}
I20260709 13:02:52.493750  1725 rpcz_store.cc:276] 0709 13:02:51.269406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.488967 (+219561us) service_pool.cc:224] Handling call
0709 13:02:51.512628 (+ 23661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.516584 (+  3956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.516629 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.540675 (+ 24046us) log.cc:844] Serialized 26 byte log entry
0709 13:02:51.598468 (+ 57793us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:51.598504 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.598566 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.476497 (+877931us) raft_consensus.cc:1685] finished
0709 13:02:52.476524 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.476929 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":17214,"thread_start_us":48464,"threads_started":1,"wal-append.queue_time_us":194652}
I20260709 13:02:52.478940  6001 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:52.484715  5999 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:52.500558  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45150.4us
I20260709 13:02:52.494674  6087 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:52.502274  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.499719  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 125126us
I20260709 13:02:52.471743  6085 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.479807  6022 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.505288  5837 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:52.506067  6032 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.505519  5780 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.509895  6010 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.432423  6030 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:52.448150  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.523828  1722 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.510995  6081 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:52.529428  6013 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:52.520390  6084 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.530887  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:52.497319  6079 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:52.590664  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2401) took 1352 ms. Trace:
I20260709 13:02:52.501080  6043 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.511979  6078 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:52.591431  1719 rpcz_store.cc:276] 0709 13:02:51.238265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.240825 (+  2560us) service_pool.cc:224] Handling call
0709 13:02:52.342296 (+1101471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.452782 (+110486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.590424 (+137642us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540010ead0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:52.590585 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466823424}
I20260709 13:02:52.539968  1720 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:52.498724  5712 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:52.559134  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.502447  6052 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.557622  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:52.559151  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:52.559926  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:52.551862  6021 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:52.581945  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 133913us
I20260709 13:02:52.588533  1576 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.571771  6057 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:52.615947  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56872.8us
I20260709 13:02:52.617492  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:52.561327  6006 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:52.568913  6035 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:52.565677  5698 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:02:52.609160  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 118801us
I20260709 13:02:52.590675  5711 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.584179  6081 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:52.684959  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 127423us
I20260709 13:02:52.686841  1721 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.592547  6020 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:52.584424  6013 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.704924  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:52.684619  6079 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.689635  1650 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:52.622568  6010 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.694805  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:52.620802  5815 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:52.759802  5815 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:52.702260  1573 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:52.706795  1568 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:02:52.730892  1571 pending_rounds.cc:85] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:02:52.734899  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:52.760310  5815 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.760941  1571 pending_rounds.cc:107] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:02:52.769019  1571 raft_consensus.cc:2889] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.737344  1569 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:52.682034  6035 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.734133  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:52.734373  5802 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.709057  5711 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.681210  6021 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.757998  6033 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.707721  6013 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.761229  1572 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:02:52.708374  6049 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.861s	user 0.016s	sys 0.000s
W20260709 13:02:52.782248  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 87620us
I20260709 13:02:52.786464  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.731585  5964 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:52.803287  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 102571us
I20260709 13:02:52.762738  6048 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:52.801976  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:52.813751  5999 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:52.681854  6030 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.818084  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:52.819020  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.826046  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:52.803915  6001 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.736125  6079 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.828680  5964 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:52.849805  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:52.809353  6054 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:52.857043  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:52.858183  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1918) took 1098 ms. Trace:
I20260709 13:02:52.862876  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:52.810122  6021 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:52.806182  6049 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.867799  6049 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:52.869719  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:52.876905  1718 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:52.868295  1645 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:52.875847  6076 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.723s	user 0.007s	sys 0.000s
I20260709 13:02:52.890056  1473 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 6 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.893002  1473 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 7, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:52.898275  6024 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.861065  1651 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:52.862810  1570 rpcz_store.cc:276] 0709 13:02:51.756224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.768199 (+ 11975us) service_pool.cc:224] Handling call
0709 13:02:52.854417 (+1086218us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540022d8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:52.854588 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591617280}
I20260709 13:02:52.907202  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:52.908047  2371 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.839361  5999 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:52.911353  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 48568us
I20260709 13:02:52.911808  1574 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.843886  6048 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:52.780017  6078 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.780247  6043 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.805976  6084 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:52.926587  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.917613  6030 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.937762  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.938588  1572 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:52.861524  6054 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:52.713464  6087 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:52.900924  6073 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:52.905102  6097 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:02:52.946059  1576 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:02:52.952622  1576 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:02:52.952965  1576 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:52.953269  1576 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.954842  6100 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.954964  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1894) took 1421 ms. Trace:
I20260709 13:02:52.955684  6100 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:52.955651  1576 rpcz_store.cc:276] 0709 13:02:51.533620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.533965 (+   345us) service_pool.cc:224] Handling call
0709 13:02:51.559114 (+ 25149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.580719 (+1021605us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540019d820. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce3a1d 00007fc833cca1a4 00007fc833d4c447 00007fc833d64fe7 00007fc833d667de 00007fc833d70b49 00007fc833d71416 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419
0709 13:02:52.954417 (+373698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.954801 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461389184}
I20260709 13:02:52.958508  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:52.968087  6058 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:52.960642  6001 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:52.917259  6049 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:52.977448  1572 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:52.980124  1688 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.838s (SENT)
I20260709 13:02:52.981001  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:52.985316  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:52.908918  6096 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:02:52.936221  6085 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.945572  6054 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.989605  1718 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:52.919247  6074 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:02:52.998395  1574 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:52.974900  6001 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.980883  6049 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.960737  6070 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.010470  1575 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:52.986455  6057 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.014110  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.015916  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:53.009793  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.020313  1722 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.026266  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.033252  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:52.981436  5712 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.012440  6001 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:53.041213  6105 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:53.042025  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:53.071537  1716 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:53.047796  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1914) took 1348 ms. Trace:
I20260709 13:02:53.048863  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:53.040764  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:53.075891  1724 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:53.035703  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:53.063701  1722 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:53.017041  6049 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.009883  6003 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:53.056195  2371 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:53.072477  1567 rpcz_store.cc:276] 0709 13:02:51.699445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.748914 (+ 49469us) service_pool.cc:224] Handling call
0709 13:02:51.775965 (+ 27051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.845763 (+ 69798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.845812 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.847903 (+  2091us) log.cc:844] Serialized 26 byte log entry
0709 13:02:52.808729 (+960826us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b180048e8c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:02:53.006355 (+197626us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:53.006401 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.006506 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.047368 (+ 40862us) raft_consensus.cc:1685] finished
0709 13:02:53.047399 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.047708 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12561,"raft.queue_time_us":59928,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":19032,"spinlock_wait_cycles":46121344,"thread_start_us":261900,"threads_started":2,"wal-append.queue_time_us":1185771}
I20260709 13:02:53.035512  5711 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:52.982617  6048 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:53.076735  6104 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:02:53.101914  1568 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:53.035610  6092 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:53.109472  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.084961  6035 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:53.121093  1651 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:53.090389  6022 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 13:02:53.115481  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:53.122113  1567 raft_consensus.cc:2368] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
I20260709 13:02:53.098850  6057 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.131526  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.133657  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.134620  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.011816  6021 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.136536  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:53.146297  2371 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:53.147899  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.129127  6079 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.158699  6058 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:53.163276  6008 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.168468  6106 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.169345  6106 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.170258  1642 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:53.139271  6057 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:53.176959  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.176892  2376 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:53.177663  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.180544  6106 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.181264  5712 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.181511  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:02:53.182081  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.903s (SENT)
I20260709 13:02:53.183069  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:53.178974  1650 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:02:53.184806  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:53.194808  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.195114  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.127395  5815 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.201340  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.195873  2380 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:02:53.189103  6029 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.691s	user 0.003s	sys 0.010s
I20260709 13:02:53.195156  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:53.204707  1650 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:53.226852  2371 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:53.199115  5836 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:53.191726  6070 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:53.247514  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 230704us
I20260709 13:02:53.191948  6010 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:53.248277  5698 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:02:53.241390  6063 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.734s	user 0.021s	sys 0.000s
I20260709 13:02:53.264292  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:53.261078  6121 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:02:53.237414  6100 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.257092  6010 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:53.262982  1688 leader_election.cc:336] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.537s (SENT)
I20260709 13:02:53.281332  1642 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:53.283619  1688 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.675s (SENT)
I20260709 13:02:53.284286  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:53.288247  1688 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.031s (SENT)
I20260709 13:02:53.280282  6063 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:53.303697  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 328948us
W20260709 13:02:53.306646  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68911.5us
I20260709 13:02:53.263757  6021 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 13:02:53.316942  6119 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:02:53.316624  5698 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:53.319701  1642 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:53.321275  1569 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:53.319602  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:53.284670  6010 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.335386  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:53.338263  6003 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:53.339107  6003 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.291810  6043 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:53.350585  6013 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.000s
W20260709 13:02:53.319064  6052 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.647s	user 0.015s	sys 0.000s
I20260709 13:02:53.351490  6013 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.352375  6052 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.355193  6013 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:53.355618  6052 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:53.357506  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:53.358361  1576 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:53.357817  1650 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:53.298859  5711 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:53.367026  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.310076  6100 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:53.367846  2374 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:53.372257  5815 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.351490  6001 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.365453  5711 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:53.376821  6001 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.377589  5711 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.379109  5711 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:53.381886  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.290927  6054 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.402617  6100 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.406133  5802 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:53.406914  5802 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.406914  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:53.409217  6051 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.409781  6051 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.410710  5802 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.413196  6100 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.411406  5802 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:53.414283  5802 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.414780  5802 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:53.415411  6051 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:53.417425  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.421522  6033 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.004s
I20260709 13:02:53.423369  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.424242  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.424443  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:53.429003  1645 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.429208  1646 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:53.425041  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.425138  6008 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:53.425302  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.425870  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.426156  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:53.427181  1688 leader_election.cc:336] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.711s (SENT)
I20260709 13:02:53.432704  1642 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:53.432003  1649 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:53.432111  1651 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:53.452378  1688 leader_election.cc:336] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.957s (SENT)
I20260709 13:02:53.453282  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:53.473546  6049 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.431707  6008 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:53.486189  6049 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.487177  5802 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:53.488157  6081 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.000s
I20260709 13:02:53.523118  1716 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:02:53.493660  6032 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.917s	user 0.015s	sys 0.000s
I20260709 13:02:53.492064  1645 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:53.571497  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.490062  6010 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.573566  6065 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:53.574307  6065 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:53.536516  6057 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.547520  6063 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.546691  6054 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.550084  6003 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.591355  6003 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.592959  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:53.595129  6082 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:53.596113  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.596518  6094 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:53.568753  6109 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:53.600162  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.601099  1642 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:53.615367  1649 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:02:53.577939  6032 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.616778  1642 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:02:53.621327  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:53.622442  1645 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:53.629601  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:53.631794  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:53.632720  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:53.633544  1571 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:53.633992  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:53.635910  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 183678us
I20260709 13:02:53.636077  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:53.637800  1647 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:53.639339  1571 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:53.641794  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:53.645079  1651 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:53.645114  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:53.645651  1576 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:53.645991  1575 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:53.649570  1688 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.009s (SENT)
I20260709 13:02:53.651857  1576 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:53.571450  6081 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.634943  6003 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.575665  6065 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:53.666718  1688 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.110s (SENT)
I20260709 13:02:53.667352  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:53.618677  6048 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:02:53.625039  6092 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:53.675321  2377 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:53.677240  6092 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.678579  1575 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:02:53.675364  1688 leader_election.cc:336] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.956s (SENT)
I20260709 13:02:53.680097  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:53.681612  6081 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:53.685645  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.687280  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.658351  5802 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:53.697538  1688 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.198s (SENT)
I20260709 13:02:53.693333  6014 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:53.692413  6065 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:53.671806  6003 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.726790  6003 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.728199  6078 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:53.733639  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1931) took 1024 ms. Trace:
I20260709 13:02:53.734436  1568 rpcz_store.cc:276] 0709 13:02:52.709070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.763892 (+ 54822us) service_pool.cc:224] Handling call
0709 13:02:52.764204 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.119715 (+355511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.733248 (+613533us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400026cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:53.733543 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776073600}
I20260709 13:02:53.736687  6010 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:53.694998  6138 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:53.673569  6082 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:53.741870  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44403.8us
W20260709 13:02:53.750398  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.704092  6109 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:02:53.758919  6139 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:53.766260  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100151us
I20260709 13:02:53.780771  1572 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:53.780691  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:53.774732  6139 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:02:53.789006  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1909) took 2249 ms (client timeout 1556 ms). Trace:
I20260709 13:02:53.798311  1575 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:02:53.798527  1573 rpcz_store.cc:269] 0709 13:02:51.539872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.554993 (+ 15121us) service_pool.cc:224] Handling call
0709 13:02:53.788608 (+2233615us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002a76d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:53.788913 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636063488}
W20260709 13:02:53.794893  6130 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.429s	user 0.000s	sys 0.002s
I20260709 13:02:53.805312  6139 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.825592  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.826906  2378 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:53.830083  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.831465  1716 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:53.836745  1571 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:53.839720  1718 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:53.841012  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:53.841651  1723 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:53.841988  1719 raft_consensus.cc:606] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:53.843106  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:53.843855  1688 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.065s (SENT)
I20260709 13:02:53.799543  6092 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.785007  6146 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:02:53.852478  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.854971  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:53.859112  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:53.862119  1688 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.235s (SENT)
I20260709 13:02:53.862160  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:53.862257  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2423) took 1338 ms. Trace:
I20260709 13:02:53.863020  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:53.864535  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.865192  1716 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:53.863210  1721 rpcz_store.cc:276] 0709 13:02:52.524179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.524460 (+   281us) service_pool.cc:224] Handling call
0709 13:02:52.524861 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.700604 (+175743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.861968 (+1161364us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001a77d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:53.862188 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342001536}
I20260709 13:02:53.855691  6139 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:53.871162  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:53.880924  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.801232  6082 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.886662  6147 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:53.887465  6147 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:53.870168  5780 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:53.901437  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.904779  1724 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:53.908363  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.914013  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51956.3us
W20260709 13:02:53.915778  1688 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.578s (SENT)
I20260709 13:02:53.916844  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:53.917943  1721 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:53.917516  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.919986  1724 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:53.920085  1568 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:53.919387  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.921240  1718 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 6.
W20260709 13:02:53.923002  1688 leader_election.cc:336] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.599s (SENT)
I20260709 13:02:53.934273  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:53.938038  1718 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:53.962112  6147 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:53.975368  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:53.984320  6014 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:53.985030  6014 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:53.986213  6014 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:53.992316  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:53.992720  6146 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:02:53.993057  6092 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:02:53.999406  6092 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:53.999931  6092 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:53.999781  1473 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 6, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 6 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:54.001907  6092 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.002707  6147 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:02:54.003721  6147 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:54.004179  6147 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:54.004132  5847 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:54.005349  6147 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.007097  6147 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.009742  6146 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:54.017472  6109 raft_consensus.cc:993] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:54.018350  6109 raft_consensus.cc:1081] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:54.003376  6092 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:54.021708  1473 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 7, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.005856  6035 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:02:54.025697  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.029297  1688 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.698s (SENT)
W20260709 13:02:54.030558  1688 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.703s (SENT)
I20260709 13:02:54.030964  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:54.041033  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.027563  6035 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:54.046449  1719 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:02:54.047708  6035 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.048189  1473 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 2 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.049352  6035 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.050406  1473 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 6, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 6 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:54.054131  1469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 205) took 30895 ms (client timeout 15000 ms). Trace:
I20260709 13:02:54.056713  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:54.057508  1720 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.057737  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:54.054814  1473 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:02:54.067931  1473 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:02:54.078310  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.059378  6078 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.034279  6111 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:54.109535  1473 catalog_manager.cc:5697] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 6, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.062031  5999 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.068497  6109 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:54.088438  6057 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.151656  6111 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:54.168267  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.170126  6003 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:54.092149  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:54.108780  6063 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.109002  6054 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.225382  6054 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.227911  6054 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:54.195719  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.083010  6065 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 13:02:54.197556  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 106905us
I20260709 13:02:54.209434  1720 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:54.211130  5925 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:54.212816  5698 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:54.212951  6147 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.214183  1473 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 8 to 7, leader changed from <none> to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:02:54.216838  1574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 1924) took 1512 ms (client timeout 1716 ms). Trace:
W20260709 13:02:54.221781  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.268038  1574 rpcz_store.cc:269] 0709 13:02:52.704251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.704438 (+   187us) service_pool.cc:224] Handling call
0709 13:02:54.216565 (+1512127us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000cbf54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:54.216751 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473460992}
I20260709 13:02:54.272984  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:54.274098  2379 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.278219  1473 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.224277  6063 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.233145  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1932) took 1376 ms. Trace:
I20260709 13:02:54.236696  6057 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.108664  6078 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.088860  5837 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.249861  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.290254  1643 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:54.291410  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.250950  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.292812  1644 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:54.252017  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.253188  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.259939  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 610) took 1739 ms. Trace:
I20260709 13:02:54.233448  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.253376  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.296648  1651 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:54.265211  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:54.270741  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.297617  1642 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:54.281754  1724 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:54.286120  1569 rpcz_store.cc:276] 0709 13:02:52.856644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.856838 (+   194us) service_pool.cc:224] Handling call
0709 13:02:52.857314 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.324681 (+467367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.232362 (+907681us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54001aa9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:54.233055 (+   693us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066587904}
I20260709 13:02:54.298067  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:54.301312  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:54.287838  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278634us
I20260709 13:02:54.305475  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.294288  2377 rpcz_store.cc:276] 0709 13:02:52.520198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.530515 (+ 10317us) service_pool.cc:224] Handling call
0709 13:02:52.530936 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.676865 (+1145929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.259591 (+582726us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540011f8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:54.259852 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2631536896}
I20260709 13:02:54.306198  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.307149  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:54.157532  5999 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.307744  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:54.157068  6109 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.315156  5780 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:54.286120  6094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.007s	sys 0.011s
W20260709 13:02:54.054644  1469 rpcz_store.cc:269] 0709 13:02:23.158429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.166139 (+  7710us) service_pool.cc:224] Handling call
0709 13:02:25.534523 (+2368384us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:02:54.053798 (+28519275us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:45.459916 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:45.556918 (+ 97002us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:45.556969 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:02:45.556988 (+    19us) tablet.cc:662] Decoding operations
0709 13:02:45.568286 (+ 11298us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:45.568389 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:45.568420 (+    31us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:02:45.641863 (+ 73443us) tablet.cc:701] Row locks acquired
0709 13:02:45.641915 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:02:45.642117 (+   202us) write_op.cc:270] Start()
0709 13:02:45.645478 (+  3361us) write_op.cc:276] Timestamp: P: 1783602165642037 usec, L: 0
0709 13:02:45.649110 (+  3632us) op_driver.cc:348] REPLICATION: starting
0709 13:02:45.649886 (+   776us) log.cc:844] Serialized 11104 byte log entry
0709 13:02:46.436244 (+786358us) op_driver.cc:464] REPLICATION: finished
0709 13:02:47.123367 (+687123us) write_op.cc:301] APPLY: starting
0709 13:02:47.123474 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:47.392187 (+268713us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:47.392406 (+   219us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:48.541579 (+1149173us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:48.555576 (+ 13997us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 13:02:48.555664 (+    88us) write_op.cc:312] APPLY: finished
0709 13:02:48.556563 (+   899us) log.cc:844] Serialized 244 byte log entry
0709 13:02:48.681350 (+124787us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.682608 (+  1258us) write_op.cc:454] Released schema lock
0709 13:02:48.682831 (+   223us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:48.693519 (+ 10688us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.693761 (+   242us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":60962,"reported_tablets":52,"spinlock_wait_cycles":70651776,"child_traces":[["op",{"apply.queue_time_us":667478,"apply.run_cpu_time_us":65894,"apply.run_wall_time_us":1565879,"cfile_cache_hit":110,"cfile_cache_hit_bytes":115115,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6220,"cfile_init":1,"lbm_read_time_us":49730,"lbm_reads_1-10_ms":3,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":4,"num_ops":28,"prepare.queue_time_us":490260,"prepare.run_cpu_time_us":18517,"prepare.run_wall_time_us":437565,"raft.queue_time_us":446966,"raft.run_cpu_time_us":30209,"raft.run_wall_time_us":328223,"replication_time_us":786955,"thread_start_us":1272009,"threads_started":5,"wal-append.queue_time_us":280719,"wal-append.run_cpu_time_us":39705,"wal-append.run_wall_time_us":2636030}]]}
I20260709 13:02:54.310247  6051 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:54.319175  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.283756  6115 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.269848  5698 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:54.316175  6058 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:54.319464  6010 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:54.321485  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:54.324805  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 242750us
I20260709 13:02:54.338371  2380 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:54.346508  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:54.316876  1473 catalog_manager.cc:5697] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 7, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 7 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:54.341639  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.343189  2379 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:02:54.319890  6094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.007s	sys 0.011s
I20260709 13:02:54.378203  1642 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:54.344444  6054 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.409548  6054 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.409225  1473 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 6 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:54.337453  6006 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:02:54.412318  5999 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.413009  6006 consensus_queue.cc:1243] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:54.335465  6109 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:54.284114  6147 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.411851  1473 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 5, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 5 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:54.415446  1473 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.418682  1473 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 6, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.423441  6057 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:54.342289  6115 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.425103  5961 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.429121  5961 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.426944  6063 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.287446  5925 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.432960  6111 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:54.435503  5961 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.329056  6014 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:54.436789  6014 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.423781  1473 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 6 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:54.439386  6014 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:54.339210  5698 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.440642  1473 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 5 to 6, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:02:54.443244  6147 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:54.413136  5999 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:54.444427  5999 consensus_queue.cc:1243] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:54.444948  5999 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:54.445318  5999 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:54.445689  6078 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:54.446918  6153 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:02:54.312602  5837 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.451260  5837 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.342461  6058 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:54.461479  6058 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:54.463696  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:54.466336  6150 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.510s	user 0.001s	sys 0.000s
I20260709 13:02:54.468575  6057 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:54.470875  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:54.476312  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.436158  5925 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:54.477051  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.477771  1651 raft_consensus.cc:2376] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:54.477900  6054 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.478858  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.458339  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.459578  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.488673  1645 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
W20260709 13:02:54.488453  1690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.716s (SENT) [suppressed 48 similar messages]
I20260709 13:02:54.460057  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.460287  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.337146  6009 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:54.490692  1648 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
W20260709 13:02:54.499514  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.499989  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.446378  5999 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.491737  1649 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.479115  6115 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.508499  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.480374  6010 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.512571  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.525593  1650 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:02:54.499199  6009 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:54.523623  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.443020  1473 catalog_manager.cc:5697] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:02:54.579954  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.011s	sys 0.016s
W20260709 13:02:54.580677  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.011s	sys 0.016s
W20260709 13:02:54.596292  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.596946  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:54.607009  2380 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:54.610397  1722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:54.611589  1722 raft_consensus.cc:606] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:54.611619  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:54.612533  1722 raft_consensus.cc:614] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:54.612926  1718 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:54.561728  6010 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.575829  6009 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.522541  6146 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:54.638511  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.639339  6087 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:54.641903  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.646212  6008 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.649186  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.591076  5999 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.653231  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:54.653795  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.654357  1721 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:54.654510  1647 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:54.663635  1649 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:02:54.665110  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2181) took 1059 ms. Trace:
I20260709 13:02:54.672039  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.674410  1644 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:54.678025  1651 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:02:54.665565  1649 rpcz_store.cc:276] 0709 13:02:53.605744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.463378 (+857634us) service_pool.cc:224] Handling call
0709 13:02:54.665033 (+201655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:54.648123  6003 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.678368  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.693178  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:54.686264  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:54.682762  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 348940us
I20260709 13:02:54.706126  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:54.696050  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.674763  1574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:02:54.708364  1574 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:54.708840  1574 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:54.710805  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:54.705560  2380 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:54.712836  1571 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:54.659255  5698 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.682801  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.692114  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.721238  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.721457  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.722373  1647 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.698267  1567 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
W20260709 13:02:54.726855  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 380450us
I20260709 13:02:54.702431  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:54.703842  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.728404  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:54.726344  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:54.730321  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:54.696223  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.734247  1651 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:54.743844  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:54.745715  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:54.747036  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.747786  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.745708  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:54.678869  6058 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.756330  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:54.772439  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:54.762444  1574 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.764602  6065 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.775075  1646 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:54.767740  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.801702  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.802484  1649 raft_consensus.cc:2376] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:54.803252  1645 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:54.754043  6164 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:54.807860  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.811025  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:54.814002  1643 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:54.766702  6057 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:54.814628  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.775650  6115 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:54.822903  1644 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:54.826062  1649 raft_consensus.cc:2376] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:54.821659  1575 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:54.801414  6140 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:54.786430  6035 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.838964  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.840590  1644 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:54.718173  6003 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.776003  5961 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:54.865190  1574 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:54.812126  6065 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:54.872476  6065 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:54.884057  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.885057  1649 raft_consensus.cc:2368] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12.
I20260709 13:02:54.829665  6115 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.887511  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:54.892352  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:54.893276  1644 raft_consensus.cc:2376] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14. Re-sending same reply.
I20260709 13:02:54.897704  1571 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:54.854621  5961 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.857329  6146 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.850677  6172 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:54.888037  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 253265us
I20260709 13:02:54.875519  6063 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:54.915220  6063 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:54.866658  6140 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:54.894085  6115 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.944526  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:54.945731  1574 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:54.851913  6035 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:54.954173  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.909457  5961 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:54.949632  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 177322us
I20260709 13:02:54.955602  1644 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:02:54.962492  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.957631  1650 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:54.953890  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:54.965993  1574 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:54.978389  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 174889us
I20260709 13:02:54.979349  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:54.964067  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:55.001745  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.916211  6058 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:54.968765  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:54.979768  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:55.004006  1574 raft_consensus.cc:2376] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:02:55.013159  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:55.031078  1574 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:55.067358  1642 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:55.013724  6077 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:55.073736  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:55.075881  1724 raft_consensus.cc:606] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:55.076361  1724 raft_consensus.cc:614] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:55.079152  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.917685  6140 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.095137  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.011559  6065 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:02:55.108353  1648 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:55.010951  6078 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:02:55.114135  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:55.115917  1724 raft_consensus.cc:606] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:55.116405  1724 raft_consensus.cc:614] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:55.118044  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 153838us
I20260709 13:02:55.120126  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:55.116271  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.125468  1648 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:55.126986  6035 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:55.127254  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:02:55.127864  1644 raft_consensus.cc:606] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:55.128203  1644 raft_consensus.cc:614] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:55.096474  5698 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:55.131686  6065 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.119557  6165 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
I20260709 13:02:55.132615  5837 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:55.132298  6065 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.133311  5837 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:55.134338  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:02:54.962257  6063 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.135140  1724 raft_consensus.cc:606] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:55.097445  6183 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:55.113977  6077 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:55.155067  1647 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:02:55.148233  6010 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.002s
I20260709 13:02:55.169899  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.170725  1647 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:55.146435  6035 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.167430   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:55.175770   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:55.161374  6183 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:55.180224  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.182446  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.184039  1647 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:55.181890  1648 raft_consensus.cc:2376] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:55.167472  6010 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.190604  1642 raft_consensus.cc:2368] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12.
W20260709 13:02:55.181324  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 238261us
W20260709 13:02:55.195220  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.200321  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.202154  1647 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:02:55.204101  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 191032us
I20260709 13:02:55.207199  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:55.189774  6010 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:55.177881  6035 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.213081  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.172595  5837 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.215044  1647 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:55.208588  6180 raft_consensus.cc:2764] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:55.223742  5961 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.224210  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.224869  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.248739  1644 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:55.249807  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.250780  1644 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:55.225276  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.224926  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.252547  1651 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:55.232507  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.253127  1648 raft_consensus.cc:2376] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10. Re-sending same reply.
I20260709 13:02:55.236925  6183 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.258499  6051 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:55.259011  6051 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.259395  6051 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:55.208065  6003 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:55.244196  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.261859  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.262394  1647 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:55.262631  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.263152  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.205875  6190 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:55.208462  6111 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:55.213316  6065 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.272933  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.265409  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.274223  1651 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:02:55.266570  1569 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:55.247761  1645 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:02:55.275089  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 155050us
W20260709 13:02:55.247850  5999 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.587s	user 0.013s	sys 0.000s
I20260709 13:02:55.319092  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.320016  1645 raft_consensus.cc:2376] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10. Re-sending same reply.
I20260709 13:02:55.277556  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:55.279891  1647 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:02:55.249643  6009 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.004s
I20260709 13:02:55.283821  5961 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.284694  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.350750  1649 raft_consensus.cc:2376] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:55.291002  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.352216  1644 raft_consensus.cc:2376] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10. Re-sending same reply.
I20260709 13:02:55.292683  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.353839  1651 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:02:55.276134  6178 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
W20260709 13:02:55.298146  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.320732  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:55.321973  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44520.1us
W20260709 13:02:55.349252  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 142150us
I20260709 13:02:55.357688  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:55.361756  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:55.364501  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:55.369732  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:55.370573  1721 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:02:55.371750  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:55.372076  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.374109  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.210155  6077 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.375100  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:55.375759  1722 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:55.285185  6122 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:02:55.285549  6111 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.354280  6142 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:55.385459  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.392227  2379 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:55.280642  6190 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:55.387518  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 66889.6us
W20260709 13:02:55.385216  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.021s (SENT)
I20260709 13:02:55.398067  1721 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:55.389052  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:55.401070  2379 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:55.384920  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.377378  5961 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:55.405524  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.407882  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:55.409057  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:55.409752  6199 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:55.409806  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:55.410168  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:55.411000  6199 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:55.411556  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:55.412524  6193 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:55.412705  6180 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:55.413517  6003 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:55.385834  6087 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:55.414578  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:55.419764  5999 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.420501  6180 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:55.425804  6180 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:55.393761  6179 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:02:55.422343  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.357324  6009 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.425879  6035 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.420477  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.427234  1647 raft_consensus.cc:2376] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:02:55.394254  6142 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:02:55.430111  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72494.4us
I20260709 13:02:55.435091  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:55.437229  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.437975  1644 raft_consensus.cc:2376] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:55.438902  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.441401  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.445741  1645 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:55.417805  6140 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:55.421725  6078 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:55.450278  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:55.455716  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.417091  6198 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:55.478909  1644 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:55.429896  5999 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:55.474007  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.406508  6197 raft_consensus.cc:2764] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:02:55.493786  6188 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:02:55.494560  2380 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:02:55.442214  6199 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:55.495550  6172 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:55.436215  6142 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.480768  1644 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:55.486006  1647 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 7: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:55.423465  5460 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.488912  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:55.428808  5961 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.473723  6003 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.433887  6087 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:55.440729  6193 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.479784  6078 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.490279  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.477133  6181 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:55.481865  6140 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.522898  6181 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:55.501556  6186 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:02:55.497399  5999 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.528152  6009 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.530508  5961 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.531605  1645 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:02:55.502465  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.497538  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:55.508286  6190 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:55.518479  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.517818  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.488826  6180 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.524032  6181 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.539872  5961 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.569568  6180 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.563141  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:55.570760  2371 raft_consensus.cc:2410] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:02:55.570824  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.571712  2375 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:55.566473  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.532938  2378 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:02:55.563802  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.532960  1647 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:55.539911  2380 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:55.496166  6199 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.573074  6111 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.566303  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.574080  6051 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:55.526145  6146 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 13:02:55.542747  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:55.580945  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 143981us
I20260709 13:02:55.581718  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.582782  1645 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:55.582511  1644 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:55.584510  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:55.585386  1649 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.578044  6048 raft_consensus.cc:993] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:55.533365  6078 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:55.499027  5460 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.580376  6008 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.542048  5999 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.583683  6146 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:55.591321  6115 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.003s
I20260709 13:02:55.580865  6208 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.611546  1649 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:55.614472  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:55.623250  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:55.593534  6054 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:55.624675  2380 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.625566  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:55.629539  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.631047  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.631858  1569 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 8: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:55.619686  1647 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:02:55.593513  6181 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.622300  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.641373  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.641783  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.642076  1570 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:55.643812  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:55.646955  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:55.612268  6190 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.648922  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.649549  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:55.649577  1572 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:02:55.642391  1569 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:55.653180  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.652275  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:55.653937  1568 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:55.654834  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:55.655155  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:55.656421  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:55.656944  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.657550  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:55.633072  5999 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:55.660305  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77447.7us
I20260709 13:02:55.625486  6193 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.660712  1573 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:55.544888  6206 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:02:55.664746  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:55.671347  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.671422  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:55.634447  6054 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.635051  6003 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:02:55.625190  6115 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.632916  6208 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.631326  6087 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:55.634933  6084 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:55.737317  2378 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:55.661756  6111 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:55.922027  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.663518  5999 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:55.716584  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.967828  1572 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
W20260709 13:02:55.736147  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.717231  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:55.734011  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.673197  1576 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:55.736047  2380 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:55.793483  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:55.755086  1724 raft_consensus.cc:1217] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Deduplicated request from leader. Original: 5.3->[9.4-10.5]   Dedup: 9.4->[10.5-10.5]
I20260709 13:02:55.963903  1567 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.970705  1724 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [9.4-10.5]
I20260709 13:02:55.746742  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:55.896737  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 231370us
W20260709 13:02:55.973555  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.871330  6051 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:55.972925  2377 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:55.697965  6140 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:55.972996  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:55.760802  6035 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:55.880802  6103 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:55.963591  2378 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:55.961683  6084 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:55.912606  6213 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:02:55.761112  6122 raft_consensus.cc:1081] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:02:56.000692  1567 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:55.671686  6172 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.662319  6197 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:55.874915  6091 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:55.787112  6063 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:55.903949  5460 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:55.970093  6164 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:55.996716  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:55.967046  5999 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.977999  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.027947  1649 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:56.001686  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:55.661996  6190 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.021085  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.027971  6091 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:55.989032  6003 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:56.064437  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.065276  1649 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.015710  6197 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:56.071400  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278015us
I20260709 13:02:56.043144  5999 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:56.082276  6173 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.079s	user 0.000s	sys 0.011s
W20260709 13:02:56.077872  6223 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:02:55.995549  6035 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.063771  5961 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:56.027776  6164 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:56.005244  6196 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 13:02:56.089586  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.001917  6103 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.118484  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.119380  1721 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.091717  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:56.024814  6140 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.085206  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.040633  6190 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.076882  6197 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:56.098064  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.102051  6035 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.115176  1649 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.042773  6063 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.123024  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:02:56.134860  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 156220us
I20260709 13:02:56.125396  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:56.127327  6115 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.097827  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.147425  1724 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:56.097777  6218 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:02:56.152820  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.154673  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:02:56.128527  5961 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:56.156672  6054 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.194993  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1965) took 1490 ms. Trace:
I20260709 13:02:56.195696  6115 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.196305  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1967) took 1473 ms. Trace:
I20260709 13:02:56.196651  1571 rpcz_store.cc:276] 0709 13:02:54.723108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.755133 (+ 32025us) service_pool.cc:224] Handling call
0709 13:02:54.755638 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.898392 (+142754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.196080 (+1297688us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54000c2e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:56.196249 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307586432}
I20260709 13:02:56.197645  6063 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:56.199893  6142 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.703s	user 0.006s	sys 0.005s
I20260709 13:02:56.159240  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.200402  6142 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.202373  2378 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:56.203495  6142 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.204970  6196 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:56.205535  6196 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.206645  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.207367  1651 raft_consensus.cc:2376] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12. Re-sending same reply.
I20260709 13:02:56.162317  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:56.158735  6234 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:56.164681  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 644) took 1910 ms. Trace:
W20260709 13:02:56.217173  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 184783us
W20260709 13:02:56.166432  6008 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.005s
I20260709 13:02:56.180743  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.181604  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.185211  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 651) took 1903 ms. Trace:
I20260709 13:02:56.186208  6103 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:56.186388  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.187788  5698 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:56.188668  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1733) took 1114 ms. Trace:
W20260709 13:02:56.190344  6205 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.699s	user 0.001s	sys 0.001s
W20260709 13:02:56.191340  6180 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.000s
I20260709 13:02:56.192606  6084 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.134076  6140 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.193228  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.194621  6208 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:56.158154  1649 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.210173  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:56.211828  6200 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.719s	user 0.000s	sys 0.002s
I20260709 13:02:56.196208  6164 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.195783  6091 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:56.184056  6078 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.002s
I20260709 13:02:56.171826  5999 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.217599  1643 rpcz_store.cc:276] 0709 13:02:54.254254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.310489 (+ 56235us) service_pool.cc:224] Handling call
0709 13:02:54.310939 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.817993 (+507054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.164435 (+1346442us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54001bc1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:56.164619 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141955840}
I20260709 13:02:56.449955  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:56.453002  2377 raft_consensus.cc:2368] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12.
I20260709 13:02:56.453269  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.297644   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:56.223989  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.454716  1644 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:02:56.451413  1688 leader_election.cc:336] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.078s (SENT)
I20260709 13:02:56.449954  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.456008  1645 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:56.452540  1642 rpcz_store.cc:276] 0709 13:02:55.074495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.094798 (+ 20303us) service_pool.cc:224] Handling call
0709 13:02:56.188431 (+1093633us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b540038b8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:56.188587 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212005760}
W20260709 13:02:56.407202  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.456212  1721 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:02:56.457185  1688 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.135s (SENT)
I20260709 13:02:56.223831  1724 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
W20260709 13:02:56.249024  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 86839.8us
I20260709 13:02:56.459151  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:56.455600  1720 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.459745  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:56.195382  1575 rpcz_store.cc:276] 0709 13:02:54.704449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.713000 (+  8551us) service_pool.cc:224] Handling call
0709 13:02:54.758160 (+ 45160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.831703 (+ 73543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.194756 (+1363053us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b54002a6cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:56.194935 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145553280}
I20260709 13:02:56.220054  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.223273  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:02:56.235605  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75987.3us
I20260709 13:02:56.457409  1646 rpcz_store.cc:276] 0709 13:02:54.282370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.316108 (+ 33738us) service_pool.cc:224] Handling call
0709 13:02:54.321950 (+  5842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.804074 (+482124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.184928 (+1380854us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400306e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:56.185145 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041058176}
W20260709 13:02:56.466202  1688 leader_election.cc:336] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.147s (SENT)
W20260709 13:02:56.467283  1688 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.199s (SENT)
I20260709 13:02:56.469633  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.471519  1569 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:56.470193  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.471971  1643 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:56.466171  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.472581  1567 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:02:56.472775  1647 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
W20260709 13:02:56.473764  1688 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.214s (SENT)
I20260709 13:02:56.478062  1644 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:56.475346  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.479866  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.480654  1721 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:56.480998  1650 raft_consensus.cc:2410] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:02:56.478799  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1761) took 1217 ms. Trace:
W20260709 13:02:56.444715  6181 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.794s	user 0.006s	sys 0.007s
W20260709 13:02:56.488057  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.481824  1644 rpcz_store.cc:276] 0709 13:02:55.261556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.477382 (+1215826us) service_pool.cc:224] Handling call
0709 13:02:56.477702 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.478518 (+   816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.478743 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.483409  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.482323  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.490613  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.493685  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.297209  5961 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:56.296028  6063 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:56.520169  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:56.288383  6196 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.497323  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:56.498119  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.500670  1724 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.373847  6142 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:56.221446  6111 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.451817  5698 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:56.501708  1642 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:56.337918  6115 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.227064  6183 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.309s
I20260709 13:02:56.223477  6199 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.503474  1651 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:56.503548  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.505769  1572 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:56.508387  1568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:02:56.561529  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1769) took 1297 ms. Trace:
I20260709 13:02:56.486836  6103 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:56.477537  6082 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.513525  5999 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.501250  6010 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.221633  6057 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:56.507764  1649 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.504591  5961 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:56.518824  6219 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:02:56.496304  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.521535  1646 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:56.471310  6078 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.530700  1716 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:02:56.421597  6003 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:56.571682  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2192) took 1002 ms. Trace:
I20260709 13:02:56.467046  6172 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:56.537986  1721 raft_consensus.cc:2410] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:02:56.521193  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:56.579370  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58251.6us
I20260709 13:02:56.541757  1718 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:02:56.547055  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1764) took 1284 ms. Trace:
I20260709 13:02:56.487561  6193 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.373500  6231 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.550128  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.567365  6239 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.558862  1645 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:02:56.559584  1719 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:02:56.559824  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.644766  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2184) took 1157 ms. Trace:
I20260709 13:02:56.645481  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.562122  1568 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:02:56.645773  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.562099  1651 rpcz_store.cc:276] 0709 13:02:55.263807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.478039 (+1214232us) service_pool.cc:224] Handling call
0709 13:02:56.500727 (+ 22688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.561130 (+ 60403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.561440 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.483450  6091 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:02:56.561427  1644 raft_consensus.cc:2376] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:02:56.565773  6103 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:56.660530  6103 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.534816  6008 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.645874  1645 rpcz_store.cc:276] 0709 13:02:55.487149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.138529 (+651380us) service_pool.cc:224] Handling call
0709 13:02:56.644646 (+506117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.573084  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1770) took 1309 ms. Trace:
I20260709 13:02:56.662869  1646 rpcz_store.cc:276] 0709 13:02:55.264063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.477044 (+1212981us) service_pool.cc:224] Handling call
0709 13:02:56.495784 (+ 18740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.572773 (+ 76989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.572970 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.575841  1649 rpcz_store.cc:276] 0709 13:02:55.568995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.152452 (+583457us) service_pool.cc:224] Handling call
0709 13:02:56.571615 (+419163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.570005  2378 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:56.571906  5961 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.571547  6057 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.667548  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.668202  1649 raft_consensus.cc:2376] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:02:56.669211  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.670890  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.571674  6010 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.572243  6219 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:56.682828  6219 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:56.573553  6077 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.683713  6003 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.582324  6082 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:56.684991  6082 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.685381  6082 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:56.654466  6193 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:56.687634  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.688097  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.572752  6115 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.688565  6193 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.690068  6140 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:56.608196  6172 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.690608  6115 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.586293  1724 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:56.692226  6172 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.692278  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.692679  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.693588  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.694770  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.588728  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.596637  6087 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:56.697542  5698 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.551872  6142 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.698833  5698 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:56.599495  6197 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.701346  6006 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:56.702026  6197 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.701928  6006 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.702756  1570 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.704010  6006 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.665311  6239 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.705641  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.542609  6063 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.711800  6063 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:56.607710  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.610165  1642 rpcz_store.cc:276] 0709 13:02:55.262409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.476984 (+1214575us) service_pool.cc:224] Handling call
0709 13:02:56.501422 (+ 24438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.546760 (+ 45338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.546989 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.559657  6235 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:56.614069  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139439us
W20260709 13:02:56.726810  1688 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.230s (SENT)
I20260709 13:02:56.727272  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:56.560154  6236 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:56.728900  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.729586  1642 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:56.724952  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:56.562741  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:56.651676  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.646176  1568 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.656646  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.658092  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.732863  1567 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:56.733071  1569 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:56.733819  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.734447  1567 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:02:56.658278  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1732) took 1680 ms (client timeout 1750 ms). Trace:
I20260709 13:02:56.281666  6234 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.665926  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.677345  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.735634  1644 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:56.667428  6008 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.681581  6057 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.740005  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:56.739876  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.740567  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.682191  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.741274  1567 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:56.693638  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.695617  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.742034  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.742614  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:56.576553  6199 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.707888  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:56.736510  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:56.708477  6006 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.604707  1721 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:56.684008  6077 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.713585  1716 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:02:56.752779  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.710021  6087 consensus_queue.cc:1243] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:56.675967  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:56.753543  1644 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.754477  2378 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:56.718267  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.592149  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:56.727965  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:56.735324  1650 rpcz_store.cc:269] 0709 13:02:54.977794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.979017 (+  1223us) service_pool.cc:224] Handling call
0709 13:02:56.658046 (+1679029us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540031bad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:56.658213 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3449363200}
W20260709 13:02:56.737275  1688 connection.cc:256] RPC call timeout handler was delayed by -1.8343693269999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 13:02:56.831830  1688 connection.cc:256] RPC call timeout handler was delayed by -1.7638417560000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 13:02:56.663784  6231 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.832324  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.833678  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.834064  6231 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.742941  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:56.834296  1650 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.834605  6010 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.744180  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:56.601915  6078 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.746894  1722 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:02:56.835527  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 91420.3us
I20260709 13:02:56.836545  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.837724  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.742761  1567 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:56.603395  1720 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.839104  1722 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:56.839396  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.839550  1720 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:56.747104  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.840448  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.841576  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.831772  1645 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:02:56.710531  5961 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.842662  1650 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.843427  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.843997  1645 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.844074  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.844911  5961 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:56.750525  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:56.845422  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.726430  6235 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:56.846536  6057 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.770615  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.847674  1570 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:56.770928  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.727295  6063 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:56.848968  6063 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.849495  6008 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.851164  6063 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.735008  6166 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:56.773737  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.852473  6166 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:56.852806  1569 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:56.852874  6166 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.853413  6166 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:56.854130  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 262079us
I20260709 13:02:56.775136  1649 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:56.787176  1568 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.858351  1568 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.752117  6054 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:56.683755  6219 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:56.860296  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.843037  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.861332  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.794524  1648 raft_consensus.cc:2376] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14. Re-sending same reply.
I20260709 13:02:56.755049  6197 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:56.842839  1720 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:56.738615  6146 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:02:56.768958  6111 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.764415  6142 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:56.738201  6236 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.887447  2375 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.769883  6006 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:56.815575  6199 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.784950  6239 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.825886  6234 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:56.827967  6240 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.002s
I20260709 13:02:56.834764  1718 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.915096  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.749538  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:56.846037  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95661.7us
I20260709 13:02:56.916404  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:02:56.765388  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89515.2us
I20260709 13:02:56.847434  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:56.854175  6242 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:56.751750  6227 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:56.795025  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.865072  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:56.870447  2380 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:56.923983  1651 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:56.775815  1572 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:56.877470  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:56.846858  6235 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.863773  6078 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.939396  1718 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:02:56.900374  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:56.940749  1570 raft_consensus.cc:2368] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12.
I20260709 13:02:56.941336  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.941972  1570 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:56.901584  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.902945  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:56.945030  1574 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:56.904039  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.948951  1567 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:02:56.949314  1567 pending_rounds.cc:85] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:56.949553  1567 pending_rounds.cc:107] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:02:56.949796  1567 raft_consensus.cc:2889] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.951115  1567 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:56.951015  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.940783  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.906576  6077 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:56.907905  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:56.954305  1568 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:56.954994  1572 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.955405  1568 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:56.956668  1568 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:02:56.956959  1568 pending_rounds.cc:85] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:56.957187  1568 pending_rounds.cc:107] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:02:56.957206  1567 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:56.957541  1568 raft_consensus.cc:2889] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.958554  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.959113  1568 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.909338  6084 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.962852  6006 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:56.963973  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1745) took 1739 ms (client timeout 1578 ms). Trace:
I20260709 13:02:56.964085  1572 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.964744  6236 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:56.965183  1643 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:56.965502  1572 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:56.910923  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.966650  1572 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:56.912590  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:56.880193  6219 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:56.923139  6164 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.870002  6008 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:56.969736  6008 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:56.970606  6008 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.974270  1647 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:56.975883  1647 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:02:56.976195  1647 pending_rounds.cc:85] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 6
I20260709 13:02:56.748118  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:56.918078  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:56.920950  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:56.925114  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47765.7us
I20260709 13:02:56.905638  1645 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:56.928580  1646 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:56.882453  6197 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:56.952986  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:56.876755  6065 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:02:56.913529  1688 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.556s (SENT)
W20260709 13:02:56.996237  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75374.5us
I20260709 13:02:56.996367  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:56.976634  6078 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.979481  6142 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:56.930574  6227 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.937106  6239 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:56.964442  1648 rpcz_store.cc:269] 0709 13:02:55.224413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.272533 (+ 48120us) service_pool.cc:224] Handling call
0709 13:02:56.223122 (+950589us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:02:56.963782 (+740660us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54000906d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:56.963892 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748164608}
I20260709 13:02:56.982015  6003 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.991683  1642 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:56.991955  1647 pending_rounds.cc:107] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:02:57.008673  1647 raft_consensus.cc:2889] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.984149  6210 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:56.992817  6054 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:02:56.993507  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.992986  6197 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:56.994441  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:57.034016  2374 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:56.995353  1650 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:56.998268  2379 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:56.997750  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:56.925359  1575 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:02:57.001644  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:57.036854  1718 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.020991  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:57.002233  1572 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:57.004258  1649 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:57.006179  1644 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:02:57.004101  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1772) took 1620 ms. Trace:
I20260709 13:02:57.010093  1568 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:02:57.020377  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:57.020965  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 107494us
I20260709 13:02:57.022228  1576 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:57.026664  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:57.027063  1569 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:57.008364  6231 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:02:57.027572  6210 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:57.028156  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:57.007484  6063 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:56.997488  6111 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.053864  1643 rpcz_store.cc:276] 0709 13:02:55.383455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.481709 (+1098254us) service_pool.cc:224] Handling call
0709 13:02:56.964855 (+483146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.965594 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.995994 (+ 30400us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54000906d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:57.004030 (+  8036us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110544128}
W20260709 13:02:57.058682  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60995.3us
I20260709 13:02:57.072604  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:57.060245  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.064726  1722 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:57.065442  1568 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:57.054692  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:57.057297  1576 pending_rounds.cc:85] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:02:57.078923  1576 pending_rounds.cc:107] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:57.079301  1576 raft_consensus.cc:2889] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:57.057368  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:57.054163  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:57.052593  1572 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:57.006791  6142 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:57.007788  6227 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:57.055732  1688 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.562s (SENT)
I20260709 13:02:57.094244  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:57.096896  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:02:57.097470  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.097633  1720 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:57.047164  6091 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.104952  1568 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
W20260709 13:02:57.105357  6262 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.003s
I20260709 13:02:57.105571  1568 pending_rounds.cc:85] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 4
I20260709 13:02:57.105973  1568 pending_rounds.cc:107] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:02:57.106102  6258 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:57.062218  6054 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.106336  1568 raft_consensus.cc:2889] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:57.110502  1569 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:57.067413  6172 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:57.008095  5961 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.086s
W20260709 13:02:57.116603  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44151.3us
I20260709 13:02:57.113397  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:57.007021  6082 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.088722  6063 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:57.089962  1718 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:57.120985  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:57.046895  6146 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:02:57.124539  2380 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:57.124995  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:57.096491  6010 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:02:57.125856  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.128001  6082 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:57.128369  1568 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:57.046934  6003 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.129906  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.130792  1568 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:02:57.133584  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102831us
I20260709 13:02:57.138108  1572 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:57.137802  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:57.115835  6168 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.060s	sys 0.059s
I20260709 13:02:57.142277  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.143199  1572 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:57.093593  6077 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:02:57.119153  6219 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.096678  5698 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:02:57.124200  6057 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.100399  6111 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.154963  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:57.122946  6063 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.163733  1574 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:02:57.131947  6193 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:57.124468  6172 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:57.166356  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 110676us
W20260709 13:02:57.147891  6168 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.061s	sys 0.059s
I20260709 13:02:57.178876  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:57.146919  5999 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:57.186138  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.182886  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96086.4us
I20260709 13:02:57.164007  6054 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:57.255872  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.260046  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.170594  6057 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:57.260804  6057 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.264008  6111 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:57.264964  6183 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:57.265377  6183 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:57.266271  6183 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.267408  6183 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:57.268393  1719 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.165184  6103 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.270401  6103 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:57.271550  6103 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.275660  6166 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.240113  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:57.240962  6232 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:57.244006  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:57.298857  1645 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:57.244822  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:57.246793  6138 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:57.277086  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:57.319980  1719 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:57.171710  6210 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.141129  6142 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.277678  6087 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:57.280469  6122 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.280669  2380 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:57.281399  2379 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:57.281491  2377 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:57.284438  6193 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:57.335232  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.237759  1572 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:57.293035  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.296895  1688 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.662s (SENT)
W20260709 13:02:57.388065  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 91215.9us
W20260709 13:02:57.388767  1688 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.689s (SENT)
W20260709 13:02:57.311174  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.390457  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.319640  1644 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:57.298321  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:57.267241  6003 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:57.286317  1568 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:57.392038  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 105459us
I20260709 13:02:57.397498  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:57.406244  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.298328  6232 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:57.408314  1688 connection.cc:256] RPC call timeout handler was delayed by -1.6819417560000147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 13:02:57.413890  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.418254  1688 connection.cc:256] RPC call timeout handler was delayed by -1.6775224149999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.49400000000000005s for a response
W20260709 13:02:57.418651  1688 connection.cc:256] RPC call timeout handler was delayed by -1.6306417560000455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 13:02:57.419020  1688 connection.cc:256] RPC call timeout handler was delayed by -1.5950934769999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.313s for a response
W20260709 13:02:57.419327  1688 connection.cc:256] RPC call timeout handler was delayed by -1.5464934770000127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3184s for a response
W20260709 13:02:57.419710  1688 connection.cc:256] RPC call timeout handler was delayed by -1.5183054839999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
W20260709 13:02:57.420637  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.421221  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.421571  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.421922  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.320876  6138 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:57.428646  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.430725  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:57.432833  1688 connection.cc:256] RPC call timeout handler was delayed by -1.3340394879999735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
W20260709 13:02:57.433271  1688 connection.cc:256] RPC call timeout handler was delayed by -1.3141028840000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 13:02:57.433663  1688 connection.cc:256] RPC call timeout handler was delayed by -1.2557934769999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3507s for a response
W20260709 13:02:57.434129  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1593028840000557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
I20260709 13:02:57.434075  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:57.434551  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1575028840000527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 13:02:57.434844  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0783028840000384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
I20260709 13:02:57.436280  1572 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:02:57.439311  1725 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:57.439643  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.437487  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:57.440363  1576 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:57.443886  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:57.455171  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 330281us
I20260709 13:02:57.458297  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:57.462126  1616 leader_election.cc:400] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12.
I20260709 13:02:57.462591  1616 leader_election.cc:403] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:02:57.399634  6077 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.467604  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:57.383236  6219 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 13:02:57.456152  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:57.512027  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 56196.5us
W20260709 13:02:57.465903  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 328180us
I20260709 13:02:57.473590  1568 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:57.509370  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1982) took 1039 ms. Trace:
I20260709 13:02:57.514125  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:57.506167  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.512813  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:57.515872  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:57.513819  1571 rpcz_store.cc:276] 0709 13:02:56.469705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.470907 (+  1202us) service_pool.cc:224] Handling call
0709 13:02:56.482071 (+ 11164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.712372 (+230301us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:57.008045 (+295673us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.439838 (+431793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.439889 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:57.439936 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.509057 (+ 69121us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54003642d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:57.509128 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.509288 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":128039,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":166157,"spinlock_wait_cycles":548462848,"thread_start_us":726639,"threads_started":3,"wal-append.queue_time_us":151983}
I20260709 13:02:57.504877  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:57.524255  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.461751  6010 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:57.538411  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75687.2us
I20260709 13:02:57.546509  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:57.394513  6122 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:57.465901  6253 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:57.548120  6269 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.454830  6197 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.461426  6003 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.583992  1573 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:02:57.584550  1573 pending_rounds.cc:85] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:02:57.469883  5698 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.470037  6006 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.465901  6199 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:57.554548  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 92614.9us
I20260709 13:02:57.465350  6258 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.503733  6077 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.584832  1573 pending_rounds.cc:107] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:02:57.622782  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:57.623123  1573 raft_consensus.cc:2889] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:57.461196  6065 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:02:57.539959  6235 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.000s
I20260709 13:02:57.545276  6008 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12.
W20260709 13:02:57.615187  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.620450  1718 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:02:57.622452  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 76071.4us
I20260709 13:02:57.573472  5999 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:57.628159  6077 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.585943  6197 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:57.663548  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 147731us
I20260709 13:02:57.676635  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:57.677032  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 168353us
I20260709 13:02:57.601190  6274 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:57.726202  5698 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:57.726004  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 48304.3us
I20260709 13:02:57.726572  6008 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.725773  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 103210us
I20260709 13:02:57.549784  6193 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.519774  6172 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.602442  6003 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.619573  6267 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:02:57.724398  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.631212  6235 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:57.572557  6232 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.880275  2375 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:57.664793  6269 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.765182  1723 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:02:57.851029  6275 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.175s	user 0.001s	sys 0.003s
I20260709 13:02:57.888280  1571 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:57.754987  1725 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:02:57.890421  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.739748  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63876.9us
W20260709 13:02:58.011996  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.815822  6063 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.001s
I20260709 13:02:57.739763  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:57.759078  6239 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:57.921209  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.883955  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:57.877453  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.806847  6197 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:57.878619  6115 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:57.754456  6078 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:57.737910  6278 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.698877  6227 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.010s
W20260709 13:02:57.742615  6273 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:02:57.746308  6264 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 13:02:58.000378  6054 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:57.745101  6277 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:02:57.818383  6183 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.000s
W20260709 13:02:57.818902  6057 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.000s
I20260709 13:02:57.854712  6142 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.864046  6077 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.879864  6271 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.765396  6008 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.812289  6258 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.816803  6122 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:57.878751  6164 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.752758  6288 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:57.762013  6274 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:57.754524  6006 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.356s
I20260709 13:02:57.755095  6010 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.355s
W20260709 13:02:58.103116  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.096426  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.050s	sys 0.090s
I20260709 13:02:58.103829  6239 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:58.110093  6003 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:58.103307  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 363632us
I20260709 13:02:58.116155  6138 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.090332  6199 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:02:58.119115  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.119642  6287 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.120836  6291 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:58.126407  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.128947  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2273) took 1675 ms (client timeout 1926 ms). Trace:
I20260709 13:02:57.830093  6282 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.130014  1716 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:57.862474  6166 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:57.822103  6236 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:57.730877  6065 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:58.132829  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.006s	sys 0.011s
W20260709 13:02:58.182018  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.006s	sys 0.011s
W20260709 13:02:58.182396  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 610288us
I20260709 13:02:58.144872  6227 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:58.148717  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 472214us
I20260709 13:02:58.131125  6115 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:57.863546  6265 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:58.143891  6281 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.822332  5999 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.162703  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 1999) took 1323 ms. Trace:
W20260709 13:02:58.168144  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.050s	sys 0.090s
I20260709 13:02:58.145277  6278 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.171254  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:58.131099  6063 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:58.175987  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.177065  1646 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:58.148658  6054 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:58.240958  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.242321  6010 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.243150  6054 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.173384  1725 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
W20260709 13:02:58.175729  1723 rpcz_store.cc:269] 0709 13:02:56.453680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.453785 (+   105us) service_pool.cc:224] Handling call
0709 13:02:58.128749 (+1674964us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b540010e5d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:58.128878 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081627008}
I20260709 13:02:58.245167  6235 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:58.245471  6048 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.246054  6048 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.180217  6282 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:58.247845  6282 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:58.249423  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.250396  6278 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.180501  6078 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:02:58.251988  6078 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.191110  6236 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:58.252521  6078 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.254261  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.253793  6236 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.254860  6078 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.255033  1723 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:58.255470  6236 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:58.087683  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.256992  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.258631  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:58.164683  6164 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:58.260022  6164 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.260536  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:58.257616  6210 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:58.164121  6288 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:02:58.264688  6288 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.264377  6183 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:58.253916  6048 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.228839  6281 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.269423  6008 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.217880  6270 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.270885  6270 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.274217  6270 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:58.255548  6078 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:58.275684  1649 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:58.276706  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.280261  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:58.281229  1642 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:58.192852  6111 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.197283  6138 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:58.209648  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.172622  6239 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.181371  6258 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:02:58.221637  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:02:58.291678  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70136.5us
I20260709 13:02:58.292876  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:58.295514  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.161201  6271 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.232484  1575 rpcz_store.cc:276] 0709 13:02:56.839237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.053639 (+214402us) service_pool.cc:224] Handling call
0709 13:02:57.065718 (+ 12079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:57.098933 (+ 33215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.098965 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.099435 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.463608 (+364173us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:57.463650 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.463714 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.839662 (+375948us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:02:58.160557 (+320895us) raft_consensus.cc:1685] finished
0709 13:02:58.160581 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.162618 (+  2037us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23481,"raft.queue_time_us":37704,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4688,"spinlock_wait_cycles":26808704,"thread_start_us":332400,"threads_started":1,"wal-append.queue_time_us":536217}
W20260709 13:02:58.237949  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66815us
I20260709 13:02:58.152679  6057 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.284852  1649 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:58.286098  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:58.221807  6103 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:58.225054  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.297694  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:58.228078  6265 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:58.305877  1722 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:58.301035  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75990.9us
I20260709 13:02:58.302243  1569 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:58.311774  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.318231  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:58.319172  1573 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.322160  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.324767  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:58.325361  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:02:58.326522  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.328562  1568 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:58.328868  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:02:58.304610  6003 raft_consensus.cc:2764] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:58.349529  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:02:58.314594  6258 consensus_queue.cc:1243] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:58.353686  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.355844  2380 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:58.344969  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.329100  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:58.366794  1575 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:58.367605  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:58.323961  6272 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:58.189925  6291 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.345299  6146 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.390003  6146 raft_consensus.cc:2764] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:58.390868  6146 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.391598  6146 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:02:58.392146  6146 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.394363  6146 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.345490  6122 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.385934  6057 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.346731  6078 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:02:58.330449  6054 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:58.396320  5698 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:58.316159  6103 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.396703  6193 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:58.402312  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48693.6us
I20260709 13:02:58.406024  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.291390  6111 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.415489  6271 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.415720  6077 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.326861  6292 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:58.416992  6292 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:58.400259  6103 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.418061  6292 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.347705  6274 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.326560  6219 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.327513  6008 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.323661  6290 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.409875  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.348402  6197 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.410372  6122 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:58.431845  6122 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:58.411397  6265 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.435079  6265 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.437371  6227 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:58.324434  6065 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:58.193266  6289 consensus_queue.cc:1175] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f737473248e4c059fa74af3b8e1f77e" responder_term: 12 status { last_received { term: 11 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:58.443188  6084 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:58.443745  6084 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:58.446630  6232 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.400769  6054 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:58.403455  6291 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.450755  6146 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:58.452539  6164 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.454043  6048 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.454623  6048 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.455235  6164 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.361063  6281 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:58.330084  6278 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.428962  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.463517  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:02:58.463948  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58065.4us
I20260709 13:02:58.348897  6210 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:02:58.409227  6193 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:58.473037  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.465381  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.421545  6166 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:58.423858  1573 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:02:58.477005  6146 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:58.459659  6111 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.480695  6063 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.421452   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:58.477078  6193 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.484660  6142 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:02:58.484848  6219 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:02:58.492789  1717 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:58.493458  6294 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:58.493953  6294 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:58.494741  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2466) took 1876 ms. Trace:
I20260709 13:02:58.495220  6294 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:02:58.485050  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2290) took 1769 ms (client timeout 2147 ms). Trace:
W20260709 13:02:58.497704  1716 rpcz_store.cc:269] 0709 13:02:56.715913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.791585 (+ 75672us) service_pool.cc:224] Handling call
0709 13:02:58.484783 (+1693198us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400232154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:58.484973 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776804096}
W20260709 13:02:58.467022  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 185902us
I20260709 13:02:58.495393  1717 rpcz_store.cc:276] 0709 13:02:56.618297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.961307 (+343010us) service_pool.cc:224] Handling call
0709 13:02:58.492545 (+1531238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.493443 (+   898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.494495 (+  1052us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400232150. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:58.494658 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3521104640}
I20260709 13:02:58.382308  6286 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:02:58.408823  6258 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:02:58.505970  1688 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.324369  6115 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.508889  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:58.509207  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:58.466779  6290 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.478581  6164 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.485060  6166 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:02:58.503499  6146 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:58.551129  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:58.549938  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 221360us
I20260709 13:02:58.507586  6278 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.557310  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:02:58.560789  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48677.4us
I20260709 13:02:58.562383  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.526401  6063 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:02:58.565620  6063 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:58.566109  6063 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:02:58.567128  6063 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.569131  6063 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.569361  6290 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:58.569850  6290 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.569461  6288 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.572588  1575 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:58.573210  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:58.522487  6142 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:02:58.574836  6048 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:58.575497  6142 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:58.576153  6142 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:58.576675  6146 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.577199  6142 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.579428  6051 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:58.579826  6051 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:58.545885  6270 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.570313  6290 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.527063  6267 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.548458  6294 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:58.604445  1473 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 218) took 46194 ms (client timeout 15000 ms). Trace:
W20260709 13:02:58.597770  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.608404  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 259976us
W20260709 13:02:58.599983  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73718.9us
I20260709 13:02:58.580650  6270 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.549445  6265 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:02:58.615207  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:58.591814  6227 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:58.591979  6199 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:58.626237  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.601646  6166 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:58.626820  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.620219  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:58.610008  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:02:58.605197  1473 rpcz_store.cc:269] 0709 13:02:12.402452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.402642 (+   190us) service_pool.cc:224] Handling call
0709 13:02:58.596842 (+46194200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:54.761717 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:54.762782 (+  1065us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:54.762857 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:02:54.762885 (+    28us) tablet.cc:662] Decoding operations
0709 13:02:54.860801 (+ 97916us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:54.860976 (+   175us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:54.861023 (+    47us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:02:54.863251 (+  2228us) tablet.cc:701] Row locks acquired
0709 13:02:54.863330 (+    79us) write_op.cc:260] PREPARE: finished
0709 13:02:54.863676 (+   346us) write_op.cc:270] Start()
0709 13:02:54.863895 (+   219us) write_op.cc:276] Timestamp: P: 1783602174863566 usec, L: 0
0709 13:02:54.863934 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:02:54.874258 (+ 10324us) log.cc:844] Serialized 7560 byte log entry
0709 13:02:55.490984 (+616726us) op_driver.cc:464] REPLICATION: finished
0709 13:02:55.724132 (+233148us) write_op.cc:301] APPLY: starting
0709 13:02:55.754777 (+ 30645us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:56.571938 (+817161us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:56.572063 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:57.419083 (+847020us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:57.429460 (+ 10377us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 13:02:57.429530 (+    70us) write_op.cc:312] APPLY: finished
0709 13:02:57.430017 (+   487us) log.cc:844] Serialized 172 byte log entry
0709 13:02:57.732449 (+302432us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:57.750690 (+ 18241us) write_op.cc:454] Released schema lock
0709 13:02:57.750786 (+    96us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:57.755714 (+  4928us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:57.755750 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2426881,"reported_tablets":53,"spinlock_wait_cycles":1792,"child_traces":[["op",{"apply.queue_time_us":164028,"apply.run_cpu_time_us":66844,"apply.run_wall_time_us":2037403,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":114586,"prepare.run_cpu_time_us":66102,"prepare.run_wall_time_us":400541,"raft.queue_time_us":191930,"raft.run_cpu_time_us":9744,"raft.run_wall_time_us":159366,"replication_time_us":584549,"spinlock_wait_cycles":1820800,"thread_start_us":789834,"threads_started":5,"wal-append.queue_time_us":721531,"wal-append.run_cpu_time_us":20942,"wal-append.run_wall_time_us":1151796}]]}
W20260709 13:02:58.651089  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278613us
I20260709 13:02:58.617002  6270 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.658052  6172 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.643175  6227 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.674381  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:58.674202  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:02:58.676978  2380 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:58.677119  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 67251.3us
W20260709 13:02:58.677625  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45760.2us
I20260709 13:02:58.678644  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.580687  6051 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.647449  6288 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:58.685642  6164 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.649888  6183 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:58.687855  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:02:58.684415  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:58.696977  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.680778  6219 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.682956  6227 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.651348  6199 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.671298  6078 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:02:58.702885  5698 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:58.724895  6278 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:58.734498  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:02:58.782128  1574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2004) took 1506 ms (client timeout 1681 ms). Trace:
W20260709 13:02:58.776602  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97943.6us
I20260709 13:02:58.778707  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:58.766453  2380 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:02:58.782661  1574 rpcz_store.cc:269] 0709 13:02:57.275415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.276843 (+  1428us) service_pool.cc:224] Handling call
0709 13:02:58.781923 (+1505080us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54004218d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:02:58.782065 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3056891648}
I20260709 13:02:58.783329  1571 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:58.786141  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.785166  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.788619  1719 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:58.785049  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:58.795493  6290 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.796702  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:58.797644  6111 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:58.798077  6111 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.798521  6111 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.798815  6290 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:58.800724  1717 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:02:58.803663  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.805500  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 71138.3us
I20260709 13:02:58.807169  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:58.692147  6239 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.708119  6122 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:02:58.764312  6051 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.844869  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:58.845676  1723 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.817317  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:58.847453  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:58.730650  5999 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:02:58.848373  1571 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:58.845527  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.856429  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.035s (SENT)
I20260709 13:02:58.857115  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:58.734308  6288 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:02:58.770066  5698 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.809267  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:58.861873  1569 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:58.862030  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52999.7us
W20260709 13:02:58.861056  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 166298us
I20260709 13:02:58.802062  6077 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:58.838238  6292 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:58.767406  6286 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:02:58.780551  6235 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:58.738056  6219 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:58.839335  6278 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:02:58.886689  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97969.2us
I20260709 13:02:58.743201  6282 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.864799  6313 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:58.867165  6303 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.260s	user 0.000s	sys 0.002s
W20260709 13:02:58.881657  6307 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:02:58.871889  5698 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.868057  6048 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.923257  1723 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:02:58.880896  6077 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:58.941813  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49147.4us
I20260709 13:02:58.931654  1718 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:02:58.947716  1688 leader_election.cc:400] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12.
I20260709 13:02:58.948621  1688 leader_election.cc:403] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:58.910132  6301 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:02:58.893735  6300 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:58.891528  6103 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.858364  6168 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:02:58.865149  6306 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:02:58.945302  6048 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:58.963064  6316 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:02:58.965309  6210 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:58.957924  6122 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:02:58.967651  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2479) took 1709 ms. Trace:
I20260709 13:02:58.893600  6235 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.941578  6065 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:02:58.985836  6239 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:58.988389  6236 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.007s
I20260709 13:02:58.988858  6199 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:02:58.989962  6318 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:02:58.972028  6320 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:02:58.981489  1722 rpcz_store.cc:276] 0709 13:02:57.257744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.258348 (+   604us) service_pool.cc:224] Handling call
0709 13:02:57.258690 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.306563 (+1047873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.967341 (+660778us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400214154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:58.967572 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407874816}
I20260709 13:02:58.993824  6183 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:58.994306  6183 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:02:58.994680  6138 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.002s
I20260709 13:02:58.995224  6183 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.981200  6210 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:58.996726  6328 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12.
I20260709 13:02:58.997320  6210 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.998898  6210 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.000900  6282 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:59.001104  6077 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:58.963632  6103 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.005571  6219 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:59.007299  6235 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:58.981580  6048 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.948371  6140 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:59.030637  6111 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.031994  6084 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.032878  6166 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:59.033509  6084 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.008893  6278 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.035159  6227 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.037694  6111 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:58.984501  6232 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.005s
I20260709 13:02:59.039038  6267 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:59.032020  6193 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
I20260709 13:02:59.040061  6270 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.061802  6227 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:59.056973  6103 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:59.063957  6270 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:59.057094  6193 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.067374  6193 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:59.068851  6323 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:02:59.059545  6172 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:59.071182  6324 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.106s	user 0.004s	sys 0.000s
I20260709 13:02:59.117229  1573 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:02:59.119223  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.122229  1571 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:59.128202  1570 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.127975  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.129940  1571 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.132277  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:59.133538  1571 raft_consensus.cc:2376] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 9. Re-sending same reply.
I20260709 13:02:59.135664  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:58.946389  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:59.137229  1571 raft_consensus.cc:2376] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 10. Re-sending same reply.
I20260709 13:02:59.138804  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:59.141999  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 194310us
I20260709 13:02:59.143275  1649 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:59.144654  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.146065  1643 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.146247  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:02:59.147639  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.149184  1571 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:59.150856  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.152392  1571 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.154129  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:59.138504  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 192173us
I20260709 13:02:59.166429  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.167624  1717 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.166571  1722 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:59.175931  1571 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:59.182417  6326 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:59.183620  6326 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:02:59.191347  6294 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.192678  6294 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.209782  6220 raft_consensus.cc:2764] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:02:59.209968  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.211297  1643 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:59.212321  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.202976  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.198294  6235 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.216857  6235 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.221189  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.222569  1569 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.223908  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.226581  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:59.188719  6326 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.233850  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.200214  6294 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:59.221207  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.245764  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.239401  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2019) took 1065 ms. Trace:
I20260709 13:02:59.241310  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.248696  2379 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:02:59.249135  2375 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:59.249641  1725 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.248071  1576 rpcz_store.cc:276] 0709 13:02:58.173797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.174012 (+   215us) service_pool.cc:224] Handling call
0709 13:02:58.174359 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.174573 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:58.193654 (+ 19081us) log.cc:844] Serialized 14 byte log entry
0709 13:02:58.799249 (+605595us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b1800080920. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:02:59.237455 (+438206us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b1800080920. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:02:59.238120 (+   665us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.238522 (+   402us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.238640 (+   118us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:59.238805 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.238915 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.239138 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":473419,"prepare.queue_time_us":1076080,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":156,"raft.queue_time_us":5429,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":8911,"spinlock_wait_cycles":11779584}
I20260709 13:02:59.258044  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.259338  1570 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:59.258159  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.269779  6220 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:59.272982  6140 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:59.272110  6220 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:02:59.275134  6140 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.275683  6140 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:59.275849  6220 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:59.273077  6227 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.276739  6220 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:59.277062  6227 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:59.278892  6325 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:59.284020  6325 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:59.263119  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:59.236294  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:59.288010  1717 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.298023  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:59.299533  2375 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:59.301832  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.303998  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.305162  1650 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.304946  6140 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.306810  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.308317  1576 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:59.308926  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.309600  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.311146  1576 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:59.309928  1575 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:02:59.278955  6227 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.307981  6220 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.313714  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.326234  6313 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:59.323931  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 61251.9us
I20260709 13:02:59.300982  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:02:59.311208  6307 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.359s	user 0.004s	sys 0.004s
I20260709 13:02:59.330080  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:59.326917  6313 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.331739  6313 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.333707  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.335052  6227 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.335424  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.325464  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.356770  1650 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:02:59.337472  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.329985  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:59.344252  6319 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.413s	user 0.005s	sys 0.003s
I20260709 13:02:59.304369  6325 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.307215  6140 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.339604  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:59.325392  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:02:59.341181  6115 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:59.341598  6271 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:59.342847  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:59.322386  6220 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.346462  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:02:59.346899  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.351606  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2204) took 2643 ms. Trace:
I20260709 13:02:59.352783  2378 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:59.354182  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.355015  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.336434  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.359596  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.359699  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.338074  6300 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:59.307955  2379 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.366173  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2018) took 1459 ms. Trace:
I20260709 13:02:59.372452  1575 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.326973  2380 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.288802  6235 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:59.377802  1643 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:02:59.382581  2371 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:02:59.382987  1650 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:02:59.383414  1574 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
W20260709 13:02:59.384189  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.013s	sys 0.011s
W20260709 13:02:59.384727  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.013s	sys 0.011s
W20260709 13:02:59.385270  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 703004us
I20260709 13:02:59.374678  1646 rpcz_store.cc:276] 0709 13:02:56.707302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.997816 (+290514us) service_pool.cc:224] Handling call
0709 13:02:56.998066 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.240772 (+1242706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.350674 (+1109902us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003fc854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:59.351130 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2711245696}
I20260709 13:02:59.373090  6271 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:59.388264  1644 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.405860  1642 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.378543  1568 rpcz_store.cc:276] 0709 13:02:57.906536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.906714 (+   178us) service_pool.cc:224] Handling call
0709 13:02:57.907158 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.329009 (+421851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.361119 (+1032110us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54003106d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:02:59.365938 (+  4819us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968728448}
I20260709 13:02:59.400844  1569 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:02:59.406287  6300 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:59.410455  6077 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:59.417115  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74511.7us
W20260709 13:02:59.420151  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 95153.4us
I20260709 13:02:59.420979  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.401002  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:59.423938  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:02:59.426285  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.407840  6271 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.431540  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.425725  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.437978  1644 raft_consensus.cc:2410] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:02:59.443884  6281 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:59.444679  2376 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:59.430483  6235 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:59.445376  6235 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:59.445894  6235 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.446183  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.447178  1572 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:59.447261  1718 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:02:59.448796  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.453135  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65147us
I20260709 13:02:59.456535  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.456816  6210 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:59.457530  6210 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.457975  6210 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.438866  6325 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.465198  1572 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:59.465639  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.467535  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.471700  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.472976  2376 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.475347  2371 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:02:59.476212  6084 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:59.477615  6289 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.478071  6289 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:02:59.478230  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.479110  2376 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:02:59.479373  6289 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.444864  6281 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:59.485913  6281 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:02:59.494566  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.069s (SENT) [suppressed 12 similar messages]
I20260709 13:02:59.499099  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.500198  2376 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:02:59.499429  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.506253  6084 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:02:59.507385  6084 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.466558  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.526485  1571 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:02:59.528201  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84774.1us
W20260709 13:02:59.518479  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0598372780000318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:02:59.545714  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.555011  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.569386  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:59.570266  2374 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.521031  6084 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:02:59.571998  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.573515  1569 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.527096  6289 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.589115  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.596915  1717 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:59.601114  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.593950  1719 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:02:59.614982  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.620288  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:02:59.626722  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 170735us
W20260709 13:02:59.628208  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 196901us
I20260709 13:02:59.646030  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.651085  1724 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:02:59.654848  6172 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.655817  6172 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.658704  6172 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:02:59.669389  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 73794.1us
I20260709 13:02:59.605139  6313 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.677441  1725 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:02:59.683107  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:02:59.685704  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.702961  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.635496  6199 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:02:59.739192  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52733.1us
I20260709 13:02:59.740597  2376 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:02:59.717077  6270 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:59.742729  6270 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:02:59.749452  6313 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:59.744107  6270 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.741149  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:02:59.757809  6008 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:59.761438  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:59.763352  6008 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:59.763833  6008 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.764446  6008 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:59.751094  6048 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.007s
I20260709 13:02:59.768882  6048 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.770697  1722 raft_consensus.cc:2368] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12.
I20260709 13:02:59.771468  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.772634  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2027) took 1108 ms. Trace:
I20260709 13:02:59.760195  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:59.776978  6065 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:59.780970  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:59.783828  6308 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:02:59.790483  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.791347  1721 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.791386  1723 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:59.797328  6308 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:59.801327  1616 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:59.803568  6065 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:59.735507  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:02:59.764050  1614 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:02:59.806218  2374 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:02:59.773998  1567 rpcz_store.cc:276] 0709 13:02:58.593492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.594045 (+   553us) service_pool.cc:224] Handling call
0709 13:02:58.594348 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.594555 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:59.376990 (+782435us) log.cc:844] Serialized 14 byte log entry
0709 13:02:59.649012 (+272022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:59.670040 (+ 21028us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:59.685012 (+ 14972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.689135 (+  4123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:59.702055 (+ 12920us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":661972,"prepare.queue_time_us":655644,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":99,"raft.queue_time_us":21967,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8355,"thread_start_us":328571,"threads_started":2}
W20260709 13:02:59.820382  6051 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.836s	user 0.021s	sys 0.002s
I20260709 13:02:59.821462  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:02:59.828549  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 64534.2us
I20260709 13:02:59.833806  6051 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:59.834609  6051 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.835112  6051 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.838476  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.849141  6183 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:59.853607  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.854979  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:02:59.857824  6258 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:59.861349  6294 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.864550  6227 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:02:59.867183  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.900736  1649 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.869128  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:02:59.872406  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.873597  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.874698  6306 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:59.879038  6332 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:59.833547  2375 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.008s
I20260709 13:02:59.905051  2375 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:02:59.892035  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 269012us
W20260709 13:02:59.895063  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.898064  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:02:59.899909  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.907294  1643 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:59.917755  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62950.5us
I20260709 13:02:59.918164  1650 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.918831  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:02:59.920076  1647 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.920861  6294 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:02:59.922753  6308 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.923502  6308 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.924237  6294 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:02:59.925161  1646 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:59.925015  6227 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.926410  6308 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:59.927111  1644 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.927882  6227 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:02:59.906606  1573 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:02:59.963392  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:02:59.966691  1721 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:02:59.969529  6258 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:02:59.980818  6006 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:59.981540  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.981638  6006 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.984539  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:02:59.985445  2376 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:02:59.986531  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:02:59.987372  1648 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:59.997674  6172 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.998634  6172 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:02:59.999886  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:00.002509  6172 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:02:59.997291  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.004148  6308 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:00.004693  6308 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.005157  6308 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.970069  6294 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:00.020591  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:00.021690  6294 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.022662  6294 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.026501  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.026875  6294 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:00.028126  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:00.029040  1569 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:02:59.984752  6006 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.038054  1575 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.604s	user 0.014s	sys 0.002s
I20260709 13:03:00.038749  1575 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:03:00.039407  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2033) took 1230 ms. Trace:
W20260709 13:03:00.039575  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.039788  1575 rpcz_store.cc:276] 0709 13:02:58.808988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.339187 (+530199us) service_pool.cc:224] Handling call
0709 13:03:00.039334 (+700147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:00.062860  1647 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:03:00.066470  1574 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.068657  6278 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:00.068935  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.069713  1648 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:00.071456  6325 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.609s	user 0.014s	sys 0.004s
I20260709 13:03:00.075021  6336 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:00.106932  1649 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:02:59.968281  6048 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:00.123178  6048 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:00.124039  6048 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.126984  6048 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:00.127624  6271 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.004s
I20260709 13:03:00.136922  6271 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.108454  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:00.142273  1721 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:00.096746  6278 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:00.145021  6278 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.146219  2380 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:00.147742  6278 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.058075  6115 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.684s	user 0.008s	sys 0.009s
W20260709 13:03:00.128188  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 128930us
I20260709 13:03:00.109098  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.144240  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:00.076560  6336 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:00.111692  1644 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:00.165066  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.169100  1650 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:03:00.162436  6210 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.003s
I20260709 13:03:00.191169  1717 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:03:00.210461  1574 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:00.212709  1646 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.213389  1721 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:00.246872  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:00.257946  2377 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:00.268263  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:00.204792  6006 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.219274  6294 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.287140  1643 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:00.273442  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.293390  6265 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.218889  6278 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.261029  6258 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.301786  6278 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.302302  6258 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.220077  6210 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.285657  6172 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.308926  6172 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.311424  1721 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:03:00.317992  6289 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.728s	user 0.015s	sys 0.000s
W20260709 13:03:00.318271  2377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 647) took 2032 ms (client timeout 2632 ms). Trace:
W20260709 13:03:00.318856  2377 rpcz_store.cc:269] 0709 13:02:58.285486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.285695 (+   209us) service_pool.cc:224] Handling call
0709 13:03:00.317947 (+2032252us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54005d25d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:00.318180 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4321920896}
I20260709 13:03:00.320163  6199 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.321010  6199 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.321841  6006 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.323637  6199 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.326936  6008 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.562s	user 0.003s	sys 0.011s
I20260709 13:03:00.327525  6008 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.333599  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:00.287434  6065 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.338497  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:00.340544  5698 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.342s	user 0.005s	sys 0.020s
I20260709 13:03:00.340695  1568 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:03:00.341656  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:00.364428  1567 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.365798  1569 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:00.342156  6325 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.342916  1571 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:03:00.366611  1575 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:00.358711  1720 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:00.375350  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 441479us
I20260709 13:03:00.376396  1648 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:00.378818  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:00.377789  1646 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:00.380867  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 353423us
I20260709 13:03:00.380223  1576 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:00.387549  1725 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:00.386746  1574 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:00.362488  6278 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.381109  6235 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.934s	user 0.019s	sys 0.001s
W20260709 13:03:00.380859  6140 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.990s	user 0.018s	sys 0.000s
W20260709 13:03:00.394549  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.398599  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:00.390218  6051 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.002s
I20260709 13:03:00.405893  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:00.406600  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:00.411803  1576 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:03:00.356896  6294 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:00.425758  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 180204us
I20260709 13:03:00.429755  1720 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:03:00.425289  6172 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.413679  6220 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.034s	user 0.015s	sys 0.002s
I20260709 13:03:00.374289  6065 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.441862  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:00.362565  6265 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.380523  5698 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.450739  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:00.451980  1720 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:00.453473  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.457746  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:00.456698  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.406335  6325 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.408006  6235 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.441237  6172 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.472632  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:00.444886  6326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.008s	sys 0.025s
I20260709 13:03:00.442260  6006 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:00.474576  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.481612  1645 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:00.461951  6140 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:00.421418  6051 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.463625  6220 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:00.489379  6258 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:00.493383  1722 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:00.502615  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 48026.2us
W20260709 13:03:00.504730  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.494105  6140 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:00.499020  6220 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:00.432870  6336 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.507514  6289 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:03:00.548534  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.529711  6140 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.502382  6172 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.536633  6220 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:00.557498  1720 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:00.549777  6336 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:00.576069  6332 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:00.576911  6227 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:00.577658  6227 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:00.548971  6008 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:00.586006  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.561576  6312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.015s	sys 0.022s
I20260709 13:03:00.579694  6227 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:00.597610  6351 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 13:03:00.587592  6289 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:00.594342  6312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.016s	sys 0.023s
I20260709 13:03:00.608209  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.633678  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:00.635015  1646 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:00.640972  1646 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:00.642233  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.643959  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:00.649271  1644 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:00.654765  6289 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.657089  1643 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:00.660567  6355 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.661629  6355 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:00.662338  6172 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:00.662850  6355 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.664261  6358 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:00.664896  6358 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:00.665324  6358 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.636368  6357 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.667706  6357 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:03:00.669422  6346 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:00.672281  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:00.659610  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2217) took 1327 ms (client timeout 1509 ms). Trace:
I20260709 13:03:00.678159  1567 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:00.633448  6227 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:00.676321  1645 rpcz_store.cc:269] 0709 13:02:59.332654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.333009 (+   355us) service_pool.cc:224] Handling call
0709 13:03:00.659315 (+1326306us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400307854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:00.659487 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2620231936}
W20260709 13:03:00.659818  6308 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.007s
I20260709 13:03:00.626026  5698 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:00.690954  1718 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:03:00.694603  6348 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.695434  6348 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.660631  6294 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.701581  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 220728us
I20260709 13:03:00.703492  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:00.703593  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 259615us
I20260709 13:03:00.704515  1647 raft_consensus.cc:2376] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:03:00.676620  6235 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:00.669346  6357 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:00.712800  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 209116us
I20260709 13:03:00.682461  6065 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:00.725922  1650 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:00.735328  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.756582  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.691995  6308 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:00.766760  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.741400  6051 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:00.773828  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 100957us
I20260709 13:03:00.755720  6084 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:00.780666  6364 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.780916  6084 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.781486  6364 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.788565  1647 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:00.791589  6210 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.792485  6350 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.226s	user 0.000s	sys 0.002s
I20260709 13:03:00.796439  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.783903  6265 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:00.797930  1650 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:00.803189  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.797784  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:00.817811  1647 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
W20260709 13:03:00.796681  6183 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.889s	user 0.008s	sys 0.009s
W20260709 13:03:00.824834  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 45881.9us
W20260709 13:03:00.818638  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 112208us
I20260709 13:03:00.829563  1467 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.784112  6364 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:00.833732  6359 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:03:00.843835  6084 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:00.845886  6362 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:00.846503  6348 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:00.848896  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.852463  1651 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:00.852795  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:00.853832  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:00.854333  1646 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:03:00.855300  1644 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:03:00.855614  6363 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:00.856685  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.877848  1724 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:00.857434  1643 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 12. Ops: []
I20260709 13:03:00.860622  6353 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:00.862309  6352 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:03:00.860594  1467 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 10, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.912773  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2312) took 1179 ms. Trace:
I20260709 13:03:00.865990  6308 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:00.832099  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:00.800514  6278 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:00.929327  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.896446  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.901044  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:00.901813  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.902472  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:00.908434  6353 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:00.879298  6362 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:00.909948  6271 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:00.911007  1642 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:00.916965  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:00.913820  1718 rpcz_store.cc:276] 0709 13:02:59.732733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.732962 (+   229us) service_pool.cc:224] Handling call
0709 13:02:59.853217 (+120255us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:00.912199 (+1058982us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b5400372154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:00.912405 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353975936}
I20260709 13:03:00.932065  1644 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:03:00.938936  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 113225us
I20260709 13:03:00.939841  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:00.938563  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:00.945585  6364 raft_consensus.cc:2764] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:00.946636  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:00.946939  6353 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:00.947579  6362 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.933475  6278 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:00.948390  1647 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:00.948546  6278 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.946746  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2314) took 1126 ms. Trace:
I20260709 13:03:00.950650  1716 rpcz_store.cc:276] 0709 13:02:59.786742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.844602 (+ 57860us) service_pool.cc:224] Handling call
0709 13:02:59.845344 (+   742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.846008 (+   664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.846046 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:59.846862 (+   816us) log.cc:844] Serialized 26 byte log entry
0709 13:03:00.217654 (+370792us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:03:00.217720 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.218189 (+   469us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:00.913268 (+695079us) raft_consensus.cc:1685] finished
0709 13:03:00.913294 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.913464 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":188,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":370027,"threads_started":1,"wal-append.queue_time_us":526721}
I20260709 13:03:00.947912  1718 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:00.945742  2377 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:00.941092  2380 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:00.891861  6325 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:00.821105  6270 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:03:00.959615  6364 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:00.960096  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:00.962060  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.993548  1650 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:00.964610  6361 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.966233  1647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:01.004686  1647 raft_consensus.cc:606] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:01.005115  1647 raft_consensus.cc:614] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:01.006130  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44174.7us
I20260709 13:03:00.880653  6363 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:00.868498  6374 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.229s
I20260709 13:03:00.974907  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:00.986769  1645 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:00.992663  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2505) took 1092 ms. Trace:
I20260709 13:03:00.976503  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:00.994076  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:00.994592  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2309) took 1298 ms. Trace:
I20260709 13:03:01.016309  2377 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.996711  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:00.908044  6220 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:01.026062  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.725s (SENT)
W20260709 13:03:01.015069  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.014672  2375 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:00.978014  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 145990us
I20260709 13:03:01.018886  1649 raft_consensus.cc:2410] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:00.996959  6353 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.026791  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:01.011165  6308 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.036381  6308 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.016626  1725 rpcz_store.cc:276] 0709 13:02:59.696571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.701390 (+  4819us) service_pool.cc:224] Handling call
0709 13:03:00.993963 (+1292573us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400233054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:00.994483 (+   520us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413802368}
I20260709 13:03:01.020239  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.017484  1721 rpcz_store.cc:276] 0709 13:02:59.902436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.312496 (+410060us) service_pool.cc:224] Handling call
0709 13:03:00.346491 (+ 33995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.396646 (+ 50155us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:00.758929 (+362283us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.987434 (+228505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.987576 (+   142us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:00.988479 (+   903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.988707 (+   228us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.992299 (+  3592us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41107,"prepare.queue_time_us":130697,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":2713,"raft.queue_time_us":59432,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":425203,"threads_started":2}
W20260709 13:03:01.029277  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.041466  1716 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:00.982781  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.042196  1723 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:01.050225  1645 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:01.050930  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:03:01.025641  6300 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.601s	user 0.012s	sys 0.014s
I20260709 13:03:01.020681  6265 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:01.013353  6364 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:01.093875  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.011425  6363 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:01.068620  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 60325.4us
I20260709 13:03:01.048764  6348 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.020531  6361 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.067595  6286 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.137486  6140 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.549s	user 0.015s	sys 0.000s
W20260709 13:03:01.140424  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.141021  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.164877  6140 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.166469  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:01.167353  1643 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:01.167521  6140 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:01.131769  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.168829  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.142388  1647 pending_rounds.cc:85] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 4
I20260709 13:03:01.107797  6265 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.169610  1647 pending_rounds.cc:107] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:03:01.170153  6265 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:01.170920  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:01.099843  6368 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.157661  1467 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 9, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:01.168268  6286 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.109189  6308 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:01.169119  6051 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:01.170471  1647 raft_consensus.cc:2889] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.068795  6300 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.172576  6210 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:01.177366  6210 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:01.159571  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 108882us
I20260709 13:03:01.178305  2374 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:01.168776  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 162395us
I20260709 13:03:01.172307  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:01.181581  2377 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:01.185060  2375 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:01.195442  6357 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:01.184835  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:01.198706  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 258890us
I20260709 13:03:01.177834  6210 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:01.203919  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.830s (SENT)
I20260709 13:03:01.175808  6051 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.223875  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:01.209465  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.222203  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:01.173815  6220 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:01.231640  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.205799  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:01.226744  1572 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:01.239975  1643 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:01.197378  6312 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:01.245733  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.243080  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:01.247203  1576 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:03:01.203608  6308 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:01.250427  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.250082  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.181773  6348 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.234800  6051 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:01.262956  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:01.293402  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.299744  1575 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:01.258919  1571 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.231611  6356 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:01.238693  6065 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:01.244671  6270 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.246404  6312 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.309911  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.319453  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:01.320108  1643 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:01.310388  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 111402us
I20260709 13:03:01.306245  6364 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.250491  6286 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.175082  1467 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 10, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.251052  6362 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.257931  6386 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:01.354527  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.266568  6235 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.321946  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:01.306838  6379 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:03:01.301673  6353 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.265128  6308 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.354866  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.322075  6312 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:01.301357  6378 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.250022  6183 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:03:01.361388  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 122130us
I20260709 13:03:01.364929  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.382193  1575 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:01.363047  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.910s (SENT)
I20260709 13:03:01.383460  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.323112  6172 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:01.355859  1567 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:01.361133  6140 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:01.347360  6356 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.376199  6348 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:01.373097  6388 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:01.381318  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:01.371207  6353 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.354971  6065 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:01.354948  6386 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:01.407111  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 99640.1us
I20260709 13:03:01.376452  6362 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:01.385635  1571 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:01.392532  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.378149  6308 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.384276  6286 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:01.418905  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.407418  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2558) took 1065 ms. Trace:
I20260709 13:03:01.423962  1645 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:01.424203  1567 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.393810  6270 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.391002  6140 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:01.435052  6286 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.435902  6172 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:01.437862  1574 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:03:01.438050  6383 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:03:01.438853  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.481523  1572 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:01.481856  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.424649  1570 rpcz_store.cc:276] 0709 13:03:00.342028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.342209 (+   181us) service_pool.cc:224] Handling call
0709 13:03:00.342574 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.612606 (+270032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.612665 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.637279 (+ 24614us) log.cc:844] Serialized 26 byte log entry
0709 13:03:00.779626 (+142347us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:00.819141 (+ 39515us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.824988 (+  5847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.825085 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.406904 (+581819us) raft_consensus.cc:1685] finished
0709 13:03:01.406926 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.407336 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28610,"prepare.queue_time_us":45413,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":133282,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":5407,"spinlock_wait_cycles":11520,"thread_start_us":319374,"threads_started":2,"wal-append.queue_time_us":484405}
W20260709 13:03:01.441874  6380 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:03:01.442504  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:01.451992  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 43869.5us
I20260709 13:03:01.483805  1576 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:01.379038  6235 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.461628  1567 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:03:01.462081  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99097.8us
I20260709 13:03:01.404786  6388 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.469826  1575 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:03:01.476563  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.419839  5698 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:01.499857  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.472064  6140 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.506989  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:01.508054  2380 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:01.386293  6378 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.519963  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.509606  2371 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:01.521605  1575 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 7.
I20260709 13:03:01.522482  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:01.527284  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.540395  1570 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:01.503026  5698 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:01.498271  6388 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.548882  2376 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.491935  6348 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:03:01.555253  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.126s (SENT)
I20260709 13:03:01.572695  6210 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.577219  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:01.579458  6393 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:01.580523  6393 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.583552  6393 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.584846  6396 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.585477  6396 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.588470  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.539302  6386 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.547315  6183 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.606144  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.607232  1570 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:03:01.584174  6387 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:01.620524  6396 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:01.622262  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96388.1us
W20260709 13:03:01.682706  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 165347us
I20260709 13:03:01.625298  6408 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:01.626379  6356 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.636870  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.638429  6373 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:01.604039  6294 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.653681  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2553) took 1317 ms. Trace:
I20260709 13:03:01.654168  6286 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:01.655692  6391 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:03:01.656982  6394 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:03:01.747236  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2561) took 1320 ms. Trace:
I20260709 13:03:01.742415  1573 rpcz_store.cc:276] 0709 13:03:00.336158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.336321 (+   163us) service_pool.cc:224] Handling call
0709 13:03:00.336886 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.614125 (+277239us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:00.758929 (+144804us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.005274 (+246345us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.317197 (+311923us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.369190 (+ 51993us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.369672 (+   482us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:01.378190 (+  8518us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.379351 (+  1161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.379499 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.653337 (+273838us) raft_consensus.cc:1685] finished
0709 13:03:01.653374 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.653584 (+   210us) inbound_call.cc:177] Queueing success response
0709 13:03:01.673765 (+ 20181us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b1800316000. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
Metrics: {"mutex_wait_us":64518,"prepare.queue_time_us":167629,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":154,"raft.queue_time_us":445417,"raft.run_cpu_time_us":700,"raft.run_wall_time_us":62713,"spinlock_wait_cycles":13250688,"thread_start_us":935114,"threads_started":4,"wal-append.queue_time_us":361525}
I20260709 13:03:01.747737  1568 rpcz_store.cc:276] 0709 13:03:00.426845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.435230 (+  8385us) service_pool.cc:224] Handling call
0709 13:03:00.447707 (+ 12477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.495091 (+ 47384us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:00.975405 (+480314us) spinlock_profiling.cc:243] Waited 989 us on lock 0x7b180041f200. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf
0709 13:03:00.975566 (+   161us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.382142 (+406576us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.408859 (+ 26717us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.409014 (+   155us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:01.409904 (+   890us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.450523 (+ 40619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.450654 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.719158 (+268504us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b180041f200. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc837c4258f 00007fc830c81571 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd
0709 13:03:01.746921 (+ 27763us) raft_consensus.cc:1685] finished
0709 13:03:01.746949 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.747140 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91308,"prepare.queue_time_us":295982,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":313770,"raft.run_cpu_time_us":559,"raft.run_wall_time_us":30418,"spinlock_wait_cycles":23649536,"thread_start_us":561940,"threads_started":3,"wal-append.queue_time_us":534234}
W20260709 13:03:01.758045  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2317) took 1855 ms (client timeout 2166 ms). Trace:
I20260709 13:03:01.758893  6373 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.762293  6300 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:01.661082  6227 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.973s	user 0.016s	sys 0.003s
I20260709 13:03:01.559352  6065 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.650421  5698 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.602377  6210 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:01.768898  6294 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:01.639483  6385 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:03:01.774688  6378 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:01.608659  6271 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:01.666289  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57169.5us
W20260709 13:03:01.671128  6278 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.722s	user 0.016s	sys 0.000s
I20260709 13:03:01.777370  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.777318  6278 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.672619  6308 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:01.780421  6210 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:01.688118  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.783916  6278 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:01.680824  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:01.693011  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:01.492780  1467 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 9, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:01.640573  6172 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.696230  6235 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:01.639549  6183 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:03:01.701575  6332 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.009s
W20260709 13:03:01.682732  6358 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.010s	user 0.011s	sys 0.005s
I20260709 13:03:01.791777  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 657) took 1908 ms. Trace:
I20260709 13:03:01.792071  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:01.814957  1644 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:01.793957  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.801460  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.704798  6051 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.705945  6408 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:01.677551  6386 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.837704  6408 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.841188  1467 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:01.842665  1644 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:01.723206  6400 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.706836  6361 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.726284  6353 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:01.683532  1614 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:01.782310  6308 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:01.678392  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 123170us
W20260709 13:03:01.784534  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.849776  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.685361  2376 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:01.789029  6210 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:01.823410  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 142711us
I20260709 13:03:01.793473  1643 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:01.860757  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:01.819103  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 135731us
I20260709 13:03:01.862211  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:01.862679  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:01.863063  1644 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:01.830878  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.817039  1651 raft_consensus.cc:2376] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14. Re-sending same reply.
I20260709 13:03:01.834076  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.865465  1645 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:03:01.815680  6368 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.004s
I20260709 13:03:01.834187  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.867906  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:01.818974  6363 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.688s	user 0.014s	sys 0.000s
I20260709 13:03:01.842125  6312 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:01.725418  6372 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.780s	user 0.005s	sys 0.000s
W20260709 13:03:01.758580  1724 rpcz_store.cc:269] 0709 13:02:59.902890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.986170 (+ 83280us) service_pool.cc:224] Handling call
0709 13:03:01.757801 (+1771631us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54003ab954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:01.757964 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985051904}
I20260709 13:03:01.814520  2374 rpcz_store.cc:276] 0709 13:02:59.883593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.883962 (+   369us) service_pool.cc:224] Handling call
0709 13:02:59.906670 (+ 22708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.178822 (+1272152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.791221 (+612399us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54004ae054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:01.791506 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2923886464}
I20260709 13:03:01.867324  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:01.834052  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.868986  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.870594  1725 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:01.880374  1647 raft_consensus.cc:2368] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:01.871289  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.881537  1723 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:01.762912  6373 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:01.847672  6265 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.677s	user 0.016s	sys 0.000s
I20260709 13:03:01.820096  6235 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:01.889736  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:01.893689  1720 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:01.886787  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.894955  1568 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:01.890216  1716 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:01.848677  6051 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:01.878221  6363 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.897733  6388 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.867115  1467 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 9, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:01.860227  6332 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:01.899365  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2502) took 2000 ms. Trace:
I20260709 13:03:01.877682  6278 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.843518  6400 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.902113  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:01.904102  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 220668us
I20260709 13:03:01.904256  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.944770  1574 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:01.822589  6358 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.914712  6235 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:01.909986  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1804) took 1058 ms. Trace:
I20260709 13:03:01.819446  6294 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.893677  1575 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:01.922132  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.924156  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.955299  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.869644  6410 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:01.926837  1720 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:01.924059  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:01.953663  1646 rpcz_store.cc:276] 0709 13:03:00.851744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.875870 (+ 24126us) service_pool.cc:224] Handling call
0709 13:03:00.903255 (+ 27385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.927598 (+ 24343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.927649 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.938817 (+ 11168us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.210918 (+272101us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:01.490201 (+279283us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.847695 (+357494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.847795 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.872515 (+ 24720us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b1800380860. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:01.909557 (+ 37042us) raft_consensus.cc:1685] finished
0709 13:03:01.909719 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.909905 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65258,"prepare.queue_time_us":53653,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":67,"raft.queue_time_us":167232,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":21934,"spinlock_wait_cycles":4791680,"thread_start_us":791997,"threads_started":3,"wal-append.queue_time_us":317980}
I20260709 13:03:01.876818  6368 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:01.931653  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.937063  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:01.940613  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.916982  6210 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:01.945724  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.907140  6312 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:01.947361  1725 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:01.928036  6355 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.019s	sys 0.012s
I20260709 13:03:01.947686  1576 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:01.947841  1571 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:01.948988  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.976130  1724 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:01.934700  1719 rpcz_store.cc:276] 0709 13:02:59.898758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.898907 (+   149us) service_pool.cc:224] Handling call
0709 13:02:59.909345 (+ 10438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.440975 (+531630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.441015 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.482918 (+ 41903us) log.cc:844] Serialized 52 byte log entry
0709 13:03:00.754832 (+271914us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:00.754911 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.755002 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.724618 (+969616us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:01.898919 (+174301us) raft_consensus.cc:1685] finished
0709 13:03:01.898948 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.899280 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8545,"raft.queue_time_us":167456,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"spinlock_wait_cycles":19428096,"thread_start_us":648746,"threads_started":2,"wal-append.queue_time_us":316730}
I20260709 13:03:01.923261  6265 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:01.918759  6353 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:01.957706  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:01.957230  1575 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:01.952179  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.934639  6332 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:01.964862  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50657.5us
I20260709 13:03:01.965603  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.966675  1722 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:01.966810  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.968606  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:01.989331  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.989574  1718 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:01.990079  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:01.990821  1569 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:01.953630  6364 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.992861  2374 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:01.999290  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:01.974857  6355 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.019s	sys 0.012s
I20260709 13:03:02.003763  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:02.005461  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47860.1us
I20260709 13:03:02.007450  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:02.008023  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:02.008241  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.009025  1720 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:01.964771  6366 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:01.971853  6294 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:02.014606  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:01.948325  6278 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.011508  2375 raft_consensus.cc:2368] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:01.988284  6332 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.014237  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:02.025168  1569 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:02.027304  1717 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:02.034075  1567 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:02.034271  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.034814  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:02.030797  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.034967  1718 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:02.002271  6210 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:02.005077  6368 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:02.038514  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.054464  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.039700  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:02.017779  6362 raft_consensus.cc:2764] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:02.052783  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:02.039558  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.052171  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.051291  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.055363  1718 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:02.017773  6051 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:02.059653  1716 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:02.018880  6300 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:02.010012  6415 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:02.066262  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58371.7us
I20260709 13:03:02.014786  6361 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:02.072456  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.059769  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:02.034921  6355 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:02.004701  6364 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.004712  6353 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.041589  6408 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:03:02.085065  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66042.6us
I20260709 13:03:02.057554  6289 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.965075  6410 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.065884  6325 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.095136  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:02.136626  1724 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.072187  6294 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:02.033855  6378 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:03:02.079181  6084 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:02.075320  6183 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.033383  6366 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.134665  1572 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:02.137974  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 204757us
I20260709 13:03:02.091581  6396 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.099220  6308 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:02.063189  6140 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.067023  6373 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:03:02.144999  6363 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:02.136425  6325 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:02.080109  6413 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:02.148061  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59981.7us
I20260709 13:03:02.099562  6368 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.100582  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:02.188941  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.084518  6405 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:03:02.090236  6415 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.222126  6415 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.221414  6332 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.227104  1721 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:02.233592  6332 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:02.135772  6289 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:02.194286  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93844.6us
I20260709 13:03:02.238243  6364 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.238947  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:02.145665  6416 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:03:02.146139  6183 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.174335  6308 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.241367  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:02.197431  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.198194  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.256525  1650 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:02.198874  6405 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:02.185916  6413 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.080242  6235 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.172174  6388 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:02.200831  6278 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:02.202227  6417 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:03:02.202729  6265 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.202821  6140 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:02.203286  6358 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:02.203863  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.287628  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.204152  6408 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:02.260056  6405 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:02.267567  6051 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.197485  1648 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:02.208585  6400 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:02.210037  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.210259  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:02.210819  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.211534  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.213419  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:02.213778  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:02.296207  1719 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.296267  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:02.214406  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:02.298666  1571 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:02.298808  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:02.216019  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2514) took 1326 ms. Trace:
I20260709 13:03:02.216689  2376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 666) took 1281 ms. Trace:
I20260709 13:03:02.212092  1570 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:02.299929  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:02.218465  1646 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:02.219633  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:02.219972  6353 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:02.234958  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1816) took 1096 ms. Trace:
I20260709 13:03:02.299642  1722 rpcz_store.cc:276] 0709 13:03:00.889887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.890108 (+   221us) service_pool.cc:224] Handling call
0709 13:03:02.215835 (+1325727us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400232b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:02.215951 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2382001920}
I20260709 13:03:02.299782  2376 rpcz_store.cc:276] 0709 13:03:00.935440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.935639 (+   199us) service_pool.cc:224] Handling call
0709 13:03:01.262661 (+327022us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b58000d1798. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:02.216463 (+953802us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540011f8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:02.216608 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601670272}
I20260709 13:03:02.093140  6336 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:03:02.228222  1573 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:02.246050  1721 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:02.258812  1724 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:02.249681  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:02.262688  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.239238  6414 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.191363  6368 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.265354  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:02.239167  6289 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:02.309788  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.173961  6396 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.274901  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:02.323606  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48784.4us
I20260709 13:03:02.250840  6183 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.188838  6362 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
W20260709 13:03:02.285743  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89055.6us
I20260709 13:03:02.259696  6308 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.157172  6210 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.289394  6140 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.200330  6410 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:02.291334  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.204655  6227 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:02.345854  6227 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:02.346907  6227 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.200889  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.348786  6362 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:02.349350  6362 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:02.350291  6362 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.267400  6413 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.351792  6362 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.141045  1467 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:02.355448  6140 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.355914  1467 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.356616  6405 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.357690  6140 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:02.358615  1467 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:02.295441  1720 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.360105  6366 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:02.326206  6294 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:02.284598  6220 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:02.366621  6220 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:02.367619  6220 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.188298  6325 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.300786  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.301560  1568 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:02.372875  6424 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:03:02.308772  6363 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.302028  1648 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:02.243862  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:02.375028  1572 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:02.375097  6363 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.296494  6051 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.302321  1643 rpcz_store.cc:276] 0709 13:03:01.139018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.354755 (+215737us) service_pool.cc:224] Handling call
0709 13:03:01.793081 (+438326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.860725 (+ 67644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.234723 (+373998us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54002e2050. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:02.234877 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966342016}
I20260709 13:03:02.285121  6357 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:02.308836  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59226.6us
I20260709 13:03:02.327767  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:02.413403  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:02.333159  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 144377us
I20260709 13:03:02.294344  6084 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:03:02.350847  1647 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:02.322150  6312 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.321841  6368 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.343139  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:02.421680  1573 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.281132  6361 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:03:02.394327  1720 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:02.412817  6286 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.429147  6286 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.309088  6265 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:02.432018  6361 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:02.432736  6265 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.433308  6361 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:02.432041  6286 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.433775  6361 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:02.375960  5698 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.435252  6265 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.434810  6361 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.435580  1573 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:02.412413  6051 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.329507  6414 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.328963  6353 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:02.320221  6300 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.307071  6408 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:02.336084  6398 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:02.358501  6405 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.420174  6357 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:02.436945  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 249057us
I20260709 13:03:02.439467  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:02.415696  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:02.348281  6227 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.413292  6363 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:02.488102  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48705.8us
I20260709 13:03:02.421937  6312 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.490114  1575 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:02.436301  6361 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.436506  6294 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:02.491176  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:02.490043  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 159992us
I20260709 13:03:02.501719  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.475888  6398 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.507205  1539 leader_election.cc:400] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12.
I20260709 13:03:02.507642  1539 leader_election.cc:403] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:03:02.450145  6353 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:02.475347  6140 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.518508  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.490061  6410 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:02.522583  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:02.523185  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.513439  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:02.531380  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:02.532327  6300 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:02.534485  6300 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:02.456073  6400 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:03:02.494652  6312 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:02.542816  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64797.2us
W20260709 13:03:02.543401  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 120376us
I20260709 13:03:02.549019  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:02.511907  6084 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:02.492460  6427 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:03:02.510252  6398 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.557024  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:02.517189  6353 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:02.559506  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:02.567353  1575 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:02.524747  6358 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.566968  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:02.568667  1570 raft_consensus.cc:2410] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:02.559621  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:02.573939  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77242.2us
I20260709 13:03:02.546461  6400 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12.
I20260709 13:03:02.579787  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:02.577028  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.547159  6355 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:02.585896  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:02.547472  6396 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.590243  1576 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:02.566359  6294 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.597652  1720 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:02.539160  6140 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.546401  6354 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:03:02.601460  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:02.600589  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60319.1us
I20260709 13:03:02.602465  1725 raft_consensus.cc:2410] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:02.602900  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.606622  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:02.582751  6210 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.607492  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.583952  6084 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:02.590174  6373 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.589299  6355 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.566233  6271 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:03:02.599675  6388 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:02.600018  6439 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:02.582855  6358 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.611424  6396 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:02.607689  6438 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:02.496973  1467 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 9, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 9 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:02.621928  6355 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:02.645282  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:02.606030  6294 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.627086  6366 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:02.664814  1719 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:03:02.660404  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 111508us
I20260709 13:03:02.662143  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:02.639249  6388 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:02.681730  6433 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:02.686432  6357 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:02.687424  6431 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:03:02.661705  6414 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:02.615199  6210 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:02.628415  6308 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.705474  6325 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:02.708066  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.729182  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59786.1us
I20260709 13:03:02.651582  6438 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:02.645876  6396 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:03.191497  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:03.070331  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 455645us
W20260709 13:03:03.073110  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.071156  6220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.007s	sys 0.012s
W20260709 13:03:03.073616  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.005s	sys 0.016s
W20260709 13:03:03.205199  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.005s	sys 0.017s
I20260709 13:03:03.100831  6441 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.675490  6084 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.189714  6414 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:03.107595  6312 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.007s
I20260709 13:03:02.590546  6408 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:02.677039  6407 raft_consensus.cc:2764] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:03:03.112047  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 444406us
W20260709 13:03:03.138271  2344 connection.cc:256] RPC call timeout handler was delayed by -1.1305891080000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 13:03:03.127242  6353 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.539s	user 0.014s	sys 0.001s
I20260709 13:03:03.187471  1723 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:03.191128  1645 raft_consensus.cc:606] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:03.120523  6386 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:03.209086  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:03.123970  6348 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:03.219789  1569 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:03.230319  6363 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.006s
I20260709 13:03:03.125313  6366 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:03.265717  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:03.211344  6441 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:03.242506  2344 connection.cc:256] RPC call timeout handler was delayed by -1.1059056029999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
I20260709 13:03:03.241773  1717 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:03.145838  6354 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.211730  6220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.008s	sys 0.012s
I20260709 13:03:03.215065  6414 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.190088  6172 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.240833  6353 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:03.284283  1574 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.052s	sys 0.056s
W20260709 13:03:03.291816  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44433.2us
I20260709 13:03:03.251332  6348 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:03.302732  1574 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.052s	sys 0.056s
I20260709 13:03:03.265385  6364 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:03.265132  6386 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:03.250396  6312 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:03.327029  6437 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 13:03:03.327452  6183 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.988s	user 0.017s	sys 0.004s
I20260709 13:03:03.328501  6366 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:03.329568  6398 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.743s	user 0.014s	sys 0.003s
I20260709 13:03:03.329959  6398 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:03.335515  6368 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.886s	user 0.015s	sys 0.005s
I20260709 13:03:03.335985  6368 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.336697  6430 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.337433  6430 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.338179  6368 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:03.339501  6405 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.852s	user 0.008s	sys 0.010s
W20260709 13:03:03.299278  6227 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.808s	user 0.019s	sys 0.001s
I20260709 13:03:03.332033  6398 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.273085  6441 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.345703  1720 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:03.302681  6294 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:03.349718  6425 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.068s	user 0.013s	sys 0.003s
I20260709 13:03:03.311605  6358 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:03.387106  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:03.387876  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:03.327852  6183 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.271934  6363 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.323302  6357 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.340880  6430 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.290632  6354 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.440323  1649 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:03.314158  6172 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.189087  1467 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 11, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:03.453480  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2587) took 1126 ms. Trace:
I20260709 13:03:03.399528  6398 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:03.313253  6388 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:03.466322  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 284744us
I20260709 13:03:03.454139  1571 rpcz_store.cc:276] 0709 13:03:02.327303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.327546 (+   243us) service_pool.cc:224] Handling call
0709 13:03:02.339091 (+ 11545us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:02.419525 (+ 80434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.419566 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.563002 (+143436us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b180041f080. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a
0709 13:03:02.563925 (+   923us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b2c00000658. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:03:02.590745 (+ 26820us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.664026 (+ 73281us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:02.664474 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.451823 (+787349us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.452978 (+  1155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.453070 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.453110 (+    40us) raft_consensus.cc:1685] finished
0709 13:03:03.453138 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.453316 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516972,"prepare.queue_time_us":71101,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":5190,"raft.queue_time_us":517445,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":30180,"spinlock_wait_cycles":67381120,"thread_start_us":296487,"threads_started":2,"wal-append.queue_time_us":53337}
I20260709 13:03:03.470273  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:03.473893  2376 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:03.455821  5698 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.021s	user 0.012s	sys 0.007s
W20260709 13:03:03.460843  6362 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.108s	user 0.022s	sys 0.003s
W20260709 13:03:03.461071  6051 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.023s	user 0.011s	sys 0.018s
I20260709 13:03:03.435110  6353 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:03.475354  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:03.482854  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:03.483668  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:03.476225  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:03.484881  1569 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:03:03.465261  6355 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.798s	user 0.032s	sys 0.018s
I20260709 13:03:03.486305  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:03.477306  1572 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:03.487344  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:03.466926  6413 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.115s	user 0.010s	sys 0.020s
I20260709 13:03:03.488147  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:03.493474  1568 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.488518  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:03.487483  1572 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:03.359002  6396 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.421289  6357 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.513547  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:03.516517  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 210224us
I20260709 13:03:03.475266  6308 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:03.520112  2371 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:03.522132  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:03.522928  1725 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:03.524644  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:03.538192  2376 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:03.374421  6438 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.463447  6398 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.543613  2371 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:03.435920  6348 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.495388  6325 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:03.541180  1572 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.548602  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:03.521309  6227 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:03.550076  6210 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.550343  6455 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 13:03:03.551548  6446 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 13:03:03.552675  6448 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:03:03.488184  6412 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:03:03.557004  6227 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.562292  2371 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:03.491320  6051 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.521656  6364 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:03.564975  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:03.572734  2374 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:03.565392  1572 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:03.573402  1725 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:03.573094  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:03.496124  6355 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:03.577070  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 57825.1us
I20260709 13:03:03.580392  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:03.485428  6386 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.512665  6413 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.559684  6445 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:03.577557  6210 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:03.565562  6227 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.610971  6364 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.577131  6348 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.617887  2374 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:03.618088  1642 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:03:03.618494  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.620467  1721 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:03.588327  6308 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:03.630582  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:03.634483  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 113307us
W20260709 13:03:03.654080  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.641105  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:03.591840  6312 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.615999  6210 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.704751  6463 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.779791  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 223805us
W20260709 13:03:03.781229  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.069s (SENT)
W20260709 13:03:03.706804  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.591570  6065 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.125s
I20260709 13:03:03.789613  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:03.637547  6445 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:03.634806  6460 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:03.758450  6210 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.714875  6414 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:03.791132  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 210897us
W20260709 13:03:03.790570  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 299534us
I20260709 13:03:03.715685  6308 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:03.705940  6430 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:03.803097  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.807073  6460 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:03.810561  6460 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.811419  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:03.814670  6308 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.815606  6445 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.820715  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:03.824539  1688 leader_election.cc:400] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:03.826658  1688 leader_election.cc:403] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:03.830219  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 189230us
I20260709 13:03:03.795853  6358 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:03.833310  1572 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:03:03.801182  6325 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:03.804389  6363 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.842005  6378 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:03.842377  6473 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:03.842674  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:03.843695  6366 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:03.864497  6465 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
I20260709 13:03:03.867527  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:03.849031  6444 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:03:03.828156  6430 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:03.843555  6413 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:03.808722  6354 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:03.849767  6469 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:03:03.824971  6456 raft_consensus.cc:2764] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:03.859122  6289 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:03.836467  1572 pending_rounds.cc:85] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
W20260709 13:03:03.871767  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90588.9us
I20260709 13:03:03.890847  1572 pending_rounds.cc:107] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:03:03.891496  1572 raft_consensus.cc:2889] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:03.790400  6450 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.370s
I20260709 13:03:03.837175  6172 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:03.840773  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:03.838872  6467 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.637002  1467 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:03.829097  6278 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:03:03.785197  6463 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.864620  6366 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.864732  6452 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:03.856647  6325 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.927110  6467 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:03.951406  6467 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.926714  6363 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:03.952683  6363 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:03.953222  6363 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.941256  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42234 (request call id 25) took 1717 ms. Trace:
I20260709 13:03:03.956593  1645 rpcz_store.cc:276] 0709 13:03:02.223398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.304360 (+ 80962us) service_pool.cc:224] Handling call
0709 13:03:03.940973 (+1636613us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54001ba8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d68853 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:03.941118 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787371392}
I20260709 13:03:03.868844  6378 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:03.944540  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2320) took 2062 ms. Trace:
I20260709 13:03:03.961167  1723 rpcz_store.cc:276] 0709 13:03:01.882505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.882674 (+   169us) service_pool.cc:224] Handling call
0709 13:03:01.892851 (+ 10177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.242222 (+1349371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.944102 (+701880us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400088754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:03.944429 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2948123008}
I20260709 13:03:03.960057  6378 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.950412  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:03.937777  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97065.8us
I20260709 13:03:03.945676  6479 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:03.950366  1467 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:03.944918  6476 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:03:03.887168  6398 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.956174  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:03.905886  6051 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:03.926055  6412 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:03.988737  1716 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.898797  6357 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:03.994912  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:03.966500  6366 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:03.899407  6289 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:03.995975  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2590) took 1437 ms. Trace:
I20260709 13:03:04.000133  2371 raft_consensus.cc:2368] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
W20260709 13:03:04.000438  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.788s (SENT)
I20260709 13:03:03.966375  6355 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:04.001355  2374 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:04.002635  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:04.025031  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.027794  2376 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:04.022711  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:04.009673  1574 rpcz_store.cc:276] 0709 13:03:02.558256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.558417 (+   161us) service_pool.cc:224] Handling call
0709 13:03:02.583276 (+ 24859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.601352 (+ 18076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.601386 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.609099 (+  7713us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.308686 (+699587us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:03.308735 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.308813 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.995688 (+686875us) raft_consensus.cc:1685] finished
0709 13:03:03.995714 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.995894 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6981,"raft.queue_time_us":14701,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":7423,"thread_start_us":664343,"threads_started":1,"wal-append.queue_time_us":814805}
W20260709 13:03:04.029844  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 162412us
W20260709 13:03:04.039129  1688 connection.cc:256] RPC call timeout handler was delayed by -1.2961812330000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:03:04.042388  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.045931  1688 leader_election.cc:400] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:04.043136  6355 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.050879  1688 leader_election.cc:400] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:04.051285  1688 leader_election.cc:403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:04.051012  1616 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:03.992252  6413 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:03:04.053467  1568 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 13:03:04.054383  1568 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:04.035120  6477 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.995937  6479 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.064687  6477 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.067500  6477 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.069460  6479 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.052915  6355 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.058826  6450 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:04.054749  6084 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:04.070452  6471 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:03:04.033479  6440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.012s	sys 0.016s
W20260709 13:03:04.074430  6440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.013s	sys 0.016s
I20260709 13:03:04.069214  6210 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.080492  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:04.041266  6353 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.081875  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:04.086899  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:04.087793  1719 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:04.089602  6325 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:04.090266  6325 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.090955  6325 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.091405  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.092522  6363 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.083855  1716 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:04.094233  6325 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:04.090245  6210 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.096134  1574 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:04.096710  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:04.100873  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.109884  1574 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:04.077046  6051 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.101866  6325 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.104501  1576 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:04.000175  1467 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 9, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.116919  1467 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 9, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:04.120570  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.121088  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.121881  1572 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:04.118610  1467 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:04.125264  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1824) took 2296 ms (client timeout 1661 ms). Trace:
I20260709 13:03:04.126362  6364 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:04.090376  6353 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:04.127617  6449 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:03:04.127741  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2536) took 1882 ms. Trace:
I20260709 13:03:04.136883  1569 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:04.136897  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.132175  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:04.135293  1717 rpcz_store.cc:276] 0709 13:03:02.245103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.245676 (+   573us) service_pool.cc:224] Handling call
0709 13:03:02.245916 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.277905 (+1031989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.127410 (+849505us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b5400087354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:04.127671 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289418624}
I20260709 13:03:04.138152  1570 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:04.105759  6289 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.142266  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.143379  1570 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:04.144161  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:04.127566  1642 rpcz_store.cc:269] 0709 13:03:01.829036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.832975 (+  3939us) service_pool.cc:224] Handling call
0709 13:03:04.125043 (+2292068us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400389fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:04.125152 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3705547136}
I20260709 13:03:04.148635  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:04.145407  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51417.9us
I20260709 13:03:04.151403  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:04.126313  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44522.2us
W20260709 13:03:04.189469  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.201534  1569 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:04.150066  1570 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:04.202440  6332 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:04.202934  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:04.054726  6473 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.122s
W20260709 13:03:04.205564  6441 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.822s	user 0.011s	sys 0.002s
I20260709 13:03:04.154867  1574 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:03:04.227833  1571 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:04.218864  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:04.147043  6235 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:04.146824  6357 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.203006  6286 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:04.215688  6332 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.210412  6051 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:04.222412  6464 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:04.229709  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.220150  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:04.236750  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.140223  6427 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:03:04.247335  1574 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:04.148761  6483 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:04.308274  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.177s (SENT)
I20260709 13:03:04.308995  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:04.235394  6227 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.001s
W20260709 13:03:04.333099  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.346618  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95555.8us
I20260709 13:03:04.347680  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:04.348913  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.349730  1574 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:04.147034  6325 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.244701  6286 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.299548  6483 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.443800  6483 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.233124  6441 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.451007  6441 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:04.359754  6366 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.299502  6398 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.245247  6332 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:04.360675  6481 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:03:04.361059  6353 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:04.362155  6411 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.363458  6396 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:04.364046  6308 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:04.364681  6210 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.366245  6065 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.368746  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.369760  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.372567  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.549407  1648 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:04.422286  1570 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:04.438169  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.441957  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.442776  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.451534  6286 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:04.483529  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 135955us
W20260709 13:03:04.482844  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 331202us
I20260709 13:03:04.442875  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.128295  1467 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 9, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:04.351205  6463 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.553458  1574 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:04.580188  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.564857  1646 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:04.582219  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.582917  1646 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:04.533951  6449 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:03:04.451965  6441 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:04.576285  1575 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:04.534241  6278 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.508812  6363 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.579411  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.575131  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.320178  6051 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:04.595402  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 287188us
W20260709 13:03:04.611323  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.774s (SENT) [suppressed 12 similar messages]
I20260709 13:03:04.536267  6398 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.619060  6366 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.621971  6172 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:04.622462  6172 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:04.626675  6051 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:04.627854  6140 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:04.629082  6490 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:03:04.623454  6172 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.579350  6460 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:04.630570  6172 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.576941  6308 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.543242  6477 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:04.637104  6308 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.594868  6441 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.637941  6441 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.246141  6357 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.389376  6227 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.672410  6450 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:04.673326  6450 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.604434  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:04.605895  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.677227  1642 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:04.607707  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.608489  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.643152  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.644071  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.650694  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.658095  6414 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:04.659847  6332 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.662328  6286 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.664112  6051 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:04.679942  1647 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:04.551786  6210 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.548298  6396 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:04.700831  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96518.3us
W20260709 13:03:04.701160  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.704140  1646 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:04.705330  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.705417  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.707391  1650 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:04.543036  6411 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.565528  6065 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.711726  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.712131  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.712483  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.717283  1642 raft_consensus.cc:2376] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12. Re-sending same reply.
W20260709 13:03:04.706697  6084 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.000s
I20260709 13:03:04.693840  6453 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:04.689563  6414 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:04.725884  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.711714  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:04.702489  6449 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:04.726532  1644 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:04.720736  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:04.668211  6227 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:04.721652  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 501567us
I20260709 13:03:04.694758  6438 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:04.723115  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.781616  1647 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.708680  6286 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.712648  6271 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.706053  6363 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.736002  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.708704  6460 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:04.785929  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.727260  6084 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:04.805167  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.808935  1646 raft_consensus.cc:2410] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:04.809237  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2592) took 2243 ms. Trace:
W20260709 13:03:04.809885  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.983s (SENT)
I20260709 13:03:04.818488  1644 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:04.819587  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.031s (SENT)
W20260709 13:03:04.788744  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59872.2us
I20260709 13:03:04.726701  6235 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:04.708909  6051 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.839956  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:04.840687  1642 raft_consensus.cc:606] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:04.841114  1642 raft_consensus.cc:614] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:04.818295  1573 rpcz_store.cc:276] 0709 13:03:02.559096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.559237 (+   141us) service_pool.cc:224] Handling call
0709 13:03:02.574189 (+ 14952us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:02.597525 (+ 23336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.597562 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.606651 (+  9089us) log.cc:844] Serialized 52 byte log entry
0709 13:03:03.112957 (+506306us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:03.145496 (+ 32539us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.192663 (+ 47167us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.519432 (+326769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.519539 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.815534 (+295995us) spinlock_profiling.cc:243] Waited 7.98 ms on lock 0x7b1800316300. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1
0709 13:03:03.816432 (+   898us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b1800316300. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:04.802434 (+986002us) raft_consensus.cc:1685] finished
0709 13:03:04.802468 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.802736 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34077,"prepare.queue_time_us":230281,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":96,"raft.queue_time_us":249963,"raft.run_cpu_time_us":1021,"raft.run_wall_time_us":52881,"spinlock_wait_cycles":28049920,"thread_start_us":782212,"threads_started":2,"wal-append.queue_time_us":785078}
I20260709 13:03:04.828552  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:04.741158  6450 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:04.840576  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 176965us
I20260709 13:03:04.845640  1644 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:03:04.849562  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:04.811591  6271 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.850517  1646 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:04.859298  1646 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:03:04.850847  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.862493  1573 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:04.863374  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.866719  1647 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:04.829476  6457 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.726495  6414 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.878983  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:04.882162  1573 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
W20260709 13:03:04.893322  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 217157us
I20260709 13:03:04.809026  6438 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.848620  6500 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:04.735110  6453 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.930083  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:04.940224  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45029.8us
I20260709 13:03:04.823658  6460 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:04.939085  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:04.945770  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96328.8us
I20260709 13:03:04.926218  6414 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.953109  6412 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.953992  6412 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:04.953212  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.959247  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:04.962006  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2541) took 1718 ms (client timeout 1751 ms). Trace:
W20260709 13:03:04.962617  1720 rpcz_store.cc:269] 0709 13:03:03.243884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.244061 (+   177us) service_pool.cc:224] Handling call
0709 13:03:04.961726 (+1717665us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b5400214154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:04.961918 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182159104}
I20260709 13:03:04.963470  6412 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:04.967499  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:04.971974  6332 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:04.973351  6210 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.977022  6271 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.978070  6183 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.978978  6286 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:04.982486  6398 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:04.983227  1721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2539) took 2397 ms (client timeout 1742 ms). Trace:
W20260709 13:03:04.983667  1721 rpcz_store.cc:269] 0709 13:03:02.585431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.585665 (+   234us) service_pool.cc:224] Handling call
0709 13:03:04.982956 (+2397291us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400233554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:04.983110 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2377283968}
I20260709 13:03:04.984292  6051 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.876471  6450 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:03:04.894203  6363 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:04.889081  6325 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:04.987653  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46010.9us
W20260709 13:03:05.001969  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.014948  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68257.2us
I20260709 13:03:04.995786  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:04.951987  6308 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:03:05.020006  6366 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:05.046948  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51347.8us
I20260709 13:03:05.047428  2372 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:05.047260  6366 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.047959  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.049217  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:05.049912  6366 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.046139  6286 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.051189  6286 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.025014  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:05.053359  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:05.028759  1650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e69630519d3c48eca350b04183abca32"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:05.053838  6286 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:04.990803  6467 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:05.054309  1650 raft_consensus.cc:606] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:05.054752  1650 raft_consensus.cc:614] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:05.054577  6467 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.014163  6332 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:05.056198  6332 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.057785  6467 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.058830  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:05.059594  6332 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:05.018149  6444 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:03:05.017562  6479 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:05.061162  6479 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.031541  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:04.884668  6457 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.032572  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.033030  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.071874  1720 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:05.034035  1722 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:05.034658  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:05.077090  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.035298  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.083984  1725 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.035936  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.038149  6438 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:05.038357  2375 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:05.032667  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.040395  2374 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:05.045408  6453 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.046046  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:04.816730  1467 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:05.059785  2379 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:04.989454  6235 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.029488  6411 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:05.063963  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:05.027966  6348 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.393s	user 0.010s	sys 0.003s
I20260709 13:03:05.064575  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.085151  1723 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:05.086851  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71019.2us
I20260709 13:03:05.091158  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.091539  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2542) took 1846 ms. Trace:
I20260709 13:03:05.107131  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.091411  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.107903  1720 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:05.091320  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.099536  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.100495  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.103027  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.107669  1724 rpcz_store.cc:276] 0709 13:03:03.245073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.245224 (+   151us) service_pool.cc:224] Handling call
0709 13:03:03.245605 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.291310 (+ 45705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.291356 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.306313 (+ 14957us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.506904 (+1200591us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b1800348460. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:04.924839 (+417935us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:04.924950 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.925130 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.065171 (+140041us) raft_consensus.cc:1685] finished
0709 13:03:05.065205 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.091470 (+ 26265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45266,"raft.queue_time_us":56774,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":12890,"spinlock_wait_cycles":2945920,"thread_start_us":388451,"threads_started":1,"wal-append.queue_time_us":1742426}
I20260709 13:03:05.110332  1717 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:03:05.111469  1717 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:05.060273  6332 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:05.112834  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60487.2us
I20260709 13:03:05.114660  1724 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:03:05.118978  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55108.8us
I20260709 13:03:05.108889  1722 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:03:05.136730  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.086257  6463 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:05.070686  6357 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.087836  6498 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.108546  6363 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:05.162459  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55460us
I20260709 13:03:05.169893  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.090854  6286 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:05.170738  6286 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:05.171276  6286 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.160625  6373 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.175433  6373 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:05.178225  1651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 696) took 2970 ms (client timeout 1774 ms). Trace:
I20260709 13:03:05.139499  6513 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.178902  1651 rpcz_store.cc:269] 0709 13:03:02.207822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.276644 (+ 68822us) service_pool.cc:224] Handling call
0709 13:03:05.177862 (+2901218us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540005c8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:05.178096 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3952096128}
I20260709 13:03:05.121733  6065 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.129200  6467 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.180308  6513 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:04.949157  6500 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.091048  6051 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:03:05.191087  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.195652  1725 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:05.192860  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.196321  1567 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:05.197636  1571 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:05.197911  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.161831  6463 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:05.138939  6509 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:05.122648  6332 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.201810  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2111) took 1071 ms. Trace:
I20260709 13:03:05.167900  6511 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:05.178925  6493 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:03:05.204689  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.207549  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.208308  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.214005  1573 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:05.214401  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:05.211016  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.205222  6463 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:05.208998  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.215900  1567 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:05.211051  1572 rpcz_store.cc:276] 0709 13:03:04.130502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.163967 (+1033465us) service_pool.cc:224] Handling call
0709 13:03:05.201157 (+ 37190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.201451 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.201476 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.201518 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:05.201551 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.201640 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.201749 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.220033  1569 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:05.220613  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:05.217154  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.222957  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.207158  6479 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.205478  6400 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:05.259557  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.248893  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.265026  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.233430  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.262631  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2236) took 3057 ms. Trace:
I20260709 13:03:05.235981  1645 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:05.389676  1572 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
W20260709 13:03:05.391052  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 170506us
W20260709 13:03:05.391526  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67807.7us
I20260709 13:03:05.392189  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.395475  1568 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:05.396262  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2595) took 1306 ms. Trace:
I20260709 13:03:05.273505  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.271600  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2107) took 1151 ms. Trace:
I20260709 13:03:05.398802  1575 rpcz_store.cc:276] 0709 13:03:04.119959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.120133 (+   174us) service_pool.cc:224] Handling call
0709 13:03:05.271319 (+1151186us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400125cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:05.271478 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991913344}
I20260709 13:03:05.295298  1647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:03:05.268100  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.401125  1647 raft_consensus.cc:606] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:05.401544  1647 raft_consensus.cc:614] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:05.398514  1570 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:05.212638  6332 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.394572  1649 rpcz_store.cc:276] 0709 13:03:02.205337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.209749 (+  4412us) service_pool.cc:224] Handling call
0709 13:03:05.262383 (+3052634us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000ba254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:05.262553 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2631152512}
I20260709 13:03:05.402863  1568 rpcz_store.cc:276] 0709 13:03:04.089960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.126927 (+ 36967us) service_pool.cc:224] Handling call
0709 13:03:05.264813 (+1137886us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:05.396184 (+131371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92685696}
I20260709 13:03:05.266078  6469 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.294814  6373 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:05.224078  6511 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:05.271394  6483 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.827s	user 0.000s	sys 0.014s
I20260709 13:03:05.224385  6325 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:03:05.241549  6441 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.006s
I20260709 13:03:05.411092  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.228048  6460 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:05.397508  6514 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.150s	user 0.000s	sys 0.008s
I20260709 13:03:05.253482  6457 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.224221  6516 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:05.299563  6051 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.403883  6410 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:05.414466  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:05.388494  6477 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.282310  6396 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.230886  6447 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:05.405351  6473 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:03:05.230262  6500 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:05.289830  6172 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.431723  1569 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 7.
I20260709 13:03:05.436710  6442 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:05.436767   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:05.364236  6286 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:05.446384  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 223518us
I20260709 13:03:05.446602  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:05.432469  6518 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.152s	user 0.004s	sys 0.000s
I20260709 13:03:05.454295  1572 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:05.483707  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.451761  6235 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:05.414022  6363 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.432673  6325 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.444674  6051 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.432116  6483 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.497870  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:05.498514  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51969.1us
I20260709 13:03:05.499300  6235 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:05.492606   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:05.434077  6441 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.503912  1572 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.504691  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:05.501554  6172 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:05.509728  6286 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.509925  6355 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:05.496117  6517 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:03:05.514443  6463 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.517774  6463 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:05.495630  6519 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:03:05.521024  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.246958  1467 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 6, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 6 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:05.529906  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:05.537431  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.452014  6477 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.539179  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.505199  6513 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:05.506618  6410 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.560925  1569 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14.
I20260709 13:03:05.532361  6511 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.563066  6354 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.550981  6500 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:05.591737  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:05.456450  6447 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.531157  6468 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.582231  1572 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:03:05.532769  6467 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.531692  6355 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.587162  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:05.594201  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.595713  2371 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.591109  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.597174  1647 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:05.591017  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.551807  6332 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.595759  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.601264  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.610224  1720 raft_consensus.cc:2368] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13.
I20260709 13:03:05.610558  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.611310  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.604863  6468 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.613431  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:05.614154  2376 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.614344  6468 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:05.594470  6500 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.602938  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:05.604457  1649 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:05.595041  6520 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:03:05.528478  6140 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:05.551118  6442 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.602849  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.616919  2378 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:05.617708  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.625399  1572 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:05.599419  6466 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:05.620112  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:05.620584  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.584733  6413 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:03:05.621829  2371 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:05.595511  6513 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:03:05.641793  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:05.642982  1648 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.644037  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:05.626832  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.635354  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.636229  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.645088  2379 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.645671  1572 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:05.636951  6472 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.808s	user 0.003s	sys 0.013s
I20260709 13:03:05.637046  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:05.637727  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:05.638188  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.648092  1647 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:05.649132  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.650266  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:05.637804  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.654474  1650 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:03:05.587335  6497 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:03:05.644045  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.657047  1723 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:05.629209  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.632920  6466 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:05.624701  6355 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:05.593524  6354 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.622249  6500 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.660625  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.661234  1651 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:05.625679  6051 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.601580  6235 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.658231  2376 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:05.717895  2380 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:05.698993  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2549) took 1750 ms (client timeout 1662 ms). Trace:
W20260709 13:03:05.718873  1716 rpcz_store.cc:269] 0709 13:03:03.948917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.950055 (+  1138us) service_pool.cc:224] Handling call
0709 13:03:05.698729 (+1748674us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b54003dd1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:05.698896 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48419712}
I20260709 13:03:05.697022  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.669435  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.670974  1717 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:05.725039  1648 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:05.724826  1716 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:05.719702  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:05.728492  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.621281  6410 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:05.750213  1720 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:05.750366  1539 connection.cc:256] RPC call timeout handler was delayed by -2.6121878189999848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
I20260709 13:03:05.706249  6441 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:05.731014  6388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.695s	user 0.022s	sys 0.058s
W20260709 13:03:05.754421  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 193483us
I20260709 13:03:05.760102  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:05.744607  1723 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:05.640012  6468 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:05.751892  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.763428  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:05.704509  6140 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:05.764394  2376 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:03:05.670918  6332 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:05.744840  6483 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:05.757161  6388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.732s	user 0.024s	sys 0.059s
I20260709 13:03:05.781159  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.775823  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:05.788189  1723 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:05.771799  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.778263  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.794968  6363 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:05.799046  6477 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.777143  6482 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.803591  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.756928  6400 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:03:05.780303  6410 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:05.818902  1723 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:05.821135  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.825740  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:05.823971  6412 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:05.828740  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.752988  6466 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.773629  6467 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:03:05.738094  6355 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.851271  6466 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.623301  1467 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 11, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.847525  1723 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:03:05.843463  2379 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:05.848027  1716 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:05.850153  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.773865  6468 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.857949  1467 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:05.858695  6140 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.860287  6468 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:05.861601  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1840) took 1484 ms (client timeout 1526 ms). Trace:
W20260709 13:03:05.862079  1645 rpcz_store.cc:269] 0709 13:03:04.377661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.958514 (+580853us) service_pool.cc:224] Handling call
0709 13:03:05.861442 (+902928us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54001badd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:05.861547 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634077568}
I20260709 13:03:05.862636  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.863318  1568 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:05.863776  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.865267  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.865800  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.866765  1569 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:05.866694  1721 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:05.867669  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.867987  1721 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:05.854346  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.869102  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.869830  1572 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:05.870499  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.871479  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:05.858383  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 788668us
I20260709 13:03:05.848472  6412 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:05.861511  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.873555  6354 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:05.873293  6412 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.864120  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.875298  1568 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:05.876132  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.876546  6325 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:05.877182  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.877738  1568 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.862123  6368 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.880874  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.882054  6289 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.885061  6051 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:05.886510  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:05.881829  6463 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:05.868207  1569 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:05.872942  6410 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.891271  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.891671  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.894862  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:05.898396  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:05.907392  1568 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:05.896834  6445 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:05.931155  6445 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:05.932679  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.932796  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:05.933223  1645 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:05.933604  1650 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.934600  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:05.913986  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.937165  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:05.937471  6355 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:05.910615  6400 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:03:05.938391  6400 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:05.938736  6400 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:05.939333  1573 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:05.915110  6513 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:05.915726  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.940514  6513 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.940918  6513 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:05.941437  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.941287  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.916069  6447 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:05.898366  6464 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:05.942157  1572 raft_consensus.cc:2376] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 9. Re-sending same reply.
I20260709 13:03:05.942507  6464 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.909628  6482 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:05.940929  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:05.944708  6482 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:05.945753  6482 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:05.928030  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.948944  1574 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:05.950546  1574 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 13:03:05.950872  1574 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:03:05.951413  1574 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 13:03:05.951712  1574 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:05.935194  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:05.952823  1576 raft_consensus.cc:2368] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13.
I20260709 13:03:05.954435  1576 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:05.956272  6286 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:05.956887  6286 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:05.958266  6413 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:05.958918  6413 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:05.959102  6286 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:05.859817  1467 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.935442  6051 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:05.910516  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:05.936437  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.962620  1648 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:05.962868  1574 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:03:05.964243  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:05.974684  1576 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:03:05.977090  1576 pending_rounds.cc:85] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:03:05.977442  1576 pending_rounds.cc:107] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:03:05.976747  1574 pending_rounds.cc:85] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:03:05.978012  1574 pending_rounds.cc:107] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 11.6
W20260709 13:03:05.979518  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 69095.2us
I20260709 13:03:05.991029  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.993533  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:05.995971  1720 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
W20260709 13:03:05.998492  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.978582  1574 raft_consensus.cc:2889] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.002731  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.004611  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.004945  1576 raft_consensus.cc:2889] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:05.981119  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:06.005721  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.006693  1648 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:06.011360  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.012074  1649 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:06.008505  1651 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:06.013204  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:06.013861  1642 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.014539  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.015151  1643 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:06.011190  6473 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.015192  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.011581  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:05.973776  1650 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:06.023710  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.023918  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.024711  1649 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:06.025127  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.025796  1648 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:06.027158  6457 raft_consensus.cc:2764] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:03:06.028005  6516 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:06.028460  6473 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.024834  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:06.028489  6516 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:06.030074  6516 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.031023  6473 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:06.030936  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:06.032141  1567 raft_consensus.cc:2410] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:03:06.032472  6477 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:06.034880  6500 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.020068  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.038811  1646 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:06.019287  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:06.040735  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.035006  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:06.042815  1567 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:06.043457  1574 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:05.939548  6400 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.046880  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.054454  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:05.975242  6051 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:06.067276  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:06.016129  6468 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.072118  1645 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:06.074477  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:06.083268  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48342.3us
I20260709 13:03:06.022782  6396 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:06.084858  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:06.049806  6457 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:06.048648  6286 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:06.062399  6363 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:06.097462  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.070987  6467 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:06.096359  6354 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:06.067358  6084 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.101447  1568 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:03:06.103834  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.104101  2371 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.104981  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:06.079290  6509 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.120510  2374 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:06.094337  6400 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:06.108619  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:06.086532  6477 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.099715  6353 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:05.980731  6413 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.107304  1642 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:03:06.150521  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62312.4us
I20260709 13:03:06.079473  6499 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:03:06.095538  6464 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:06.179795  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:06.094139  6396 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.188861  1568 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:06.105268  6364 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:06.190817  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:06.206732  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 312716us
I20260709 13:03:06.111621  6463 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.149246  6473 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:06.203717  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95210.5us
I20260709 13:03:06.107064  6354 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.220868  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.193794  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:06.101356  6227 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:03:06.106021  6457 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:06.111862  6467 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.104935  6286 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:06.252408  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.257951  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.247098  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:06.259861  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.201611  6084 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:06.200666  6363 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:06.247691  6354 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:06.267194  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73491.6us
I20260709 13:03:06.220888  6348 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:06.278512  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:06.279215  2371 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:06.190948  6477 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.320896  6388 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:06.321431  6388 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:06.266884  6084 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:06.271836  6332 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:06.245368  6463 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.322288  6388 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.200043  6051 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.277096  6348 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:06.337877  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.314471  6140 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.322983  6362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.567s	user 0.017s	sys 0.034s
I20260709 13:03:06.342639  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:06.350195  2371 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:06.314919  6500 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:06.331110  6473 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.314427  6452 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:03:06.353204  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:06.363173  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 308824us
I20260709 13:03:06.317229  6373 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:03:06.332959  6332 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:06.276820  6363 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.191798  1467 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 11, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.347332  6348 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:06.379475  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:06.351562  6362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.613s	user 0.018s	sys 0.034s
I20260709 13:03:06.369952  1539 leader_election.cc:400] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13.
I20260709 13:03:06.386715  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.387570  1642 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:06.386806  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.390257  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.390437  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:06.361769  6051 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.405414  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.367193  6511 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.407483  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.369212  6388 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.372313  6332 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.407554  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:06.416394  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.414897  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.417193  1642 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:06.417364  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:06.418196  6509 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:03:06.419685  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 137583us
I20260709 13:03:06.353212  6140 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.423290  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.428506  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.430574  6286 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:06.369702  6457 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.411441  6530 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:03:06.418849  6541 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.439567  1642 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:06.534322  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:06.535387  1642 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:06.439083  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:06.445858  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.371614  6473 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.449744  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:06.475376  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.450956  6396 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:06.479908  1646 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:06.496304  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:06.499693  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:06.501262  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 110881us
I20260709 13:03:06.537983  2375 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:06.545854  1717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.007s	sys 0.006s
I20260709 13:03:06.481029  6467 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:06.401510  6084 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.495994  6364 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:03:06.521688  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 186164us
I20260709 13:03:06.563086  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2624) took 1356 ms. Trace:
W20260709 13:03:06.539561  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 111214us
I20260709 13:03:06.539979  1642 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:03:06.543757  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 303305us
W20260709 13:03:06.564440  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.866s (SENT)
I20260709 13:03:06.551154  2371 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:06.475915  6531 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:06.563582  1575 rpcz_store.cc:276] 0709 13:03:05.206201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.399909 (+193708us) service_pool.cc:224] Handling call
0709 13:03:05.400393 (+   484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:05.429151 (+ 28758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.429189 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:05.434424 (+  5235us) log.cc:844] Serialized 52 byte log entry
0709 13:03:05.831628 (+397204us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b180034d500. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:06.198033 (+366405us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:06.404941 (+206908us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.543261 (+138320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.543362 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.543417 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:06.562070 (+ 18653us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.562931 (+   861us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25130,"prepare.queue_time_us":270534,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":29166,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":4096,"spinlock_wait_cycles":18402688,"thread_start_us":674797,"threads_started":3,"wal-append.queue_time_us":883320}
I20260709 13:03:06.577000  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:06.580039  1645 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.581636  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.587482  1651 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:03:06.542636  6511 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.521477  6534 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 13:03:06.586227  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87948.1us
I20260709 13:03:06.561992  6477 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:06.596891   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:06.574575  6531 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:06.586222  6289 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.697s	user 0.013s	sys 0.001s
I20260709 13:03:06.566134  6540 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.580744  6396 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.578625  6463 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:06.623591  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.584403  6535 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.642040  1648 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:06.641564  1651 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:06.651491  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:06.673813  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.626075  6477 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:06.691347  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.622997  6531 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.695385  1651 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:06.699579  1724 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:06.653573  6363 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:06.706382  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 143797us
I20260709 13:03:06.707432  1725 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:06.709378  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 118428us
I20260709 13:03:06.717458  1648 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:06.659166  6467 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:03:06.720014  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.721601  1648 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:06.724030  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.703923  6396 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:06.725406  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 148551us
I20260709 13:03:06.748824  1645 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [6.4-11.7]
I20260709 13:03:06.709127  6442 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.719905  6466 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.868s	user 0.015s	sys 0.000s
I20260709 13:03:06.743759  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:06.711747  6468 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:06.714063  6140 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:06.757686  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:06.701565  6477 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.727001  6396 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:06.724817  6363 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:06.785562  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:06.786330  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:06.786963  1721 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:06.787688  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:06.788961  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.788275  1720 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:06.786361  1719 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:06.802260  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:06.806502  1720 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:06.809442  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.811264  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:06.779713  6396 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.814448  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:06.815668  6536 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:03:06.790114  6513 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.848s	user 0.013s	sys 0.000s
I20260709 13:03:06.817500  1717 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:06.801206  6538 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:03:06.800725  6526 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:03:06.758738  6442 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:06.801446  6553 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:03:06.815922  6547 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:03:06.815862  6564 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:06.816728  6539 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:03:06.821060  6513 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.812626  6540 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:06.816237  6565 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:06.822134  6477 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:06.801100  6555 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
W20260709 13:03:06.819307  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 222983us
W20260709 13:03:06.851351  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96669.8us
I20260709 13:03:06.769814  6468 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.826334  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:06.859119  1719 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:06.836679  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2121) took 1266 ms. Trace:
I20260709 13:03:06.826606  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.816936  6467 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:06.809918  6473 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:06.816459  6546 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:06.852223  6051 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:06.853729  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:06.857249  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:06.856385  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:06.865612  1716 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.860179  1573 rpcz_store.cc:276] 0709 13:03:05.570382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.940083 (+369701us) service_pool.cc:224] Handling call
0709 13:03:05.960937 (+ 20854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.979243 (+ 18306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:05.979280 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:05.980576 (+  1296us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.371491 (+390915us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:06.371551 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.371651 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:06.836387 (+464736us) raft_consensus.cc:1685] finished
0709 13:03:06.836415 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:06.836621 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43855,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":25987,"thread_start_us":317800,"threads_started":1,"wal-append.queue_time_us":480838}
I20260709 13:03:06.823475  6556 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:06.839498  6509 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.843962  6564 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:06.868052  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.869418  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:06.860378  6545 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:03:06.878939  6556 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.882493  6566 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:06.883006  6566 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:06.883224  6556 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:06.869587  6567 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
W20260709 13:03:06.884886  6549 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:03:06.886358  6516 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.000s
I20260709 13:03:06.886796  6516 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:06.887951  1570 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:06.889369  6533 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:06.889637  6468 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:06.889767  6533 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:06.890836  6468 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:06.890790  6533 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:06.893813  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 67359.2us
I20260709 13:03:06.907346  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.895208  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:06.905695  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:06.905815  1575 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.864784  6467 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:06.916201  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.864490  6546 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:06.926187  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 182495us
W20260709 13:03:06.926889  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72548.7us
I20260709 13:03:06.863920  6051 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:06.883422  6566 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:06.931454  1717 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:06.876019  6364 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:06.944087  1572 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.944581  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:06.875834  6289 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:06.892292  6533 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.940819  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:06.877272  6564 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.946681  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.926187  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:06.939591  6467 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:06.913892  6555 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:06.888183  6477 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 13:03:06.968108  6348 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.001s
I20260709 13:03:06.858515  6540 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.948057  6289 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.959993  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:06.961031  6558 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:06.961431  6550 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:06.964594  6565 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.945238  6051 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:06.969319  6557 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:03:06.970711  6354 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.000s
I20260709 13:03:07.001888  6567 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:07.001983  6354 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.002768  6567 consensus_queue.cc:1243] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:07.004251  6354 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:06.972354  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46234.4us
I20260709 13:03:07.004830  6546 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:06.892607  6509 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.006119  6546 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:07.007089  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 249472us
I20260709 13:03:07.007817  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:06.972082  6568 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:03:07.012172  6582 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:07.012884  6582 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:07.014052  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1877) took 1046 ms. Trace:
I20260709 13:03:07.013845  6582 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.018491  1614 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:06.997690  6576 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:07.021855  6513 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:07.022001  6534 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:07.022604  6534 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:07.023402  1720 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:06.979151  6473 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:06.948410  6325 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:06.868948  6412 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.973225  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:06.948119  6362 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.944599  6364 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:06.982615  6555 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:06.951777  6575 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:06.944214  6556 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:03:07.027853  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.029191  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:07.055760  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:06.956521  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:07.014515  1649 rpcz_store.cc:276] 0709 13:03:05.967259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.025380 (+ 58121us) service_pool.cc:224] Handling call
0709 13:03:06.025754 (+   374us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:06.045114 (+ 19360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.045145 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:06.066006 (+ 20861us) log.cc:844] Serialized 78 byte log entry
0709 13:03:06.333081 (+267075us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:06.524613 (+191532us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.545583 (+ 20970us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.710799 (+165216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.710873 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.013761 (+302888us) raft_consensus.cc:1685] finished
0709 13:03:07.013786 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.013981 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57090,"prepare.queue_time_us":463691,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":6867,"raft.queue_time_us":214555,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":44731,"spinlock_wait_cycles":24391552,"thread_start_us":550998,"threads_started":3,"wal-append.queue_time_us":434757}
I20260709 13:03:07.030666  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:07.031817  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.033926  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.059592  1721 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:03:07.036631  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.038156  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.041232  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.008163  6509 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:06.987493  6289 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:06.979842  6400 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.002s
I20260709 13:03:07.051190  6516 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.033175  6473 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:07.024042  6530 consensus_queue.cc:1175] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4f737473248e4c059fa74af3b8e1f77e" responder_term: 12 status { last_received { term: 12 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 12 index: 8 } }
I20260709 13:03:07.042269  6325 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:07.052920  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79899.7us
I20260709 13:03:07.061316  1572 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:07.063576  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:07.063676  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.169900  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.065639  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:07.072469  1722 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:07.041934  6558 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.088933  6567 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.056298  6580 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:07.089887  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.175474  1723 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:07.140786  1568 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:03:07.177104  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 113517us
I20260709 13:03:06.987555  6348 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.177991  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:07.181931  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.182839  1717 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:07.184063  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:07.175555  6325 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:07.188854  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2335) took 1585 ms (client timeout 1630 ms). Trace:
I20260709 13:03:07.189016  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:07.189338  1725 rpcz_store.cc:269] 0709 13:03:05.603270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.607358 (+  4088us) service_pool.cc:224] Handling call
0709 13:03:07.188683 (+1581325us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400232654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:07.188797 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2373519616}
I20260709 13:03:07.191047  1567 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:07.191879  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 526) took 1603 ms. Trace:
I20260709 13:03:07.133778  6362 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:07.200208  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.139779  6473 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:07.199307  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.199745  1569 rpcz_store.cc:276] 0709 13:03:05.588733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.915488 (+326755us) service_pool.cc:224] Handling call
0709 13:03:07.188043 (+1272555us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54001aa4d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:07.191814 (+  3771us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354414464}
W20260709 13:03:07.224831  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 268414us
I20260709 13:03:07.230021  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:07.228875  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 222523us
I20260709 13:03:07.165238  6575 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.181011  6580 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:07.236680  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52741.4us
I20260709 13:03:07.239651  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:07.202142  6576 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:07.246999  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:07.228597  6362 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:07.229343  6577 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:07.185748  6558 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.234275  6227 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:07.251684  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.235149  6509 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.237071  6353 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:07.255730  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.279460  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:07.245934  6578 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:07.254259  6577 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.258658  6362 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.262485  6558 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:07.264051  6572 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 13:03:07.272367  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 94444.8us
I20260709 13:03:07.242467  6580 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:07.268883  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:07.249583  6575 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.237639  6550 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:03:07.290201  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.255052  6571 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:07.269451  6227 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:07.255810  6581 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:07.211699  6534 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.293509  6464 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:07.319156  2373 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:07.319561  6464 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:07.284003  6353 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.320482  6593 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.320964  6464 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.321319  6593 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.322386  6464 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.322676  6353 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:07.281203  6540 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:07.319837  6594 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:03:07.281484  6578 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:07.284397  6577 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:07.328974  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 77348.5us
I20260709 13:03:07.286839  6574 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:07.335354  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:07.235682  6364 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.230959  6555 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:07.341233  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51149.2us
I20260709 13:03:07.345566  6227 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.351096  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:07.346976  6568 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:07.348248  6353 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.324096  6593 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.317296  6585 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.281044  6509 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.366936  6509 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:07.316658  6571 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:07.367322  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:07.367831  6571 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.368592  6595 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:07.369130  6595 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:07.369616  6595 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.369971  6051 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.328101  6362 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.275081  6566 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:07.374076  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 130419us
I20260709 13:03:07.338949  6578 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.331595  6581 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:07.338105  6574 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.327625  6499 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:07.375838  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:07.361338  6396 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.524s	user 0.004s	sys 0.008s
I20260709 13:03:07.358762  6546 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:07.384817  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:07.315033  6592 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:03:07.356588  6568 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.381026  6567 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:07.387398  6593 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:07.356073  6556 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:07.381567  6227 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:07.387478  6587 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:03:07.421223  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2137) took 1616 ms. Trace:
I20260709 13:03:07.474766  1567 rpcz_store.cc:276] 0709 13:03:05.804586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.045588 (+241002us) service_pool.cc:224] Handling call
0709 13:03:07.188244 (+1142656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.191450 (+  3206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.420892 (+229442us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001aa4d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:07.421136 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2627243776}
W20260709 13:03:07.395283  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 101426us
I20260709 13:03:07.357951  6353 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.462615  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 563) took 1216 ms. Trace:
W20260709 13:03:07.405488  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.413534  6589 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:03:07.385577  6574 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.460738  6548 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:07.394114  6546 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:07.462162  6332 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.035s	user 0.010s	sys 0.002s
W20260709 13:03:07.452706  6541 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.884s	user 0.007s	sys 0.007s
I20260709 13:03:07.435909  6576 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 13:03:07.413534  6586 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:03:07.389801  6499 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.538358  6473 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.542829  6140 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:07.543006  6289 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:07.564020  6412 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.006s
I20260709 13:03:07.611531  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:07.484761  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:07.477154  6509 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:07.614162  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:07.455207  6531 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.753s	user 0.008s	sys 0.002s
I20260709 13:03:07.621232  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2141) took 1731 ms. Trace:
I20260709 13:03:07.621948  1570 rpcz_store.cc:276] 0709 13:03:05.889737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.097617 (+207880us) service_pool.cc:224] Handling call
0709 13:03:06.098013 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.888652 (+790639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.620823 (+732171us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54000262d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:07.621152 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816249088}
I20260709 13:03:07.395779  6540 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:07.546835  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.494733  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2356) took 1633 ms. Trace:
W20260709 13:03:07.455207  6388 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.033s	user 0.012s	sys 0.000s
I20260709 13:03:07.440130  6348 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:07.431597  6597 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.004s
I20260709 13:03:07.482038  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.545305  2374 rpcz_store.cc:276] 0709 13:03:06.246521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.246673 (+   152us) service_pool.cc:224] Handling call
0709 13:03:06.247069 (+   396us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:06.247463 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.247497 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:06.248378 (+   881us) log.cc:844] Serialized 78 byte log entry
0709 13:03:06.497242 (+248864us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:06.718034 (+220792us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.718592 (+   558us) log.cc:844] Serialized 14 byte log entry
0709 13:03:06.719022 (+   430us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.461924 (+742902us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b1800478720. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287
0709 13:03:07.462099 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.462280 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.462324 (+    44us) raft_consensus.cc:1685] finished
0709 13:03:07.462336 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.462509 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13182,"prepare.queue_time_us":395353,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":58,"raft.queue_time_us":480107,"raft.run_cpu_time_us":719,"raft.run_wall_time_us":83759,"spinlock_wait_cycles":20642432,"thread_start_us":597271,"threads_started":3,"wal-append.queue_time_us":440639}
I20260709 13:03:07.549268  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:07.527606  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:07.437073  6051 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.491923  6442 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.550367  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.556913  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.557619  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2357) took 1695 ms. Trace:
I20260709 13:03:07.462849  6558 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:07.467420  6567 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.462198  6566 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.660959  6601 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:07.467358  6593 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:07.536419  6427 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.662446  6593 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.627599  6509 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.662617  6427 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.664582  6540 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.665254  6540 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.667984  6540 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:07.498919  6533 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.001s
W20260709 13:03:07.562887  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 174738us
I20260709 13:03:07.669425  1717 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:07.590653  6332 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.394374  6396 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.586277  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:07.418270  6575 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 13:03:07.601715  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:07.604301  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:07.480969  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.524s (SENT)
I20260709 13:03:07.429920  6467 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.612977  1576 raft_consensus.cc:2376] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 9. Re-sending same reply.
I20260709 13:03:07.398101  6568 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.562474  6546 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:07.563639  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.547914  6548 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:07.624044  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:07.590710  6602 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.628077  1725 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:07.627642  1716 rpcz_store.cc:276] 0709 13:03:05.861437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.874248 (+ 12811us) service_pool.cc:224] Handling call
0709 13:03:05.874568 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.866254 (+991686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.494348 (+628094us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400233a54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:07.494644 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278801792}
I20260709 13:03:07.481906  6550 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:03:07.482056  6516 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.138s
I20260709 13:03:07.640164  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.650962  6388 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:07.658382  6580 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:07.491102  6581 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:07.689872  6564 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.715s	user 0.016s	sys 0.000s
I20260709 13:03:07.797472  1650 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.661666  6601 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.597615  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:07.671000  1721 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:07.671209  1574 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:07.672703  1571 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:07.806985  2380 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:07.808434  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:07.809516  1571 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.671811  1724 rpcz_store.cc:276] 0709 13:03:05.861773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.877538 (+ 15765us) service_pool.cc:224] Handling call
0709 13:03:05.877804 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.700100 (+822296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.557225 (+857125us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b5400233554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:07.557452 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889542016}
I20260709 13:03:07.686509  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.712016  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:07.721583  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.722492  1570 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:07.693610  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.814843  1720 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:03:07.718088  1648 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 13:03:07.689888  1723 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.723598  1567 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:07.816604  1569 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.816906  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.817902  1567 raft_consensus.cc:2410] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:07.689122  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:07.695935  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:07.729514  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.760145  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:07.767087  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.700508  6353 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:07.625811  6541 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:07.702466  6325 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.016s
I20260709 13:03:07.787561  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:07.660531  6567 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:07.789000  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 225695us
I20260709 13:03:07.790357  1572 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:07.792102  1646 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:07.697597  6442 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.799412  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:07.664631  6348 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:07.804311  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.819721  1568 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 8.
I20260709 13:03:07.686851  6413 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.698242  6355 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 13:03:07.700158  6473 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.766040  6602 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.701648  6499 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:07.671496  6558 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:07.902628  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:07.905045  1721 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:07.908077  1723 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:07.910945  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.630232  6531 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:07.836570  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 355647us
I20260709 13:03:07.837877  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.916996  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:07.841532  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:07.842360  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:07.845579  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.921046  1716 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:07.850181  1644 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.853313  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.841590  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.924877  1567 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 8.
I20260709 13:03:07.926380  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.927284  1567 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:07.841840  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:07.929414  1645 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.842104  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:07.933218  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.943179  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.945588  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:07.961532  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 116065us
I20260709 13:03:07.962843  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.949501  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:07.973940  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.788745  6447 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.976027  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.986392  1717 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:07.978128  1642 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:07.992865  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.997298  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:07.984757  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.999063  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:07.855480  6355 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:07.936528  6592 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:08.005321  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:07.998593  1574 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:07.850127  6477 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.944633  6499 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.793741  6388 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.890985  6332 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:07.883867  6602 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:07.885608  6467 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.026126  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:07.982038  6565 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.700217  6577 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.055455  6412 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:08.017890  6566 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:08.008948  6355 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:07.835043  6595 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.002974  6445 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:03:08.072106  6227 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.004s
I20260709 13:03:08.071835  6457 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.995095  6051 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:08.072893  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:08.218967  1576 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.850569  6348 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.009222  6576 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:07.849783  6442 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:08.081456  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55457.8us
I20260709 13:03:08.223273  1646 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:08.052166  1571 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:08.224484  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:08.224967  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:07.842173  6325 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.238317  6348 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.239370  1724 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:08.239873  6325 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.017606  6592 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:08.241293  6592 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.242794  6592 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.243659  6442 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:08.093938  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:08.244282  6353 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.244764  1650 raft_consensus.cc:2376] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
I20260709 13:03:08.244879  6353 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:07.854305  6413 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.247527  6353 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:07.864923  6567 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.248438  6533 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:08.249089  6533 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.249800  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:08.250691  1575 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:08.100160  1649 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:07.838496  6564 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.049021  6400 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:08.062106  6578 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.003s
I20260709 13:03:08.042181  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:08.082363  6362 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.710s	user 0.013s	sys 0.003s
I20260709 13:03:08.058627  6575 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:08.255332  6400 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.057037  6558 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.256644  6362 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:08.257341  6558 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.257376  6362 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.190148  6609 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.190727  6588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.033s	sys 0.042s
I20260709 13:03:08.271524  1724 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:08.090281  6412 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:08.088739  6516 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.095585  6332 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:08.102741  6605 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.044404  6499 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:08.198773  6571 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.830s	user 0.010s	sys 0.007s
I20260709 13:03:08.192227  6364 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.198189  6396 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.005820  6447 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.125092  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:08.217034  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 272974us
I20260709 13:03:08.217224  1645 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:08.215996  1644 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:08.205679  6580 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.231361  1569 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:08.235816  6613 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:03:08.257925  6540 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.006036  6548 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.258252  1573 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:03:08.252967  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:08.259630  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.261605  1575 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:08.164105  6355 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:08.210000  6582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.031s	sys 0.042s
I20260709 13:03:08.270473  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:08.256599  6575 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.273679  6566 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:08.164301  6289 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.247999  6413 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:08.270143  6574 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.005s
I20260709 13:03:08.636096  6412 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:08.213172  6577 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.256026  6509 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:08.636847  6574 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:08.612543  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.360898  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:08.385969  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:08.533473  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:08.517409  1650 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:08.580232  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 192673us
W20260709 13:03:08.386543  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 364533us
I20260709 13:03:08.543749  1719 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:08.628078  1573 pending_rounds.cc:85] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:03:08.308729  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:08.546443  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 571) took 1295 ms. Trace:
W20260709 13:03:08.304678  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 389440us
I20260709 13:03:08.403293  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:08.560598  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:08.219434  6601 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:08.354454  6607 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 13:03:08.221993  6576 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:08.671438  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 539) took 1108 ms. Trace:
W20260709 13:03:08.274617  6588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.034s	sys 0.044s
I20260709 13:03:08.658597  1573 pending_rounds.cc:107] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:08.375608  1576 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:03:08.659751  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.406075  6593 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.743s	user 0.010s	sys 0.003s
I20260709 13:03:08.628659  1646 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:08.574658  1615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.013s
I20260709 13:03:08.398211  6427 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.510701  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2584) took 1393 ms. Trace:
W20260709 13:03:08.495955  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.631448  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:08.650554  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.662588  2372 rpcz_store.cc:276] 0709 13:03:07.251148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.251268 (+   120us) service_pool.cc:224] Handling call
0709 13:03:08.512098 (+1260830us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54005c30d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:08.546312 (+ 34214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040602368}
I20260709 13:03:08.666905  1651 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:08.345444  6567 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.356747  6550 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:08.262391  6457 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.272078  6609 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:08.669124  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 135825us
I20260709 13:03:08.764859  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:08.667237  1568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.004s	sys 0.020s
I20260709 13:03:08.269392  6388 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.456153  6362 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.274255  6596 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.805389  6596 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.805738  6609 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:08.807104  6627 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:03:08.807600  6596 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.666602  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:08.661523  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 352900us
I20260709 13:03:08.660426  1572 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.690580  1573 raft_consensus.cc:2889] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:08.389837  6084 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.269207  6531 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.555557  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:08.390453  6363 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:08.473982  6620 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:08.718850  1615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.006s	sys 0.016s
I20260709 13:03:08.255339  6578 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:08.777776  1568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.005s	sys 0.020s
I20260709 13:03:08.826128  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:08.751381  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:08.512617  6467 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.712548  1719 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:08.638485  6566 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.692186  1643 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:08.781728  1570 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:08.733719  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2589) took 1038 ms. Trace:
I20260709 13:03:08.672667  1567 rpcz_store.cc:276] 0709 13:03:07.563574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.928321 (+364747us) service_pool.cc:224] Handling call
0709 13:03:07.935895 (+  7574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.010961 (+ 75066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.011026 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.011846 (+   820us) log.cc:844] Serialized 26 byte log entry
0709 13:03:08.405350 (+393504us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b18003687a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:08.578290 (+172940us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:08.619318 (+ 41028us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.619672 (+   354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.619767 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:08.653917 (+ 34150us) raft_consensus.cc:1685] finished
0709 13:03:08.671226 (+ 17309us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.671378 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73805,"prepare.queue_time_us":40816,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":38,"raft.queue_time_us":183467,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":35987,"spinlock_wait_cycles":10790144,"thread_start_us":164044,"threads_started":1,"wal-append.queue_time_us":584089}
I20260709 13:03:08.832315  1720 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:03:08.733816  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2158) took 1185 ms. Trace:
I20260709 13:03:08.832458  1724 rpcz_store.cc:276] 0709 13:03:07.695007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.902206 (+207199us) service_pool.cc:224] Handling call
0709 13:03:08.733389 (+831183us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b540010dbd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:08.733607 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":772249856}
I20260709 13:03:08.833066  1569 rpcz_store.cc:276] 0709 13:03:07.548808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.548977 (+   169us) service_pool.cc:224] Handling call
0709 13:03:08.733552 (+1184575us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002152d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:08.733713 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423544192}
I20260709 13:03:08.833920  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:08.734359  1723 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:08.834664  1720 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:08.783154  1725 rpcz_store.cc:276] 0709 13:03:07.117336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.190294 (+ 72958us) service_pool.cc:224] Handling call
0709 13:03:07.190626 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.722738 (+532112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.510213 (+787475us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54001716d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:08.510595 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005481344}
W20260709 13:03:08.784155  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.634289  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:08.836776  1571 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:03:08.837219  1572 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:08.838253  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2168) took 1144 ms. Trace:
I20260709 13:03:08.838749  1571 rpcz_store.cc:276] 0709 13:03:07.693455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.837481 (+1144026us) service_pool.cc:224] Handling call
0709 13:03:08.837782 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.837961 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.837985 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.838025 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.838052 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.838099 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.838189 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.757647  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 542) took 1188 ms. Trace:
I20260709 13:03:08.840150  1575 rpcz_store.cc:276] 0709 13:03:07.568837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.251543 (+682706us) service_pool.cc:224] Handling call
0709 13:03:08.251940 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.634288 (+382348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.757253 (+122965us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b54003647d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:08.757559 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21597952}
I20260709 13:03:08.842237  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2171) took 1147 ms. Trace:
I20260709 13:03:08.842587  1575 rpcz_store.cc:276] 0709 13:03:07.694339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.841273 (+1146934us) service_pool.cc:224] Handling call
0709 13:03:08.841621 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.841833 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.841876 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.841929 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:08.841981 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.842051 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.842173 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.785465  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:08.566061  6564 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:08.352710  6614 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:03:08.543421  6610 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.639196  6332 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.797509  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:08.733947  6601 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 13:03:08.798640  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.798508  2378 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:08.454710  6400 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.801656  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:08.823901  6289 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.355795  6595 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.453637  6533 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.817456  6427 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:03:08.803076  6550 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.665510  6577 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:08.763715  6354 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:03:08.767320  6348 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:03:08.649006  6516 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.859241  6332 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.667105  6355 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:08.808724  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 142247us
I20260709 13:03:08.816268  1716 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:08.887477  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:08.809202  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:08.675186  6576 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.691787  6580 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:08.820953  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.774s (SENT)
I20260709 13:03:08.892799  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:08.821936  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:08.828439  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:08.894157  1719 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:08.766331  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:08.648669  6540 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.801492  6442 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:08.846952  1722 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:08.847592  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.691937  6445 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 13:03:08.850898  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2165) took 1210 ms. Trace:
I20260709 13:03:08.909013  1569 rpcz_store.cc:276] 0709 13:03:07.640860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.850170 (+1209310us) service_pool.cc:224] Handling call
0709 13:03:08.850430 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.850590 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.850614 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.850652 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.850681 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.850731 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:08.850832 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:08.638491  6571 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.818420  6388 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.808662  6626 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.337499  6482 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:08.859051  2371 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:08.820353  6363 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:08.950896  6363 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.333412  6555 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:08.847249  6628 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 13:03:08.863499  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61920.8us
I20260709 13:03:08.892493  6580 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:08.866508  6457 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.800730  6593 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.778946  6581 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:03:08.872593  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2157) took 1325 ms. Trace:
I20260709 13:03:08.979202  1573 rpcz_store.cc:276] 0709 13:03:07.547536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.547824 (+   288us) service_pool.cc:224] Handling call
0709 13:03:07.548256 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.812865 (+1264609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.872326 (+ 59461us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54002b0054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:08.872534 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631960832}
I20260709 13:03:08.980158  1719 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:03:08.981109  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:08.981843  1573 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:03:08.982812  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:08.983863  1573 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:03:08.879745  1720 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:08.665993  6511 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:08.869220  6565 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.889758  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80638us
I20260709 13:03:08.988948  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:08.894662  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 117291us
I20260709 13:03:08.849987  6467 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:08.870900  6596 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.010330  6596 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.011469  6467 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:08.902849  6564 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:08.904950  6592 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.005s
I20260709 13:03:08.905499  6540 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.884752  6533 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:09.012653  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:08.884472  6595 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.014233  6550 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.896833  6534 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:08.949523  6482 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:09.013252  6533 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.015125  6534 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.015583  6534 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:09.016870  1567 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:08.919516  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:09.017108  6533 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:08.909533  6558 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.651s	user 0.013s	sys 0.000s
W20260709 13:03:09.017859  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 98520.5us
I20260709 13:03:09.018105  6558 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:09.018875  6558 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:09.019547  6558 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:08.908514  6568 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.129s	user 0.018s	sys 0.000s
I20260709 13:03:09.021001  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:09.020925  6568 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.014595  6595 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.877430  6289 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.024021  6595 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.025800  6289 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:08.971474  6580 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.025935  6540 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.027257  6568 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.027634  6540 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.028132  6540 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.028594  1576 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:03:09.029026  1576 pending_rounds.cc:85] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:03:09.029275  1576 pending_rounds.cc:107] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:09.029520  1576 raft_consensus.cc:2889] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.031561  6571 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.865788  6353 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.028143  6568 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:09.033154  6596 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.960742  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:09.032773  6353 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.993345  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.034578  2375 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:08.968622  6363 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:08.996837  6636 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:08.969547  6546 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.181s	user 0.017s	sys 0.000s
I20260709 13:03:09.000816  1572 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:09.002876  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2271) took 1403 ms. Trace:
W20260709 13:03:09.004945  6629 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:03:09.004091  6578 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.005550  6566 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.008970  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:09.010761  6516 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.013588  2380 raft_consensus.cc:2368] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:09.077024  1647 rpcz_store.cc:276] 0709 13:03:07.599431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.599708 (+   277us) service_pool.cc:224] Handling call
0709 13:03:07.615801 (+ 16093us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:07.616468 (+   667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.616518 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:07.627412 (+ 10894us) log.cc:844] Serialized 78 byte log entry
0709 13:03:07.974116 (+346704us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:08.306715 (+332599us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.327703 (+ 20988us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.327918 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.328016 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.002540 (+674524us) raft_consensus.cc:1685] finished
0709 13:03:09.002565 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.002774 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73655,"prepare.queue_time_us":110188,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":99,"raft.queue_time_us":45117,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":25836,"thread_start_us":533554,"threads_started":2,"wal-append.queue_time_us":529022}
I20260709 13:03:09.014487  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:08.907612  1574 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:08.836398  6620 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.016108  6482 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:08.935810  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.114419  1569 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:09.021463  2378 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:03:08.954926  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 174452us
I20260709 13:03:08.862958  6610 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:09.033805  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77581.4us
I20260709 13:03:08.935982  6575 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:08.990041  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:09.130806  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 140843us
I20260709 13:03:09.073159  6605 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:09.035570  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.039963  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:09.078497  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:09.140180  1573 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:09.140517  2374 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.953718  6626 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.062146  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:08.987210  6499 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:08.937129  6609 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:03:09.044173  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:08.970685  6574 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.160085  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:09.100437  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.161049  2378 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:09.102161  2377 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:09.105595  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.162420  1716 raft_consensus.cc:2368] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:03:09.106388  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.107100  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.163329  1721 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.107775  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.164001  1724 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.034318  6353 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.107736  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:09.041783  6634 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.369s	user 0.002s	sys 0.000s
I20260709 13:03:09.171348  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.100787  6595 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:03:08.991189  6457 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.952730  6555 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.115552  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:09.109683  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.124136  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:09.184496  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60490.1us
I20260709 13:03:09.185261  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:09.038749  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.133237  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:09.145840  6582 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:09.150461  1569 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:09.154305  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.154644  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.189664  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.155619  6463 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.168556  6575 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.193849  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:09.155514  6576 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:09.156672  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 335777us
I20260709 13:03:09.156968  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:09.201756  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.000s (SENT)
I20260709 13:03:09.202031  1650 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.203843  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:09.122538  6568 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.206212  6568 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.158464  1720 raft_consensus.cc:2368] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:09.208021  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.209211  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.210580  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.210882  6555 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.211928  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.165642  2374 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:09.167833  1574 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:09.169258  2376 raft_consensus.cc:2410] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:09.097404  6445 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:03:09.171813  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:09.217227  1644 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:09.112882  2372 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:09.218498  1650 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:09.172291  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.219245  1647 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:09.025591  6565 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.167440  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.221221  1643 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.174135  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.143180  6566 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:09.177134  1573 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:09.221832  2378 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.221949  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.222630  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.222690  2377 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:09.097826  6546 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.121868  6592 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:03:09.034291  6289 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:09.233948  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:09.186081  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 70627us
I20260709 13:03:09.186910  2375 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:09.189710  1646 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:09.160928  6388 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:09.237018  6388 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.239815  6388 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:09.195771  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 117357us
I20260709 13:03:09.254581  6568 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.229008  6555 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:03:09.113735  6564 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:09.214035  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:09.181619  6457 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.261194  6457 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.130126  6541 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:09.261968  6541 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.225510  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.191382  6637 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:09.263866  2374 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:09.264648  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:09.265301  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51428.1us
I20260709 13:03:09.190790  6499 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:09.099295  6636 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.272976  6457 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:09.273876  6412 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.637s	user 0.013s	sys 0.000s
I20260709 13:03:09.274644  6412 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.216137  6354 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:09.222105  6442 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:09.277720  6442 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.280069  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:09.281108  1722 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.286736  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2169) took 1593 ms. Trace:
I20260709 13:03:09.287298  1572 rpcz_store.cc:276] 0709 13:03:07.693764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.837838 (+1144074us) service_pool.cc:224] Handling call
0709 13:03:08.838070 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.077152 (+239082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.286408 (+209256us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54000cbf54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:09.286650 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373750144}
I20260709 13:03:09.290510  6325 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:09.291054  6325 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.291447  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 541) took 1723 ms. Trace:
I20260709 13:03:09.292204  1568 rpcz_store.cc:276] 0709 13:03:07.568376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.961924 (+393548us) service_pool.cc:224] Handling call
0709 13:03:07.962496 (+   572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.826375 (+863879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.826413 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.846902 (+ 20489us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.097105 (+250203us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:09.121038 (+ 23933us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.125329 (+  4291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.125433 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.291092 (+165659us) raft_consensus.cc:1685] finished
0709 13:03:09.291132 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.291348 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69845,"prepare.queue_time_us":23875,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4536,"raft.queue_time_us":157038,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":37175,"spinlock_wait_cycles":19331840,"thread_start_us":1015909,"threads_started":2,"wal-append.queue_time_us":335684}
I20260709 13:03:09.295778  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2162) took 1669 ms. Trace:
I20260709 13:03:09.296330  1576 rpcz_store.cc:276] 0709 13:03:07.626702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.782530 (+1155828us) service_pool.cc:224] Handling call
0709 13:03:08.782896 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.029992 (+247096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.295422 (+265430us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54001aaed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:09.295682 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564067072}
I20260709 13:03:09.298861  6626 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.300838  1724 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:03:09.300899  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.168905  6647 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:03:09.303321  6568 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:09.303772  6568 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:09.304843  6568 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.223026  6533 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:09.305892  6534 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.242861  6289 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.223562  6348 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:09.307260  6348 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:09.244211  6364 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.002s
I20260709 13:03:09.255769  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:09.307662  6289 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:09.308326  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52724.7us
I20260709 13:03:09.307672  6348 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:09.310163  6544 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:09.310823  6544 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:09.202019  6463 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.312991  6362 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.313393  6051 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:09.313505  6362 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.315009  6567 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:09.315407  6567 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:09.241530  6413 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:09.316996  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2164) took 1679 ms. Trace:
I20260709 13:03:09.212697  6609 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.319082  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2174) took 1314 ms. Trace:
I20260709 13:03:09.318470  6609 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.319495  1575 rpcz_store.cc:276] 0709 13:03:08.005208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.854527 (+849319us) service_pool.cc:224] Handling call
0709 13:03:08.854940 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.957587 (+102647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.957618 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.958158 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.105614 (+147456us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:09.105666 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.318722 (+213056us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b540034fb54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:09.318772 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:09.318821 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:09.318835 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.319029 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3855,"raft.queue_time_us":34370,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":127,"spinlock_wait_cycles":226134784,"thread_start_us":147108,"threads_started":1,"wal-append.queue_time_us":188500}
I20260709 13:03:09.321487  6447 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:09.242090  6546 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.322134  6571 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:09.213258  6575 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.323342  6400 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:09.323776  6400 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:09.324575  6400 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.325500  6400 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:09.326960  6580 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:09.266494  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.226240  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.328958  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2170) took 1634 ms. Trace:
I20260709 13:03:09.329082  2373 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:09.329465  1571 rpcz_store.cc:276] 0709 13:03:07.694050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.839922 (+1145872us) service_pool.cc:224] Handling call
0709 13:03:08.848149 (+  8227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.891941 (+ 43792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.891984 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:08.892037 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:08.892082 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.328707 (+436625us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b5400309b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:09.328756 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:09.328904 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100173696}
I20260709 13:03:09.332672  6593 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.333396  6610 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:09.220038  6582 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:09.334937  6582 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.267019  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:09.339371  6582 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.235762  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:09.312018  6544 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.262372  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 543) took 1693 ms. Trace:
I20260709 13:03:09.278411  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:09.216423  6235 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:03:09.316668  6567 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.317410  1567 rpcz_store.cc:276] 0709 13:03:07.637331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.849714 (+1212383us) service_pool.cc:224] Handling call
0709 13:03:08.850001 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.017284 (+167283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.316762 (+299478us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400214dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:09.316938 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383024384}
I20260709 13:03:09.322562  6571 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:09.339705  6601 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:09.268479  1644 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:09.200862  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.301748  6620 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.265196  1569 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:09.267987  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:09.354218  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 233564us
I20260709 13:03:09.284039  1722 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:03:09.390146  1722 pending_rounds.cc:85] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 7
I20260709 13:03:09.390385  1722 pending_rounds.cc:107] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:03:09.390618  1722 raft_consensus.cc:2889] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.359807  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.374696  1570 rpcz_store.cc:276] 0709 13:03:07.569150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.681868 (+1112718us) service_pool.cc:224] Handling call
0709 13:03:08.781429 (+ 99561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.831992 (+ 50563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.261995 (+430003us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b54002a6cd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:09.262264 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227787648}
I20260709 13:03:09.352000  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.375697  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.228437  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:09.343921  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 76980.8us
I20260709 13:03:09.375002  1725 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.355419  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.355470  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.379063  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:09.393460  1646 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.393401  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.351963  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:09.386214  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:09.395367  1644 raft_consensus.cc:2368] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:09.389771  1716 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.396517  1642 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:09.397270  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:09.397421  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.398034  1643 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.398294  1539 leader_election.cc:400] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:03:09.398648  1539 leader_election.cc:403] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:09.391522  1572 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:09.398128  1576 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:09.399422  1643 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [9.8-12.10]
I20260709 13:03:09.399591  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.400637  1539 leader_election.cc:400] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:09.401069  1539 leader_election.cc:403] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:09.350461  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:09.403337  2371 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:03:09.408424  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:09.407819  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:09.411415  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.412269  1575 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:09.414810  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64544.6us
I20260709 13:03:09.416067  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:09.416271  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:09.414832  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:09.263648  6566 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:09.417814  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 182103us
I20260709 13:03:09.419991  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:09.421595  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.423120  1646 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:09.426828  1575 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:09.427683  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2638) took 1254 ms. Trace:
I20260709 13:03:09.428119  1575 rpcz_store.cc:276] 0709 13:03:08.173295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.413055 (+1239760us) service_pool.cc:224] Handling call
0709 13:03:09.413381 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.427360 (+ 13979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.427613 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:09.430390  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:09.341197  6636 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.431384  1644 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [11.9-12.10]
I20260709 13:03:09.432190  1645 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 13:03:09.437214  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:09.426268  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.432788  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.433432  2373 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:03:09.380177  6565 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.378480  6637 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.444481  1650 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:09.447526  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.447685  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:09.448446  1567 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:09.399984  1570 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.448576  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:09.448988  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.449534  1646 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
W20260709 13:03:09.447568  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 223237us
I20260709 13:03:09.451268  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.383847  6354 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.456472  1716 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:09.451969  1645 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.452385  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.454167  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:09.459798  1568 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:09.460919  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.410000  6593 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:03:09.403447  6548 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:09.464733  1722 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:09.460986  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:09.466373  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.467070  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.467983  1718 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:09.470937  1646 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:09.510416  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.431562  6567 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.424391  6582 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:09.429173  6578 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:09.430980  6620 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
W20260709 13:03:09.525516  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64619us
I20260709 13:03:09.529027  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.529907  1716 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:09.530079  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:09.530967  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:09.462534  6447 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:09.471300  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.461936  6084 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:09.292107  6325 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.515389  1720 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:09.450600  6642 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:03:09.456295  6585 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:09.462378  6413 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:09.461294  6546 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:09.469141  6533 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.508877  6466 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:03:09.554812  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.487056  1725 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:09.468572  6595 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:09.471300  6530 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:09.450428  6574 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:03:09.468890  6564 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:09.509334  6463 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.455452  6626 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:09.512796  6509 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.434283  6540 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.507422  6427 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:03:09.583745  6534 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.508877  6601 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:03:09.517527  6592 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:03:09.588798  6575 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.580138  6447 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.590538  6533 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.514101  6571 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.461632  6457 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:09.598130  6355 raft_consensus.cc:2764] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:03:09.597694  6546 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:09.602626  6595 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.630311  6509 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.584609  6637 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.619004  6564 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.677927  6564 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:09.679562  6661 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:09.666265  6580 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.605185  6413 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.615988  6530 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:09.666386  6576 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.666168  6447 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.645869  6533 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.668144  6546 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.669617  6457 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.670827  6571 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:09.700507  6457 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:09.672624  6595 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.550076  6582 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.708547  6530 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.598806  6566 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.712515  6582 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:09.716874  6558 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.697s	user 0.013s	sys 0.000s
I20260709 13:03:09.692502  6540 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.726745  6558 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.729382  6558 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.469542  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:09.777546  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:09.778829  1573 raft_consensus.cc:2368] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14.
I20260709 13:03:09.779524  1568 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:09.782987  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.469673  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.785975  1576 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.787822  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:09.789201  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 324594us
I20260709 13:03:09.480186  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.549420  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.564126  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:09.824851  1568 raft_consensus.cc:1217] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[9.6-13.8]   Dedup: 9.6->[11.7-13.8]
I20260709 13:03:09.614194  6354 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.825793  1568 pending_rounds.cc:85] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:03:09.826567  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.570094  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.826187  1568 pending_rounds.cc:107] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:09.592424  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:09.573928  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 358429us
I20260709 13:03:09.586956  6084 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:09.834152  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.089s (SENT) [suppressed 16 similar messages]
I20260709 13:03:09.586174  6235 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:09.836419  6235 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.838619  6235 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:09.670729  6363 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.000s
I20260709 13:03:09.832985  6540 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:09.619966  6325 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:09.841918  6540 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.634847  6466 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:09.845079  6466 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:09.831941  1568 raft_consensus.cc:2889] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.846956  6540 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.801855  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:09.801918  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:09.820024  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.850226  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:09.850750  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.850549  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:09.852489  6574 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:09.856186  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:09.847237  6466 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.876956  6534 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.878247  6534 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.881078  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.883018  1650 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:09.622720  6362 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.886924  1724 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:09.833424  6592 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:09.899264  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.902165  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.590844  1722 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:09.890028  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.890362  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:09.832841  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 363405us
I20260709 13:03:09.895949  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:09.906539  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50913.1us
I20260709 13:03:09.833458  1725 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:09.915805  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59911.4us
I20260709 13:03:09.915992  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:09.917590  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.917742  6580 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:09.886986  6457 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.922241  6457 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.914489  6466 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.927249  6636 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:09.932627  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.933856  1649 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:09.936550  6544 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.937390  6544 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:09.919528  1646 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.910953  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.935549  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.943344  1649 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:09.932649  1645 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.866626  6413 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.946316  1643 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:09.943534  6084 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:03:09.954937  6566 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:09.953676  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 442141us
I20260709 13:03:09.944247  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:09.969144  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:09.970482  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:09.971998  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:09.982770  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:09.984170  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43799.6us
I20260709 13:03:09.985178  6580 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:09.986155  6664 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
W20260709 13:03:09.987668  6666 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:09.990993  1650 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:09.991451  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 721) took 1475 ms. Trace:
I20260709 13:03:09.993590  6534 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:09.995988  1571 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:09.998845  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:10.004527  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.957130  6574 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:03:10.007675  6580 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:10.056216  6580 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:09.956174  6566 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.061584  1646 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:10.062039  6442 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.014451  6289 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:10.013934  1642 rpcz_store.cc:276] 0709 13:03:08.515627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.515796 (+   169us) service_pool.cc:224] Handling call
0709 13:03:08.561446 (+ 45650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.402218 (+840772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.990702 (+588484us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b540005c3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:09.991174 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1902187392}
I20260709 13:03:10.023643  6541 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:09.985219  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:10.029537  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 499425us
I20260709 13:03:10.063482  6412 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.065280  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:10.068156  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.069358  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:10.070134  1570 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:10.071031  6084 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:10.045053  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.073089  1649 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:10.062819  6289 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.074409  6567 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.011s
I20260709 13:03:10.074688  6289 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:10.075933  6412 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:10.046744  1576 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:03:10.080119  1576 pending_rounds.cc:85] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 5
I20260709 13:03:10.080631  1576 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:03:10.080981  1576 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.082129  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2641) took 1873 ms. Trace:
I20260709 13:03:10.082479  1576 rpcz_store.cc:276] 0709 13:03:08.208382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.914644 (+1706262us) service_pool.cc:224] Handling call
0709 13:03:09.963519 (+ 48875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.081762 (+118243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.082019 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.050906  6642 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:10.063954  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.069945  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:10.071820  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.072140  6084 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:10.030319  6571 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:10.070962  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2643) took 1856 ms. Trace:
W20260709 13:03:10.084714  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 162789us
I20260709 13:03:10.065651  6325 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:10.069242  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:10.108938  6642 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:03:10.069551  6353 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.884s	user 0.006s	sys 0.015s
I20260709 13:03:10.120555  1570 rpcz_store.cc:276] 0709 13:03:08.214981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.069419 (+1854438us) service_pool.cc:224] Handling call
0709 13:03:10.069840 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.070628 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.070843 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:10.128312  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2575) took 3063 ms (client timeout 1804 ms). Trace:
W20260709 13:03:10.128899  1718 rpcz_store.cc:269] 0709 13:03:07.065156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.065313 (+   157us) service_pool.cc:224] Handling call
0709 13:03:10.127849 (+3062536us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b5400294fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:10.128046 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5075568384}
W20260709 13:03:10.134389  6548 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.009s
W20260709 13:03:10.135321  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.136260  6348 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.826s	user 0.013s	sys 0.012s
I20260709 13:03:10.107791  6609 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.136776  6348 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:10.140719  6593 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.628s	user 0.019s	sys 0.004s
I20260709 13:03:10.141607  6593 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 8. Current term is 9.
I20260709 13:03:10.144682  6550 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.145777  6550 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.147209  6286 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:10.148483  6286 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.151731  6568 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:10.183951  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 112285us
I20260709 13:03:10.157507  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2650) took 1823 ms. Trace:
I20260709 13:03:10.164783  1567 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:10.197042  1570 rpcz_store.cc:276] 0709 13:03:08.334750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.137246 (+1802496us) service_pool.cc:224] Handling call
0709 13:03:10.138035 (+   789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.138695 (+   660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.156954 (+ 18259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.157033 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:10.157162 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.157247 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.157389 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.160543  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 659) took 1608 ms. Trace:
I20260709 13:03:10.151866  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:10.158245  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:10.136579  1569 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:03:10.157573  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88659.9us
I20260709 13:03:10.166810  1642 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:10.144690  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 159775us
W20260709 13:03:10.158506  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88696.9us
I20260709 13:03:10.118250  6084 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.152149  6286 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:10.148197  6509 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:10.204268  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.174700  6348 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.153481  6578 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:10.205542  6578 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:10.207023  6578 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.157644  6649 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:10.208873  6610 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:10.209909  1569 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:10.210785  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2177) took 1312 ms. Trace:
I20260709 13:03:10.211242  1569 rpcz_store.cc:276] 0709 13:03:08.898874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.208968 (+1310094us) service_pool.cc:224] Handling call
0709 13:03:10.209570 (+   602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.210461 (+   891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.210710 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.156378  6544 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.135526  1648 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:10.198736  1723 rpcz_store.cc:276] 0709 13:03:08.552390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.553139 (+   749us) service_pool.cc:224] Handling call
0709 13:03:08.734057 (+180918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.834917 (+100860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.140208 (+1305291us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540010dbd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:10.160274 (+ 20066us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414866304}
I20260709 13:03:10.155941  6533 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.214654  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:10.215121  6642 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.216849  6642 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.216912  6355 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:10.217657  6355 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:10.219077  6355 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.220889  6578 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.221997  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:10.224843  6467 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.176393  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:10.120515  6571 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:10.169611  6400 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.785s	user 0.009s	sys 0.012s
I20260709 13:03:10.244306  1645 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
W20260709 13:03:10.164165  6332 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.267s	user 0.010s	sys 0.008s
I20260709 13:03:10.155598  6482 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.245337  6571 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.156785  6567 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.095s
W20260709 13:03:10.270282  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 118765us
I20260709 13:03:10.169348  6595 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:10.280429  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.200794  6353 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.257566  6649 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:10.268721  6533 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:10.288728  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2183) took 1338 ms. Trace:
I20260709 13:03:10.298439  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:10.305867  1571 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:10.299652  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:10.300638  1569 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:10.302327  2371 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:10.254745  6467 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.305907  1567 rpcz_store.cc:276] 0709 13:03:08.950123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.253454 (+1303331us) service_pool.cc:224] Handling call
0709 13:03:10.274848 (+ 21394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.288079 (+ 13231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.288122 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.288218 (+    96us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:10.288280 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.288386 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.288602 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.317667  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2178) took 1409 ms. Trace:
I20260709 13:03:10.306896  1721 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:10.285518  6544 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:10.320372  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:10.322989  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2651) took 1986 ms. Trace:
I20260709 13:03:10.342043  1570 rpcz_store.cc:276] 0709 13:03:08.336209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.198497 (+1862288us) service_pool.cc:224] Handling call
0709 13:03:10.263130 (+ 64633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.319105 (+ 55975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.319151 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.322611 (+  3460us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:10.322683 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.322770 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.322916 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.266736  6332 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.335623  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:10.339071  1569 rpcz_store.cc:276] 0709 13:03:08.899142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.232387 (+1333245us) service_pool.cc:224] Handling call
0709 13:03:10.281625 (+ 49238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.307705 (+ 26080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.308001 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.343093  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:03:10.358270  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.355770  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73670.3us
I20260709 13:03:10.341578  6544 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.369421  6325 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:10.369865  1719 raft_consensus.cc:2368] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:10.370124  6325 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.370970  6325 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.371769  1570 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:10.372247  1570 pending_rounds.cc:85] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:03:10.372581  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2644) took 2105 ms. Trace:
I20260709 13:03:10.373142  1572 rpcz_store.cc:276] 0709 13:03:08.259218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.081233 (+1822015us) service_pool.cc:224] Handling call
0709 13:03:10.081897 (+   664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.082631 (+   734us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:10.083269 (+   638us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.362920 (+279651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.362990 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:10.363055 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.363144 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.364448 (+  1304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16549,"prepare.queue_time_us":322,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":17097,"thread_start_us":262399,"threads_started":1}
I20260709 13:03:10.372591  1570 pending_rounds.cc:107] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:10.379057  1570 raft_consensus.cc:2889] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.379904  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2659) took 1143 ms. Trace:
I20260709 13:03:10.380226  1570 rpcz_store.cc:276] 0709 13:03:09.236673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.354661 (+1117988us) service_pool.cc:224] Handling call
0709 13:03:10.370988 (+ 16327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.379661 (+  8673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.379855 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.383405  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2642) took 2171 ms. Trace:
I20260709 13:03:10.383944  1575 rpcz_store.cc:276] 0709 13:03:08.211902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.986315 (+1774413us) service_pool.cc:224] Handling call
0709 13:03:09.987195 (+   880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.988056 (+   861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.988095 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:10.041169 (+ 53074us) log.cc:844] Serialized 26 byte log entry
0709 13:03:10.326146 (+284977us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:10.326204 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.326294 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.383017 (+ 56723us) raft_consensus.cc:1685] finished
0709 13:03:10.383040 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.383234 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9306,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":189,"thread_start_us":277579,"threads_started":1,"wal-append.queue_time_us":333338}
W20260709 13:03:10.387821  6670 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.255s	user 0.000s	sys 0.003s
I20260709 13:03:10.389204  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2655) took 1249 ms. Trace:
I20260709 13:03:10.389755  1567 rpcz_store.cc:276] 0709 13:03:09.140203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.338566 (+1198363us) service_pool.cc:224] Handling call
0709 13:03:10.355162 (+ 16596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.388723 (+ 33561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.388755 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.388845 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:10.388886 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.388957 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.389121 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:10.340754  6534 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:10.402515  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:10.409440  6675 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:03:10.412540  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.413460  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.416404  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:10.417567  6332 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.417858  6084 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.418294  6534 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.421154  6534 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.422322  1569 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:03:10.422766  1569 pending_rounds.cc:85] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:03:10.423030  1569 pending_rounds.cc:107] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:10.423311  1569 raft_consensus.cc:2889] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:10.423413  6669 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:03:10.366735  6457 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:10.370201  6595 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.392500  6530 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:03:10.425974  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2373) took 1364 ms. Trace:
I20260709 13:03:10.426527  1719 rpcz_store.cc:276] 0709 13:03:09.061561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.061783 (+   222us) service_pool.cc:224] Handling call
0709 13:03:10.425569 (+1363786us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400087354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:10.425855 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2718604672}
W20260709 13:03:10.429332  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 1899) took 2610 ms (client timeout 2676 ms). Trace:
W20260709 13:03:10.429852  1648 rpcz_store.cc:269] 0709 13:03:07.818660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.949037 (+130377us) service_pool.cc:224] Handling call
0709 13:03:10.429065 (+2480028us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b54001ba8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:10.429223 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4979896064}
I20260709 13:03:10.420903  6084 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.434558  6353 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.368206  6544 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:10.436225  1574 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:03:10.392779  6467 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.451344  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:10.431684  6571 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.454123  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2652) took 1306 ms. Trace:
W20260709 13:03:10.454887  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.454818  1574 rpcz_store.cc:276] 0709 13:03:09.132687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.311536 (+178849us) service_pool.cc:224] Handling call
0709 13:03:10.416023 (+1104487us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:10.438602 (+ 22579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411712}
I20260709 13:03:10.426429  6325 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:10.477890  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.526s (SENT)
I20260709 13:03:10.476146  6642 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:10.487623  6325 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.489856  6396 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.490142  6581 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:10.490792  6396 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.494428  6396 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.495927  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2647) took 2223 ms. Trace:
W20260709 13:03:10.500675  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 173977us
I20260709 13:03:10.511581  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:10.491526  6581 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.517002  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:10.528985  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.529625  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.529780  1644 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:10.530046  1648 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:10.530401  1647 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:10.525858  6533 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:10.533008  6533 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.540870  6662 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:10.542310  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53535.4us
I20260709 13:03:10.535979  6533 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.543648  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.510372  6595 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.479538  6671 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
W20260709 13:03:10.549391  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75816.3us
W20260709 13:03:10.553954  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 108817us
I20260709 13:03:10.498832  1576 rpcz_store.cc:276] 0709 13:03:08.272144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.107796 (+1835652us) service_pool.cc:224] Handling call
0709 13:03:10.152956 (+ 45160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.153393 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:10.153851 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.354329 (+200478us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.494904 (+140575us) spinlock_profiling.cc:243] Waited 918 us on lock 0x7b180032d5e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:10.495156 (+   252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.495223 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:10.495333 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.495433 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.495761 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139840,"prepare.queue_time_us":2372,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":1884,"raft.queue_time_us":155289,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":6327,"spinlock_wait_cycles":2110720,"thread_start_us":183139,"threads_started":1,"wal-append.queue_time_us":285843}
I20260709 13:03:10.541790  6662 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.562143  1614 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:10.556613  6595 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.566838  6662 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.597543  6624 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:10.599817  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:10.568934  6596 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:10.602864  6680 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:03:10.605134  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2185) took 1654 ms. Trace:
W20260709 13:03:10.622128  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 544) took 1426 ms (client timeout 1526 ms). Trace:
I20260709 13:03:10.601636  6566 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:10.616163  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.645118  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:10.619305  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:10.624358  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 70736.5us
I20260709 13:03:10.639385  1571 rpcz_store.cc:276] 0709 13:03:08.950805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.313765 (+1362960us) service_pool.cc:224] Handling call
0709 13:03:10.338105 (+ 24340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.455090 (+116985us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:10.465608 (+ 10518us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.601226 (+135618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.601307 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:10.601403 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.601492 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.605023 (+  3531us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2858,"prepare.run_cpu_time_us":172,"prepare.run_wall_time_us":11572,"raft.queue_time_us":10356,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"thread_start_us":86387,"threads_started":1}
I20260709 13:03:10.629757  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:10.629989  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:10.631618  6674 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.472s	user 0.002s	sys 0.000s
I20260709 13:03:10.632899  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:10.633625  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:10.640147  1573 rpcz_store.cc:269] 0709 13:03:09.195551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.222337 (+ 26786us) service_pool.cc:224] Handling call
0709 13:03:10.603384 (+1381047us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54000c1f54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:10.622028 (+ 18644us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273616640}
I20260709 13:03:10.566022  6289 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.661258  1647 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:10.680382  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:10.690503  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.694684  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:10.694897  6447 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:03:10.698189  6447 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:10.698794  6447 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:10.700960  6396 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:10.702597  6396 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:10.696733  1719 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 9: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:10.705917  6595 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.706050  6546 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:10.706645  6546 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:10.713773  6672 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.574s	user 0.003s	sys 0.002s
I20260709 13:03:10.715128  6571 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.719955  6620 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.721299  6620 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.708207  6546 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.723608  6546 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:10.719967  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 131077us
I20260709 13:03:10.731452  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2639) took 2557 ms. Trace:
I20260709 13:03:10.733286  6550 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.722103  6642 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:10.742357  1647 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:10.753935  6568 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.000s
I20260709 13:03:10.753640  6289 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:10.754840  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.760697  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.765573  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.737151  6457 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:10.768364  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:10.767328  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.769922  6550 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:10.774544  6550 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.732442  1568 rpcz_store.cc:276] 0709 13:03:08.174620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.823025 (+1648405us) service_pool.cc:224] Handling call
0709 13:03:09.824032 (+  1007us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:09.977670 (+153638us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:09.989494 (+ 11824us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.190238 (+200744us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:10.202527 (+ 12289us) log.cc:844] Serialized 52 byte log entry
0709 13:03:10.202745 (+   218us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:10.203196 (+   451us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.226653 (+ 23457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.226769 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.630510 (+403741us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:10.730948 (+100438us) raft_consensus.cc:1685] finished
0709 13:03:10.730979 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.731324 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156808,"prepare.queue_time_us":4397,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":100,"raft.queue_time_us":80090,"raft.run_cpu_time_us":565,"raft.run_wall_time_us":43361,"spinlock_wait_cycles":83817344,"thread_start_us":127862,"threads_started":1,"wal-append.queue_time_us":168903}
I20260709 13:03:10.779240  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:10.779996  2375 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:10.784513  6457 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.784878  6580 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.785425  1648 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:10.785871  6580 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.789932  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:10.790416  6544 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.791149  2373 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:03:10.792944  6550 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.793962  6662 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.796444  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.797747  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.799543  1576 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:10.797749  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.804675  1573 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:10.807966  1718 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:10.809687  6581 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.812304  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:10.813359  1574 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:10.817384  2375 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:10.819329  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:10.749580  6620 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:10.727787  6447 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.842890  1647 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:10.859124  1569 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:10.871628  1567 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:10.857432  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.884555  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:10.888552  2374 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:10.890842  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:10.879443  6482 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.570s	user 0.015s	sys 0.000s
W20260709 13:03:10.935974  6692 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.092s	user 0.004s	sys 0.000s
I20260709 13:03:10.900699  2380 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:10.909196  1647 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:10.912427  6595 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:10.917620  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.922953  1643 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:10.925447  6647 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:10.927716  6571 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:10.932482  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.938385  6580 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.935215  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.893079  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:10.943979  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 294546us
I20260709 13:03:10.893847  6662 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:10.941041  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:10.948249  1644 raft_consensus.cc:2376] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12. Re-sending same reply.
I20260709 13:03:10.948519  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:10.949330  1718 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:10.949226  6533 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.954874  1568 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:10.985402  1574 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:10.986837  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1564828209999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:03:10.988763  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.989054  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:10.990001  1646 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:10.990855  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.993767  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:10.995270  1570 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:10.997476  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.997880  6533 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:10.998296  1647 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:03:10.998799  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:10.963900  6595 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:10.999466  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:10.999961  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.000622  1642 raft_consensus.cc:2376] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14. Re-sending same reply.
I20260709 13:03:11.001127  1575 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:11.001353  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.002206  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.002483  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.002781  1575 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.003103  1572 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.004240  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:10.962812  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:10.993567  6289 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:10.969933  6647 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:11.006247  6289 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.006989  6647 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:10.981782  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:10.989245  6662 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:11.009770  1576 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.980068  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.005512  1716 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:10.988862  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:11.013278  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.978322  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.016242  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.018468  6621 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.019229  6621 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.020669  1688 leader_election.cc:400] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:11.023274  6582 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:10.996291  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.030352  1570 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.024228  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:11.028774  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.032672  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:11.028312  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.027617  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:11.018663  1539 leader_election.cc:400] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:11.039938  1574 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:11.022475  6621 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.048127  1720 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:11.045853  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.000219  6533 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.024528  6659 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:11.066886  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.053562  1567 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:11.055634  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.024238  6582 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:11.072542  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:11.077474  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.076313  1569 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:11.079259  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.045254  6580 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:11.049001  6482 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.115316  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:11.116472  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89050.8us
I20260709 13:03:11.117705  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:11.083643  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.062069  6447 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.056186  6647 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.124907  1567 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:11.125602  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 101462us
I20260709 13:03:11.131937  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.133375  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:11.025112  6355 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:03:11.140285  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.081728  6353 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.134948  1719 raft_consensus.cc:2376] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15. Re-sending same reply.
W20260709 13:03:11.143291  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 124939us
I20260709 13:03:11.082319  6580 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.144783  1642 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:11.139680  6621 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.120292  6482 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.154994  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:11.155967  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.156283  1569 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.161547  1720 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:11.142303  1574 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 9.
I20260709 13:03:11.163753  1574 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:11.155303  6482 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.185549  1642 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:11.186225  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:11.187384  1642 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:11.187433  1644 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:11.188855  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:11.189220  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:11.189971  1642 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:11.190016  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:11.190862  1646 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:11.193190  1646 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:11.193796  6571 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.194509  6571 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.198592  6571 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.203670  1569 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:11.025216  6595 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:11.225277  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 107706us
W20260709 13:03:11.236562  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.237475  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.244092  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.082245  6662 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.148180  6544 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.213263  6580 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.263916  1720 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:11.118602  6582 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.269852  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:11.279811  6621 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:11.232931  6620 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:11.294075  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:11.298329  1723 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:11.303161  1719 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:11.074293  6659 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:11.274670  6595 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.313099  1576 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:11.314895  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.070972  6289 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:11.317227  1575 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:11.330128  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:11.326783  6659 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.345186  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.346549  1569 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:11.372139  1572 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:11.376777  1574 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:11.364202  1650 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:11.381575  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.315871  6235 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:11.374081  1570 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:11.389271  6659 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:11.392916  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 130862us
I20260709 13:03:11.397156  6647 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.459916  1570 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:11.459746  6647 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.460827  6659 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:11.400638  1717 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:11.401448  6544 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.414575  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.430627  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:11.438951  1649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.035s	sys 0.062s
W20260709 13:03:11.479218  1649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.035s	sys 0.062s
I20260709 13:03:11.445876  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.435518  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.465807  1723 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:11.481251  1575 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:11.485389  1721 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:11.486070  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 166284us
I20260709 13:03:11.378855  6621 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.490240  1569 raft_consensus.cc:2410] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:03:11.408305  6662 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:11.505439  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.481703  6546 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.006s
W20260709 13:03:11.505806  6697 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.002s
I20260709 13:03:11.329545  6620 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.523602  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:11.527626  1645 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
W20260709 13:03:11.535272  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49588.6us
I20260709 13:03:11.519495  6571 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.547153  1573 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
W20260709 13:03:11.560659  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 418384us
I20260709 13:03:11.563798  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:11.570386  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.506093  6621 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.582994  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:11.585542  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.579126  1572 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.525125  6544 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.551468  6647 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:11.598836  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 444010us
I20260709 13:03:11.579547  6624 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.612775  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89339.7us
I20260709 13:03:11.606169  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:11.576180  6396 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:11.619987  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:11.588557  6582 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.552006   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:11.579165  6662 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:11.569319  6571 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.602308  6546 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.591398  6235 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:03:11.647022  1572 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:11.638705  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:11.628271  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:11.660871  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.673004  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.637538  6659 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:11.624572  6286 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:03:11.629599  6355 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:03:11.658167  6482 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:11.714833  6482 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.717376  6354 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:11.717916  6354 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:11.718861  6354 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.718981  6482 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:11.721063  6694 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.604s	user 0.000s	sys 0.005s
I20260709 13:03:11.721252  6355 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:11.688879  6457 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.903s	user 0.018s	sys 0.002s
I20260709 13:03:11.713841  6544 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.622108  6624 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.694787  6705 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:11.685124  6659 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:11.700251  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51344.2us
I20260709 13:03:11.748868  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:11.686216  6647 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:03:11.725833  6355 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:11.744899  6659 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:11.742556  6705 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:11.777760  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.780716  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.782588  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:11.731640  6457 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:11.786324  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 108716us
I20260709 13:03:11.793216  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:11.790419  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 168028us
W20260709 13:03:11.798211  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.775766  6706 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:11.782043  6286 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.805860  6457 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:11.807899  6647 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.819991  6582 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:11.820444  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.820642  6582 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:11.831037  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 198538us
I20260709 13:03:11.810128  6706 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:11.835948  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:11.812083  6621 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.834580  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:11.842124  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 219531us
I20260709 13:03:11.843154  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:11.814414  6482 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:03:11.840957  6595 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.001s
I20260709 13:03:11.843919  6286 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.849049  6714 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.852478  6571 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.853008  6706 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.854884  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:11.870633  6704 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:11.840673  6705 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.875695  6705 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.867277  6713 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:11.876837  6713 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:11.877188  6713 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.865890  6715 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:11.878893  6715 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.869095  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:11.882495  6715 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:11.885043  6595 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:11.885823  6595 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:11.885912  6582 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.887627  6582 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:11.887260  6595 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:11.888773  6709 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:03:11.891630  6700 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.209s	user 0.000s	sys 0.002s
W20260709 13:03:11.892970  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53004.2us
I20260709 13:03:11.895049  6706 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:11.844169  6647 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:11.904393  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 93386.6us
I20260709 13:03:11.905967  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:11.865527  6355 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:11.910126  6457 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:11.920568  6647 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.902067  6714 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.896907  6353 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:11.901607  6715 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.870649  6707 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:03:11.950328  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:11.957430  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:11.964537  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 145140us
W20260709 13:03:11.938928  6447 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.800s	user 0.013s	sys 0.002s
I20260709 13:03:11.977027  1723 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:03:11.965662  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 172655us
W20260709 13:03:11.969940  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100955us
I20260709 13:03:11.972669  6716 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:11.973490  6289 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:11.974175  6457 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.974198  6499 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:11.975029  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64136us
I20260709 13:03:11.979573  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:11.982815  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:11.984561  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 202071us
I20260709 13:03:11.966799  6624 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:11.959209  6353 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:11.943152  6715 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:11.998416  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:12.010877  6724 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:12.011811  6719 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:12.022843  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.891s (SENT)
I20260709 13:03:12.029707  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:12.033294  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:12.034663  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:12.035907  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57019.5us
I20260709 13:03:12.042527  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:12.041944  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:12.045269  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:12.045404  1723 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:03:12.000371  6716 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:12.035781  6595 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.052358  6718 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.053781  6718 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:12.054625  6714 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:12.054411  6718 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.054802  6705 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:12.055521  6705 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.056586  6546 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.056841  6353 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.057652  6717 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.043497  6719 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:12.058323  6717 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.059504  6719 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.061184  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:12.000377  6289 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.043322  6235 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:12.078289  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 77487.9us
I20260709 13:03:12.079366  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:12.064109  6620 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:12.054652  6706 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.081676  6706 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.083916  6706 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.085057  6705 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.030867  6354 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.086139  6717 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.086912  6706 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:12.087577  6706 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:12.047302  6396 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:12.067524  6595 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:12.088696  6706 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.050544  6457 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.092852  1717 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:12.093462  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.030323  6499 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:12.066797  6580 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.006s
I20260709 13:03:12.087975  6354 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:12.112119  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:12.112941  6721 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:12.113935  6721 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:12.113121  6620 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.115212  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:12.115123  6721 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.106909  6235 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:12.117911  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49628.8us
I20260709 13:03:12.127772  6716 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.129284  6659 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.130487  6235 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.131587  6659 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.132513  6620 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:12.134008  6289 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:12.134464  6647 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.134959  6289 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.144332  1567 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:12.145766  6499 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:12.146337  1571 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:12.148124  6715 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.150135  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:12.150039  6713 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.151265  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:12.153020  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:12.152333  6713 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:12.154137  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:12.154839  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.155462  1571 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:12.155961  6286 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:03:12.139660  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 116871us
I20260709 13:03:12.157506  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:12.157783  6624 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:12.159184  6624 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:12.161836  1567 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:03:12.165668  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 93044.6us
I20260709 13:03:12.171828  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:12.172243  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 124986us
I20260709 13:03:12.183526  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:12.184268  1572 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:03:12.184751  1572 pending_rounds.cc:85] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:03:12.184999  1572 pending_rounds.cc:107] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:12.185245  1572 raft_consensus.cc:2889] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:12.110085  6580 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.188910  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.188905  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.174857  6544 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:12.190824  1576 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:12.193583  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:12.194542  1645 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.169245  6457 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:12.196808  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:12.197845  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.197731  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.167081  6715 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:12.200253  1568 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:03:12.180546  6620 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.199429  1573 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:12.200724  1568 pending_rounds.cc:85] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:03:12.205395  1568 pending_rounds.cc:107] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:12.205883  1568 raft_consensus.cc:2889] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:12.199469  1648 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:12.174579  6595 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:12.182776  6621 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:12.207245  1569 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:12.207762  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:12.214480  1642 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:12.215124  1651 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.207391  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.211829  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:12.208527  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:12.215843  1647 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:12.165359  6289 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.213164  1650 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:12.216691  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:12.213789  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:12.217945  1649 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.210858  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.204864  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:12.219378  1643 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.193471  6544 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.221835  1571 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:12.226449  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.089s (SENT)
I20260709 13:03:12.227177  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:12.236851  6719 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:12.237671  6289 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.243817  1645 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:12.200724  6286 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.220549  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:12.249780  1646 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.199100  6457 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:12.257953  6720 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.261240  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:12.261508  2371 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:12.262537  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:12.264449  2373 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:12.264508  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:12.267277  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:12.272138  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56817.5us
I20260709 13:03:12.275825  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:12.273452  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:12.276752  1572 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:12.214133  6621 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:12.283812  2380 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:12.220485  6595 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:12.246467  6730 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.273329  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:12.296168  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:12.239161  6544 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:12.299791  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81761.4us
I20260709 13:03:12.274072  6457 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.317631  1649 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
W20260709 13:03:12.322337  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49107.6us
I20260709 13:03:12.271413  6714 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:12.331051  6714 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.332521  6499 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:12.274421  6719 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.332226  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.332841  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:12.347205  6719 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.350078  1649 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:12.238545  6566 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:03:12.339984  6714 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:12.220575  6715 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:12.396083  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65523.2us
I20260709 13:03:12.427982  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:12.432472  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73364.3us
I20260709 13:03:12.428978  1647 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:12.352625  6713 raft_consensus.cc:2764] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:12.289948  6354 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:03:12.468771  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:12.329739  6289 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:12.442451  6735 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:03:12.475574  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:12.494625  2379 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:12.496968  6729 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:03:12.490031  6706 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:03:12.497367  6717 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:12.498086  6740 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.005s
I20260709 13:03:12.390309  6647 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:12.349216  6621 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.507249  1572 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:12.289635  6720 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.523263  6647 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.525501  6355 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:12.529886  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:12.591148  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:12.592321  1572 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:12.531724  6734 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:03:12.532598  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102854us
W20260709 13:03:12.532954  6582 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.010s
I20260709 13:03:12.527976  6715 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:12.554287  1651 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
W20260709 13:03:12.562752  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 170395us
I20260709 13:03:12.587018  1646 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:12.588472  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 393860us
I20260709 13:03:12.594769  1717 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:12.601868  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 403751us
I20260709 13:03:12.621361  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:12.518364  6713 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:12.621554  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:12.321645  6730 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:12.627851  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.477s (SENT)
I20260709 13:03:12.523938  6738 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:12.635241  1574 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:12.596318  6647 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.641059  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:12.636976  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:12.700613  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63757.7us
W20260709 13:03:12.699137  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58239.6us
I20260709 13:03:12.586453  6705 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 13:03:12.709358  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68387.1us
I20260709 13:03:12.718569  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:12.626613  6713 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:12.637925  6738 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.640049  6716 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.698585  6289 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:12.674099  6621 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.672435  6396 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:12.705084  6715 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.674355  6752 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:12.524192  6595 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:12.793268  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 172006us
I20260709 13:03:12.812435  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:12.789381  6739 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 13:03:12.788800  6741 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 13:03:12.778090  6738 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.716369  6580 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:12.755414  6713 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:12.788015  6743 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 13:03:12.782092  6736 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:12.793443  1643 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.000s
I20260709 13:03:12.837342  1643 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:12.715565  6325 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:12.834976  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:12.698980  6582 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.131s
W20260709 13:03:12.840701  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 212902us
I20260709 13:03:12.804363  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:12.851778  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.800460  6716 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.810207  6286 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:03:12.792394  6732 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
W20260709 13:03:12.884897  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 171756us
I20260709 13:03:12.840260  6714 raft_consensus.cc:2764] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:12.920789  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:12.842681  6325 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.840579  6571 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.841805  6736 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:12.845119  6289 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:12.850203  6396 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.921567  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:12.814260  6752 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:12.996598  6742 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.998405  6745 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.012712  6718 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:13.013294  6744 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 13:03:13.013671  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.014679  2371 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:13.016150  6749 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 13:03:13.027661  6755 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.425s	user 0.000s	sys 0.002s
W20260709 13:03:12.955472  6743 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:03:12.853111  6624 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:03:12.798272  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:13.032617  6719 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:12.830472  6715 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:12.925961  6662 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.102005  1642 raft_consensus.cc:606] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:13.103093  1642 raft_consensus.cc:614] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:13.043427  6736 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:13.049988  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 128555us
I20260709 13:03:13.049610  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:13.051687  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 167557us
I20260709 13:03:13.041579  6325 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:13.104152  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 51487.7us
W20260709 13:03:13.038594  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 203713us
I20260709 13:03:13.128472  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:13.130683  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 679) took 1139 ms. Trace:
I20260709 13:03:13.130942  6546 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:13.057859  6759 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:13.131557  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.131628  6546 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.132483  6546 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:13.132787  6499 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:03:13.093559  6756 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:03:13.131204  1721 rpcz_store.cc:276] 0709 13:03:11.990860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.991075 (+   215us) service_pool.cc:224] Handling call
0709 13:03:12.010428 (+ 19353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.160613 (+150185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.160652 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.168647 (+  7995us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.661314 (+492667us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:12.661361 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.130244 (+468883us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003dc7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:13.130300 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.130363 (+    63us) raft_consensus.cc:1685] finished
0709 13:03:13.130377 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.130622 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38434,"raft.queue_time_us":23786,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9506,"spinlock_wait_cycles":265703680,"thread_start_us":422596,"threads_started":1,"wal-append.queue_time_us":640165}
I20260709 13:03:13.045063  6571 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.147675  6571 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.105381  6745 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.148873  6745 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.042407  6647 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:13.170469  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.171891  1567 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:13.176477  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:13.179838  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:13.180603  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:13.183881  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:13.199057  1643 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:13.170126  2373 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:13.205518  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.206432  1724 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:13.202286  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.206507  1719 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:13.187952  2379 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.005s
I20260709 13:03:13.099489  6355 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:13.060026  6714 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:13.201206  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.207423  1718 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:13.214694  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:13.194957  1645 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:13.197980  1648 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.208662  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:13.062423  6720 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.214989  1723 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:03:13.289942  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.063s (SENT)
I20260709 13:03:13.290555  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:13.215660  1649 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:13.208317  2379 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:13.217547  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.219097  1573 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:13.219141  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2688) took 1032 ms. Trace:
I20260709 13:03:13.302006  1576 rpcz_store.cc:276] 0709 13:03:12.186930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.294688 (+107758us) service_pool.cc:224] Handling call
0709 13:03:12.352206 (+ 57518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.390696 (+ 38490us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:12.390739 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.437678 (+ 46939us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.599045 (+161367us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:12.599097 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.599194 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.218778 (+619584us) raft_consensus.cc:1685] finished
0709 13:03:13.218814 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.219040 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":389,"raft.queue_time_us":44169,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":15592,"thread_start_us":87656,"threads_started":1,"wal-append.queue_time_us":316548}
I20260709 13:03:13.220466  1725 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:13.199558  6763 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
I20260709 13:03:13.191100  6482 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.055790  6742 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.209578  6396 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:03:13.222272  1724 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:13.224170  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:13.310144  1719 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:13.227653  6544 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.800s	user 0.006s	sys 0.005s
I20260709 13:03:13.233558  6716 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:13.233660  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.237552  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.282063  6546 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.239997  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.242038  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.155396  6289 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.244441  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:13.245167  6750 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.644s	user 0.002s	sys 0.001s
I20260709 13:03:13.250182  1648 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
W20260709 13:03:13.337190  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 92875.8us
W20260709 13:03:13.254729  6457 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.864s	user 0.012s	sys 0.000s
I20260709 13:03:13.261708  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.339036  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:13.265841  6715 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.310257  6730 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:13.279181  6752 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.198549  6759 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:13.211519  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:13.238583  6745 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:13.284164  6757 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:03:13.319373  1721 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:13.311900  6482 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.340042  1648 raft_consensus.cc:2410] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:03:13.341238  6767 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:03:13.345161  6738 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.213477  6355 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:13.360441  1717 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:13.313560  6721 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.359634  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:13.412515  1576 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:03:13.319515  6544 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.364611  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2217) took 1200 ms. Trace:
W20260709 13:03:13.366844  6768 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.165s	user 0.000s	sys 0.003s
I20260709 13:03:13.367646  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:13.368705  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78813.5us
I20260709 13:03:13.369547  6742 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.371910  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:13.372807  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 159262us
W20260709 13:03:13.376425  6620 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.169s	user 0.009s	sys 0.006s
I20260709 13:03:13.322467  6762 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:13.376940  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 251779us
I20260709 13:03:13.381449  6720 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:13.381991  6647 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.382618  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:13.397265  6752 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:13.409368  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:13.406021  1567 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:13.408860  6745 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:13.462136  6745 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.465332  6745 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.419561  1723 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:13.387861  6714 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:13.359961  6457 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:13.424017  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 215358us
I20260709 13:03:13.433061  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:13.435111  1719 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:13.415122  1570 rpcz_store.cc:276] 0709 13:03:12.164069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.164205 (+   136us) service_pool.cc:224] Handling call
0709 13:03:12.174964 (+ 10759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.190843 (+ 15879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.190879 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.203834 (+ 12955us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.495688 (+291854us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:12.495732 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.495815 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.363266 (+867451us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b240013cef8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830b079c4 00007fc830b05acb 00007fc830cc0dc9 00007fc830cc0a55 00007fc830cbffb0 00007fc833936488 00007fc833936be8 00007fc83390e398 00007fc83390976a 00007fc833908bb6
0709 13:03:13.364154 (+   888us) raft_consensus.cc:1685] finished
0709 13:03:13.364196 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.364509 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6491,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":9813,"spinlock_wait_cycles":38419712,"thread_start_us":211419,"threads_started":1,"wal-append.queue_time_us":373533}
I20260709 13:03:13.443994  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:13.436486  6720 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.444237  6759 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.489095  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:13.446878  6713 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.004s
W20260709 13:03:13.450434  6621 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.000s
I20260709 13:03:13.454217  6738 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:13.436532  6620 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.455705  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:13.505540  1643 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:13.458313  6765 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 13:03:13.394263  6566 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:03:13.485294  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 587) took 1201 ms. Trace:
I20260709 13:03:13.537868  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:13.503798  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:13.473699  6752 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.488271  6717 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:13.504500  6713 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.501080  6720 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:13.528301  6544 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.528996  6620 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.529139  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 100647us
I20260709 13:03:13.529655  1575 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:13.530263  6457 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.529896  6355 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.508710  6724 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.445287  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:13.538708  2379 rpcz_store.cc:276] 0709 13:03:12.265448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.266736 (+  1288us) service_pool.cc:224] Handling call
0709 13:03:13.466887 (+1200151us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54005fee54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:13.467022 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467948928}
I20260709 13:03:13.544775  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1938) took 1352 ms. Trace:
I20260709 13:03:13.546357  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:13.545238  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:13.560065  1643 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:13.561231  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:13.565310  1576 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:13.566859  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 121693us
I20260709 13:03:13.565701  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:13.546216  6738 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:13.584786  6546 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:13.625509  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 136532us
I20260709 13:03:13.626272  6772 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.626626  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:13.627007  6772 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:13.627537  6772 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.627602  1472 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 7 to 9, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:03:13.634016  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:13.634757  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.592635  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:13.631397  1472 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 6 to 7, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 7 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:13.647624  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:13.577283  6289 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.584950  6447 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.572844  6762 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.577852  6742 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:13.637558  6355 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:13.578629  6738 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.557794  1644 rpcz_store.cc:276] 0709 13:03:12.192039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.192216 (+   177us) service_pool.cc:224] Handling call
0709 13:03:12.211019 (+ 18803us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:13.132370 (+921351us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b1800481420. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d66bfb 00007fc833d7164b
0709 13:03:13.132504 (+   134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.132528 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:13.133142 (+   614us) log.cc:844] Serialized 52 byte log entry
0709 13:03:13.344790 (+211648us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:13.469100 (+124310us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.472981 (+  3881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.473157 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.538438 (+ 65281us) raft_consensus.cc:1685] finished
0709 13:03:13.544490 (+  6052us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.544691 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":163715,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":109645,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":54834,"spinlock_wait_cycles":54326656,"thread_start_us":452964,"threads_started":3,"wal-append.queue_time_us":281543}
I20260709 13:03:13.638532  6716 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:13.657701  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 155191us
I20260709 13:03:13.642545  6544 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.585549  6580 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:13.637032  6325 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.656069  6289 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.571578  6620 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.670889  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:13.673714  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81235.1us
I20260709 13:03:13.658912  6762 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.680915  1572 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:13.665593  6738 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:13.652958  6736 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.667512  6716 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:13.708390  6716 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.708766  6780 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:13.709370  6780 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:13.709843  6780 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:13.712687  6788 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.678706  6289 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.713469  6582 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:13.713946  6582 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:13.670146  6742 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:13.691080  6235 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.718739  6235 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.719945  6482 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:13.659161  6447 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.722070  6235 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:13.693821  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.672432  6544 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:13.704295  6770 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:13.705127  6713 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.682725  6396 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:03:13.682857  6580 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:13.746977  6235 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:13.747712  6580 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.749575  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:13.750418  1720 raft_consensus.cc:606] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:13.750841  1720 raft_consensus.cc:614] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:13.675316  6779 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.752977  6779 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.754865  6447 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:13.755474  6779 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.717435  6742 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:13.691622  6724 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.756491  6724 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.758472  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.759032  6724 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:13.721415  6783 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:13.721288  6647 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:13.585047  6714 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:13.734254  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46183.8us
I20260709 13:03:13.741758  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:13.775905  1721 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.743507  6620 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:13.762475  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:13.775218  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46776.1us
I20260709 13:03:13.686023  6762 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.784132  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:13.774158  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:13.784211  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:13.786856  1722 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:13.759275  6544 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.794554  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:13.835908  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.798317  6759 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.722066  6785 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:03:13.839083  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:13.810382  6620 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.820358  6546 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:13.816828  6715 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.843859  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:13.844588  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58124.5us
I20260709 13:03:13.850528  1576 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:13.872536  6624 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:13.877843  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:13.832926  6235 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:03:13.848794  6546 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.842804  6752 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:13.936497  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58736.5us
I20260709 13:03:13.843922  6620 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:13.939710  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55022.4us
I20260709 13:03:13.943991  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:13.939615  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:13.854068  6706 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:03:13.944741  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62608.2us
I20260709 13:03:13.951086  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:13.878724  6724 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:03:13.956987  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:13.939335  6546 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:13.943681  6730 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:13.962205  6796 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:03:13.962776  6325 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:13.946684  6620 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.887701  6624 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:13.980594  1721 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:13.959640  6752 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.983291  6544 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.974510  6595 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.959088  6662 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:13.997059  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 57553.6us
I20260709 13:03:13.997970  6736 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:13.998317  6582 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.001662  1573 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:14.004168  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:13.986864  6624 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.009999  6772 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.010610  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:14.030156  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45459.7us
I20260709 13:03:14.038286  1567 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:03:14.049088  6764 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:14.051447  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:14.051607  6738 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.052312  1573 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:14.057165  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:14.070662  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:14.072218  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:14.071897  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:14.074652  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2222) took 1902 ms (client timeout 2236 ms). Trace:
W20260709 13:03:14.075207  1575 rpcz_store.cc:269] 0709 13:03:12.172102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.172223 (+   121us) service_pool.cc:224] Handling call
0709 13:03:14.074437 (+1902214us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400309b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:14.074572 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3075098752}
I20260709 13:03:14.076009  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:14.020352  6662 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:14.073004  1576 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:14.084900  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:14.053683  6717 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.002s
I20260709 13:03:14.087553  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:14.091979  1570 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:14.093089  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.094090  1645 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:14.096796  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.097815  1648 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:14.096855  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:14.102277  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.104451  2373 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:14.105398  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:14.106331  1576 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.109980  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.111667  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.112685  1643 raft_consensus.cc:2376] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14. Re-sending same reply.
I20260709 13:03:14.116460  1646 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:14.117059  6738 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.118289  1647 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:14.118642  1645 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:14.053192  6706 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:03:14.120180  1650 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:14.053371  6759 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:03:14.077610  6732 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:14.052628  6499 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:03:14.122926  6772 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.066259  6780 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.124055  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:14.124802  2380 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.126075  6780 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.126892  1572 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:14.106307  6717 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:14.091252  6662 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.117090  2375 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:14.126921  1644 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:14.136085  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.110741  1642 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:14.142526  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.142589  1575 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.144090  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.143042  2374 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:03:14.150501  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 97466.1us
I20260709 13:03:14.151587  1570 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:14.047919  6716 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.159773  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.160642  2374 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:14.162369  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.163082  1651 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.162879  6762 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.043273  6289 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.181973  1575 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:14.144122  6621 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.185128  6621 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.193198  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.195293  2375 raft_consensus.cc:2368] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:14.197476  1570 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:14.198720  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:14.201014  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.203281  6717 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.203944  6717 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.206491  6717 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:14.207154  6717 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:14.207551  6717 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.207991  6717 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:14.209367  6720 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.011s
I20260709 13:03:14.210244  6720 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.220772  6730 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.221508  6730 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.223377  1649 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:14.223899  6730 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:14.224871  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.225549  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.226661  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.227340  1644 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.227648  6730 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:14.229511  1570 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:14.229955  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:14.231509  6730 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.179522  6546 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.245494  1539 leader_election.cc:400] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:14.246287  1539 leader_election.cc:403] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:14.237504  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:14.197180  1651 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:14.250663  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:14.254429  2380 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:14.281575  1576 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:14.282043  1644 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:14.249164  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:14.179977  6289 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:14.185014  6620 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.154922  6582 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:14.304771  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66654.6us
I20260709 13:03:14.296267  6736 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.307200  6736 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:14.310835  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 239105us
I20260709 13:03:14.318500  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:14.287817  6325 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.320008  2375 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.321120  6289 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.293665  6571 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:14.330060  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.332522  2379 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:14.299669  6732 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:14.337942  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:14.339639  6659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.020s	sys 0.010s
I20260709 13:03:14.344535  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:14.345057  6659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.020s	sys 0.010s
I20260709 13:03:14.345770  6659 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:14.306115  6803 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
W20260709 13:03:14.350473  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 99898.7us
I20260709 13:03:14.351792  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:14.325341  6724 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:14.319339  6736 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:14.353677  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:14.292750  6717 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.346246  6659 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.358089  6659 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:14.358621  6799 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:03:14.336084  6780 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:14.343276  6571 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:14.376915  6780 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:14.377467  6772 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:14.350427  6325 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:03:14.379364  6780 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:14.379849  6780 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:14.380555  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.380822  6325 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:14.381297  6325 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:14.381330  1646 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:14.380623  6806 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:03:14.382481  6325 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.383419  6720 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.384007  6325 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:14.401769  1570 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.384730  6732 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.386302  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.410218  1646 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:14.382895  6779 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:14.356247  6355 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:14.417008  6355 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.404630  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:14.406785  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.373155  6289 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:14.408761  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.420269  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.420703  1651 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.421182  1647 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:14.414289  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.420045  1646 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:14.421844  1650 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:14.400398  6624 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.417572  6355 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.400681  6286 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:14.360661  6724 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.429941  6289 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:14.455588  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 90237.5us
I20260709 13:03:14.435777  6620 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.460832  1647 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:14.436460  6647 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:03:14.441653  6286 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:14.466769  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.468526  6715 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:14.470855  1646 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:14.466820  1688 leader_election.cc:400] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:14.471715  1688 leader_election.cc:403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:14.472332  1651 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:14.475008  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:14.475095  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.477191  6730 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.459771  6571 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.439381  6720 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:03:14.486107  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:14.489113  1647 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.490932  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:14.496627  6717 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:14.492591  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.452896  6716 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:14.511168  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.440215  6624 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.516074  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.517689  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.521987  1645 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:14.525471  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:14.526479  2379 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.530069  1643 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:14.530244  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:14.486311  6546 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:14.485828  6580 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:14.537869  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.499297  6730 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.540783  1576 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:14.500157  6720 consensus_queue.cc:1243] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:14.539278  2380 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:03:14.470816  6499 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:03:14.521658  6717 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:14.545943  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.557888  2375 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:14.560480  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.560696  6289 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.569897  2379 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:03:14.570888  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.572691  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.555981  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.574705  1645 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:14.577456  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.586069  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:14.587837  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3744135439999923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 13:03:14.588482  1539 connection.cc:256] RPC call timeout handler was delayed by -2.2826135439999575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:03:14.588963  1539 connection.cc:256] RPC call timeout handler was delayed by -2.2808135439999546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:03:14.589380  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1350135439999463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:03:14.589800  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1017135439999493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
I20260709 13:03:14.590610  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:14.592635  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.594861  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.595769  1725 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:14.482197  6780 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.605191  1576 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:14.605686  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.609926  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.612282  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.613519  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.615448  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2243) took 1302 ms. Trace:
I20260709 13:03:14.615854  1572 rpcz_store.cc:276] 0709 13:03:13.313075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.734934 (+421859us) service_pool.cc:224] Handling call
0709 13:03:13.786075 (+ 51141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:14.127362 (+341287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.615150 (+487788us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54002b0054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:14.615359 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783375104}
I20260709 13:03:14.562577  6720 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:14.579011  6457 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.648313  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 103631us
I20260709 13:03:14.649556  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:14.652985  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 162179us
I20260709 13:03:14.601315  6396 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.609297  6482 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.601325  6717 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:14.658849  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:14.661108  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.664810  1645 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:14.674364  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 83860.6us
I20260709 13:03:14.678689  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:14.688956  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.692330  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.644590  6720 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.699487  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:14.705021  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.647279  6714 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:14.663410  6759 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:14.656975  6647 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.741832  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.714140  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.687119  6780 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.751386  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.754091  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.583461  6286 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.760005  6829 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:14.762477  1717 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:14.763414  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:14.774973  2377 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.765710  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.701762  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:14.776310  1718 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.769042  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.777125  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.656764  6457 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:14.781180  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 79529.6us
I20260709 13:03:14.777093  1645 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:14.686262  6396 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:14.702105  6720 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:14.701195  6620 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:14.794742  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:14.701748  6235 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:03:14.690276  6716 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:03:14.799386  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:14.804622  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.805665  1645 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.693892  6482 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:14.795956  6814 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:03:14.781625  6714 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.764007  6647 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:14.698021  6815 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
W20260709 13:03:14.846522  6662 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.718s	user 0.019s	sys 0.001s
I20260709 13:03:14.847513  6355 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:14.851480  6798 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.802s	user 0.002s	sys 0.000s
I20260709 13:03:14.910977  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2236) took 1732 ms. Trace:
I20260709 13:03:14.911999  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2255) took 1128 ms. Trace:
I20260709 13:03:14.911568  1569 rpcz_store.cc:276] 0709 13:03:13.178308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.459351 (+281043us) service_pool.cc:224] Handling call
0709 13:03:13.459745 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.638700 (+178955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.638744 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.644929 (+  6185us) log.cc:844] Serialized 26 byte log entry
0709 13:03:13.958121 (+313192us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:13.958164 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.958244 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.850439 (+892195us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:14.910549 (+ 60110us) raft_consensus.cc:1685] finished
0709 13:03:14.910578 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.910888 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19504,"raft.queue_time_us":61105,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":85,"spinlock_wait_cycles":12038912,"thread_start_us":426378,"threads_started":2,"wal-append.queue_time_us":377955}
I20260709 13:03:14.912956  1575 rpcz_store.cc:276] 0709 13:03:13.783964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.374886 (+590922us) service_pool.cc:224] Handling call
0709 13:03:14.375523 (+   637us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.498987 (+123464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.499032 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.499644 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 13:03:14.657277 (+157633us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:14.657344 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.657424 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.911689 (+254265us) raft_consensus.cc:1685] finished
0709 13:03:14.911724 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.911912 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6966,"raft.queue_time_us":106100,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":13756,"thread_start_us":116309,"threads_started":1,"wal-append.queue_time_us":281535}
I20260709 13:03:14.888753  6544 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.886343  6813 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.292s	user 0.000s	sys 0.002s
I20260709 13:03:14.854162  6396 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:14.857074  6789 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.165s	user 0.004s	sys 0.000s
I20260709 13:03:14.768154  6499 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:03:14.859481  6621 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.797051  6286 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.865991  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.938699  1724 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:14.886883  1721 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:14.860236  6457 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:14.793462  6829 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:14.853627  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:14.889073  6736 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.936983  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 142406us
I20260709 13:03:14.888302  6772 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:14.937948  1645 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:14.956804  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:14.961149  6827 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:14.967710  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:14.970920  1718 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:14.978906  1573 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:14.924531  6544 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:14.954380  6814 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:03:15.018046  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:15.025043  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.181s (SENT)
I20260709 13:03:15.025002  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.026433  1569 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:14.808467  6717 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:15.032476  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.035153  6824 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 13:03:15.038964  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 71375.2us
I20260709 13:03:15.087472  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:15.039956  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:14.926414  6662 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.065047  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.100004  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.101414  1645 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:14.932541  6580 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:15.088611  1642 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:14.963922  6714 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:15.093915  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68913.2us
I20260709 13:03:14.986061  6772 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:15.112711  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:15.089814  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:15.129650  1718 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:15.097689  6620 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:14.984747  6736 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.168848  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:15.173357  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2237) took 1978 ms. Trace:
I20260709 13:03:15.174083  1574 rpcz_store.cc:276] 0709 13:03:13.195190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.471688 (+276498us) service_pool.cc:224] Handling call
0709 13:03:13.508900 (+ 37212us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:13.509291 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.509323 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:13.539109 (+ 29786us) log.cc:844] Serialized 52 byte log entry
0709 13:03:13.675824 (+136715us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:13.675874 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.675955 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:14.856026 (+1180071us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:15.153521 (+297495us) raft_consensus.cc:1685] finished
0709 13:03:15.153552 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.173263 (+ 19711us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21944,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":80201088,"thread_start_us":104578,"threads_started":1,"wal-append.queue_time_us":143135}
I20260709 13:03:15.177999  6544 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.179363  6779 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:15.132124  6714 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.180197  6779 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:15.167055  1718 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:15.132820  6772 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:15.161165  6582 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.010s
I20260709 13:03:15.254354  1724 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:03:15.256783  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.257946  1645 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:15.261236  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.262492  1718 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
W20260709 13:03:15.263767  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.011s	sys 0.000s
W20260709 13:03:15.265194  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.011s	sys 0.000s
W20260709 13:03:15.265571  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 577644us
I20260709 13:03:15.267045  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:15.269726  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.271322  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.286645  6762 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:15.287192  6835 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:03:15.289914  6662 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:15.316877  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 227199us
W20260709 13:03:15.293375  6823 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
I20260709 13:03:15.296531  6732 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:15.298911  6833 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:03:15.297652  6580 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.182657  6624 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.332722  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.302157  6447 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:15.303458  6785 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:15.304949  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "740ac262ab184918992607d66446b8f3"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:15.339356  1645 raft_consensus.cc:606] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:15.307206  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:15.339870  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.340991  1569 raft_consensus.cc:2410] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:15.341320  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:15.266083  6824 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:03:15.311563  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.340991  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.344095  1718 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:15.313784  1642 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:15.345053  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:15.345515  1642 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:15.278937  6325 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.870s	user 0.012s	sys 0.004s
I20260709 13:03:15.254762  6544 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.322764  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.281716  6714 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:15.327342  1570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.016s	sys 0.041s
W20260709 13:03:15.358873  1570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.016s	sys 0.041s
W20260709 13:03:15.319458  6790 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.578s	user 0.001s	sys 0.004s
I20260709 13:03:15.355353  6355 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:15.363365  6714 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:15.363866  6714 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:15.364231  6714 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:15.329110  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.795s (SENT)
I20260709 13:03:15.365051  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.366513  6837 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:15.317428  6787 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.264s	user 0.008s	sys 0.030s
W20260709 13:03:15.367271  6787 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.009s	sys 0.030s
W20260709 13:03:15.323843  6713 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.016s	sys 0.029s
I20260709 13:03:15.348445  1643 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:15.348644  1572 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:15.350109  1650 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
I20260709 13:03:15.372359  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:15.351365  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.356293  1724 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:15.354676  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.373576  1568 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:15.358930  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.359412  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:15.361456  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54369.4us
I20260709 13:03:15.169075  6829 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.313611  6716 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:03:15.377537  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.380271  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:15.369948  2377 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.002s
W20260709 13:03:15.376106  6713 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.017s	sys 0.029s
I20260709 13:03:15.379081  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:15.340081  6447 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:15.393843  1572 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:15.338896  6785 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:15.392412  2377 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:15.397747  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:15.400396  1574 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.403198  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.405812  1572 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:15.407310  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.409374  6447 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.417016  6447 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:15.383991  6834 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:03:15.424144  1646 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.367944  6732 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:15.382694  6836 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:15.390920  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.415809  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:15.404155  1642 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:15.434834  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.431881  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:15.503285  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87582.8us
I20260709 13:03:15.477977  1719 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.477145  1645 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:15.369985  6544 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.425818  6620 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:15.509621  6816 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:15.383249  6736 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.376655  6706 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:15.514473  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.515233  1575 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:15.514463  1645 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:15.431253  6815 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:15.532295  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:15.532558  1574 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:15.534037  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.522094  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.537003  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:15.537497  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.538192  1575 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 11: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:15.518997  1645 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.512171  6620 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.512787  6816 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:15.543882  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:15.549372  1724 raft_consensus.cc:606] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:15.549753  1724 raft_consensus.cc:614] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:15.544368  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.544840  1645 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:15.545915  1573 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:03:15.531479  6659 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.173s	user 0.015s	sys 0.002s
I20260709 13:03:15.521957  6803 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:15.560232  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.561758  1573 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:03:15.563038  6647 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.547484  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:15.540619  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:15.555142  2373 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.555411  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1971) took 1045 ms. Trace:
I20260709 13:03:15.435519  6759 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:03:15.540995  6571 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:15.543006  6730 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.998s	user 0.016s	sys 0.000s
I20260709 13:03:15.547082  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:15.566244  1644 rpcz_store.cc:276] 0709 13:03:14.510431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.511171 (+   740us) service_pool.cc:224] Handling call
0709 13:03:14.522547 (+ 11376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:14.563617 (+ 41070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.563655 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:14.607399 (+ 43744us) log.cc:844] Serialized 52 byte log entry
0709 13:03:14.784008 (+176609us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:14.784056 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.554962 (+770906us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b54000f5ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:15.555023 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.555087 (+    64us) raft_consensus.cc:1685] finished
0709 13:03:15.555103 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.555352 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199,"raft.queue_time_us":240,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":22160,"spinlock_wait_cycles":45780352,"thread_start_us":94468,"threads_started":1,"wal-append.queue_time_us":189918}
I20260709 13:03:15.569698  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:15.577392  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:15.578125  1724 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.582154  1645 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:03:15.583642  1645 pending_rounds.cc:85] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 8
I20260709 13:03:15.584084  1645 pending_rounds.cc:107] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:03:15.584435  1645 raft_consensus.cc:2889] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:15.591198  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.592461  1644 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:15.594635  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1978) took 1053 ms. Trace:
I20260709 13:03:15.594772  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:15.595034  2375 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.595145  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:15.531489  6355 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:03:15.595384  1725 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.528798  6706 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.595167  1645 rpcz_store.cc:276] 0709 13:03:14.531968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.400703 (+868735us) service_pool.cc:224] Handling call
0709 13:03:15.476941 (+ 76238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.585092 (+108151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.585356 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:15.610287  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.596786  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.588060  6815 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:15.595952  6730 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:15.619793  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:15.514566  6785 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.601378  6571 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:15.596915  6847 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.619093  1724 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:15.627859  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.641732  1717 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:15.637854  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.609997  6647 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:15.564325  6659 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.569744  6803 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.631951  6816 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.632541  6846 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:15.633579  6815 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:15.637141  6854 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:15.645612  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:15.648651  1642 raft_consensus.cc:606] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:15.649037  1642 raft_consensus.cc:614] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:15.646704  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.608771  6788 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:03:15.666733  1721 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:15.657738  6647 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.661096  6845 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:15.646221  6846 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:15.661746  6854 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:15.681068  6286 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.002s
I20260709 13:03:15.665516  6815 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.687523  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:15.688907  6848 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:03:15.658524  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:15.663995  6659 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:15.670651  6842 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.671705  1642 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:15.675767  6785 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:15.667126  6762 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:15.678541  6846 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:15.679107  6845 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:15.682159  6854 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:15.696628  1725 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:03:15.659547  6720 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.001s
I20260709 13:03:15.679939  6571 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.684171  6647 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.693807  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:15.737793  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44031.1us
I20260709 13:03:15.694583  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.667659  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.697523  6762 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.698956  1716 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:15.694564  6847 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:15.765960  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.701148  6845 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:15.702036  6779 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.005s
W20260709 13:03:15.702755  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.682442  6482 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.712230  6730 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.714818  6706 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.715844  6715 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:15.717994  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2316) took 1228 ms. Trace:
I20260709 13:03:15.778959  1650 rpcz_store.cc:276] 0709 13:03:14.491971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.492121 (+   150us) service_pool.cc:224] Handling call
0709 13:03:15.717741 (+1225620us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54000915d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:15.717904 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764688256}
I20260709 13:03:15.721217  1725 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.774413  6544 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.696611  1723 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
W20260709 13:03:15.734586  6840 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.449s	user 0.002s	sys 0.000s
W20260709 13:03:15.749475  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81979us
I20260709 13:03:15.764631  6457 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:15.723851  6797 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.671s	user 0.000s	sys 0.005s
I20260709 13:03:15.766419  6847 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:15.781738  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:15.781944  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.732537  6714 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.820578  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1974) took 1267 ms. Trace:
I20260709 13:03:15.823946  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.818318  1720 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.738940  6720 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.828197  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.776353  6647 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.778716  6715 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:15.842056  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:15.823709  1649 rpcz_store.cc:276] 0709 13:03:14.519880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.293353 (+773473us) service_pool.cc:224] Handling call
0709 13:03:15.293982 (+   629us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:15.294460 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.294505 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:15.306248 (+ 11743us) log.cc:844] Serialized 52 byte log entry
0709 13:03:15.484650 (+178402us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:15.636793 (+152143us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.641038 (+  4245us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.641628 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.641727 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.721179 (+ 79452us) spinlock_profiling.cc:243] Waited 9.82 ms on lock 0x7b180035a760. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:15.786669 (+ 65490us) raft_consensus.cc:1685] finished
0709 13:03:15.786708 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.786940 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151049,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":699,"raft.queue_time_us":16108,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":14570,"spinlock_wait_cycles":28562944,"thread_start_us":252377,"threads_started":2,"wal-append.queue_time_us":184565}
W20260709 13:03:15.844384  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.848908  1572 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:15.848178  6620 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:15.853283  6620 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.853708  6620 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.854900  6842 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.855264  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2645) took 1091 ms. Trace:
I20260709 13:03:15.774657  6779 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.829596  6482 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:15.778054  6730 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.864421  6715 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.867348  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.865459  6720 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.855700  1722 rpcz_store.cc:276] 0709 13:03:14.763913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.939505 (+175592us) service_pool.cc:224] Handling call
0709 13:03:14.992708 (+ 53203us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:15.026902 (+ 34194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.027141 (+   239us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:15.048550 (+ 21409us) log.cc:844] Serialized 52 byte log entry
0709 13:03:15.250940 (+202390us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:15.251016 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.251155 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.733805 (+482650us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:15.854832 (+121027us) raft_consensus.cc:1685] finished
0709 13:03:15.854855 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.855100 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32044,"raft.queue_time_us":33478,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":4703,"spinlock_wait_cycles":48517504,"thread_start_us":155783,"threads_started":1,"wal-append.queue_time_us":190308}
I20260709 13:03:15.841149  6457 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:15.870947  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.862864  6482 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:15.873121  6482 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:15.874708  6842 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.871919  6847 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.868444  6715 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:15.876444  6847 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:15.871910  6713 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.882072  6779 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:15.882277  6859 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:03:15.865917  6788 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.883206  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2646) took 1119 ms. Trace:
I20260709 13:03:15.869690  6714 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.886356  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 762) took 1108 ms. Trace:
I20260709 13:03:15.886790  1646 rpcz_store.cc:276] 0709 13:03:14.778404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.472647 (+694243us) service_pool.cc:224] Handling call
0709 13:03:15.473130 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.534681 (+ 61551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.534718 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.535170 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.602304 (+ 67134us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:15.602345 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.885424 (+283079us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b18003ce980. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419
0709 13:03:15.886019 (+   595us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b540005cdd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:15.886064 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.886115 (+    51us) raft_consensus.cc:1685] finished
0709 13:03:15.886130 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.886291 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1531,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5978,"spinlock_wait_cycles":161415168,"thread_start_us":40827,"threads_started":1,"wal-append.queue_time_us":41539}
I20260709 13:03:15.893275  6762 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:15.894747  6772 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.002s
I20260709 13:03:15.895254  6772 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:15.866283  6780 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.098s	user 0.013s	sys 0.003s
I20260709 13:03:15.896317  1569 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.786085  6846 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.883601  1721 rpcz_store.cc:276] 0709 13:03:14.764210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.984052 (+219842us) service_pool.cc:224] Handling call
0709 13:03:15.086408 (+102356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.667211 (+580803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.882958 (+215747us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400088754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:15.883146 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334228864}
I20260709 13:03:15.901983  6846 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.903070  2377 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:15.903963  6730 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:15.906955  1569 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:15.909139  6772 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:15.911660  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:15.914455  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:15.915169  1646 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:15.916108  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:15.922302  6447 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:15.923058  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.923834  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.924571  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.924603  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.928s (SENT)
I20260709 13:03:15.925143  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.925374  6785 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.926134  6785 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:15.927016  6826 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.813s	user 0.002s	sys 0.000s
I20260709 13:03:15.930145  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2317) took 1425 ms. Trace:
I20260709 13:03:15.930541  1647 rpcz_store.cc:276] 0709 13:03:14.505150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.505344 (+   194us) service_pool.cc:224] Handling call
0709 13:03:14.505942 (+   598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:14.506516 (+   574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.506563 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:14.528488 (+ 21925us) log.cc:844] Serialized 52 byte log entry
0709 13:03:14.764996 (+236508us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:14.800852 (+ 35856us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.801284 (+   432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.801405 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.422762 (+621357us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b18004049e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:15.926189 (+503427us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:15.929858 (+  3669us) raft_consensus.cc:1685] finished
0709 13:03:15.929883 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.930078 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41347,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":10707,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":6069,"spinlock_wait_cycles":53339392,"thread_start_us":200071,"threads_started":1,"wal-append.queue_time_us":397832}
W20260709 13:03:15.932566  6851 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.250s	user 0.002s	sys 0.001s
W20260709 13:03:15.925396  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.292s	user 0.005s	sys 0.037s
I20260709 13:03:15.933346  6785 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.922600  1721 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:15.935943  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:15.935968  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:15.937497  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.099s (SENT)
I20260709 13:03:15.937870  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.940831  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.943420  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.936252  6762 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.942384  6846 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:15.945792  6846 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:15.946702  6846 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.947080  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:15.955263  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2423) took 1194 ms. Trace:
I20260709 13:03:15.900071  6620 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.955844  1720 rpcz_store.cc:276] 0709 13:03:14.760751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.778803 (+ 18052us) service_pool.cc:224] Handling call
0709 13:03:14.779284 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.825778 (+1046494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.954844 (+129066us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400232654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:15.955167 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389227904}
I20260709 13:03:15.942167  6580 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.939356  6772 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:15.964371  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:15.969624  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.336s	user 0.005s	sys 0.037s
I20260709 13:03:15.953572  6780 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:15.962471  6571 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:03:15.970747  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.980599  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2238) took 2782 ms. Trace:
I20260709 13:03:15.982636  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:15.968961  6714 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:03:15.978724  6580 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.975549  6720 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.986457  6580 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.986685  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 698) took 1224 ms. Trace:
I20260709 13:03:15.980474  6457 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.987165  1723 rpcz_store.cc:276] 0709 13:03:14.762527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.779887 (+ 17360us) service_pool.cc:224] Handling call
0709 13:03:15.986479 (+1206592us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b5400232b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:15.986620 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942044800}
I20260709 13:03:15.978224  6786 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:15.978729  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:15.989657  6724 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.992230  6325 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:15.992578  6788 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:15.992712  6325 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:15.993258  6788 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:15.993935  6325 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:15.994270  6788 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:15.995023  6325 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:15.996613  6842 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:15.996781  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:15.997378  6779 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:15.998842  6803 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.999215  6482 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:15.989508  6843 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.572s	user 0.001s	sys 0.000s
I20260709 13:03:16.001060  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:16.001168  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.003685  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.004925  1719 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:15.989508  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.007447  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.064186  1647 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:16.009183  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.009119  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:15.982012  1571 rpcz_store.cc:276] 0709 13:03:13.195467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.536959 (+341492us) service_pool.cc:224] Handling call
0709 13:03:13.537361 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.537698 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:13.537723 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:13.544040 (+  6317us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.976930 (+2432890us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:15.976994 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.977112 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.977173 (+    61us) raft_consensus.cc:1685] finished
0709 13:03:15.977191 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.977374 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5709,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":2162,"thread_start_us":2343513,"threads_started":1,"wal-append.queue_time_us":669532,"wal-append.run_cpu_time_us":3681,"wal-append.run_wall_time_us":1126397}
I20260709 13:03:15.978669  1721 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:16.017467  6620 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:15.978485  6780 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:16.069370  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:16.069979  1571 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.068217  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.071146  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:16.018356  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:16.071784  1575 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:16.018672  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1972) took 1500 ms. Trace:
I20260709 13:03:16.018754  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:16.020466  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:15.967783  6847 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:16.027201  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.028000  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.028587  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:16.030177  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:16.035422  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.084126  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:16.084985  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.048594  1718 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:15.981199  6772 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:16.012938  6571 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:16.054898  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:16.056631  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.050824  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.090463  1723 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:16.022648  6713 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.026101  6742 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:16.074635  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65595us
I20260709 13:03:16.093797  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:16.086252  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.072203  1648 rpcz_store.cc:276] 0709 13:03:14.517867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.518657 (+   790us) service_pool.cc:224] Handling call
0709 13:03:14.519229 (+   572us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:14.519995 (+   766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.520042 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:14.520809 (+   767us) log.cc:844] Serialized 52 byte log entry
0709 13:03:14.645885 (+125076us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:14.701897 (+ 56012us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.702173 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.702260 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.934101 (+1231841us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b18003e5420. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:16.018278 (+ 84177us) raft_consensus.cc:1685] finished
0709 13:03:16.018388 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.018615 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48971,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":4969,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":43867,"spinlock_wait_cycles":15524096,"thread_start_us":124503,"threads_started":1,"wal-append.queue_time_us":221551}
I20260709 13:03:16.089649  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:16.092574  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.063683  6803 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:16.095088  1575 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:16.103503  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.104044  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.098145  1650 raft_consensus.cc:2368] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:16.104741  1575 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:16.105537  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.105759  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:16.101624  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.027486  6730 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:03:16.104305  1648 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:16.098501  1568 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:16.121809  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:16.122875  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.123910  1568 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:16.124861  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.125082  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:16.125732  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.125686  1568 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 9.
I20260709 13:03:16.126403  1650 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:03:16.127219  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.128326  1648 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:16.074478  6779 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.076865  6780 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.054740  6738 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:03:16.088840  6715 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:16.082531  6447 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:03:16.089241  6659 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:16.123111  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.140997  1571 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:16.129797  1648 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:16.130286  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:16.142930  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.143718  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2706) took 2148 ms. Trace:
I20260709 13:03:16.113198  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.065080  6815 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:16.097455  6620 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:16.096417  6720 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.103421  6803 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:16.107008  1573 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:16.144100  1570 rpcz_store.cc:276] 0709 13:03:13.995767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.432539 (+436772us) service_pool.cc:224] Handling call
0709 13:03:14.443632 (+ 11093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.501589 (+ 57957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.501643 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.522896 (+ 21253us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.359704 (+836808us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:15.359767 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.359870 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.988769 (+628899us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:16.140373 (+151604us) raft_consensus.cc:1685] finished
0709 13:03:16.140393 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.143657 (+  3264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65339,"raft.queue_time_us":71092,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9424,"spinlock_wait_cycles":20512640,"thread_start_us":814345,"threads_started":1,"wal-append.queue_time_us":884564}
I20260709 13:03:16.100582  6713 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:16.147500  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53790.8us
I20260709 13:03:16.148819  6847 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:16.144819  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:16.134599  6778 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:16.158406  1642 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:16.160609  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.161566  1571 raft_consensus.cc:2410] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:03:16.137247  6714 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:16.168622  6736 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:16.176872  1574 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:16.167825  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.165967  6778 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.181272  6778 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:16.184576  6872 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:03:16.186196  6871 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:03:16.184646  6829 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.190524  6713 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:16.192471  6772 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:16.196063  1539 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.666s (SENT) [suppressed 17 similar messages]
I20260709 13:03:16.197746  1568 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:16.205916  1567 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:16.207304  6713 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:16.210855  1567 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:16.213871  1567 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:16.211131  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45817.1us
I20260709 13:03:16.215723  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1977) took 1686 ms. Trace:
W20260709 13:03:16.219143  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.216259  1643 rpcz_store.cc:276] 0709 13:03:14.529064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.300194 (+771130us) service_pool.cc:224] Handling call
0709 13:03:15.348204 (+ 48010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.371687 (+ 23483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.215478 (+843791us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b54001bb2d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:16.215648 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109194496}
I20260709 13:03:16.155313  6544 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:16.159080  6788 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:16.224669  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 111662us
I20260709 13:03:16.225713  1642 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:16.231805  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.232440  1642 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.135957  6779 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.242936  6779 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:16.236897  1573 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:16.168625  6714 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:16.245275  6780 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.246531  6714 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:16.239166  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.162920  6715 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:16.248409  1643 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:16.249555  1643 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:16.249894  1643 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:16.250872  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.250696  1643 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:16.251703  1643 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.240288  6482 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:16.252626  1569 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:16.196237  6815 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:16.253656  1567 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:16.196597  6620 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:16.144994  6816 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:03:16.249075  6715 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:16.255304  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.228418  6788 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.161247  6785 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:03:16.275391  6788 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.214309  6736 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:16.248054  6780 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:16.165305  6842 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:03:16.228519  6544 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:16.209574  6876 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:03:16.263564  6815 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:16.225212  6713 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:16.301913  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:16.304796  1642 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:16.305580  1645 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:16.306007  1642 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:16.308503  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:16.309533  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:16.324081  2379 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
W20260709 13:03:16.319685  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 98127.8us
I20260709 13:03:16.336637  6786 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:16.337497  6786 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.345629  6736 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.352039  6778 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.353348  1643 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:03:16.353724  1643 pending_rounds.cc:85] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 8
I20260709 13:03:16.353933  1643 pending_rounds.cc:107] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:03:16.354827  6778 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:16.364135  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:16.366919  1643 raft_consensus.cc:2889] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:16.367817  6832 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:16.368561  6762 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:16.369040  6832 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.369411  6832 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.371088  6762 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:16.371671  6762 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:16.376926  6874 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.148s	user 0.000s	sys 0.003s
I20260709 13:03:16.384011  6832 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:16.389285  6832 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:16.394527  6762 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:16.395699  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.324973  6876 consensus_queue.cc:1243] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 13.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:16.399619  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:16.402477  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2418) took 1698 ms (client timeout 2234 ms). Trace:
I20260709 13:03:16.442132  1643 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.411298  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:16.239166  6847 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.293860  6544 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.448563  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.425398  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.318544  6842 consensus_queue.cc:1243] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:16.455559  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:16.437178  1645 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:16.263442  6882 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:03:16.442297  1716 rpcz_store.cc:269] 0709 13:03:14.704534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.704778 (+   244us) service_pool.cc:224] Handling call
0709 13:03:16.402308 (+1697530us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b540010d6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:16.402419 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397933824}
I20260709 13:03:16.451300  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.450101  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.459445  1725 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:16.453884  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.460453  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:16.461020  1717 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.450950  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:16.447961  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.461829  1719 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:16.462519  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.432727  6752 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.467744  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.465925  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.466112  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:16.471581  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.472332  1716 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:03:16.465073  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 269045us
I20260709 13:03:16.476258  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:16.511170  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 183781us
I20260709 13:03:16.510398  1643 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:16.512215  6772 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.478425  6724 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.473951  6286 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.463655  6888 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.516835  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:16.533694  1567 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:16.521035  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.524902  2380 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:16.527179  2375 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:16.528941  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:16.518829  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.539783  2379 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:16.518678  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.529455  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.542747  1643 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:16.529117  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:16.440744  6620 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.566390  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:16.573342  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:16.574965  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54041us
I20260709 13:03:16.584807  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:16.471653  6752 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:16.514513  6893 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:16.466115  6815 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.582463  6714 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:16.570334  6778 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:16.514477  6886 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.589802  6286 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:03:16.527927  6785 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:03:16.577761  6738 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.591182  6715 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:03:16.525779  6772 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.590400  6888 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.555265  6806 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
W20260709 13:03:16.596210  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.667048  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.678558  1645 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:16.596993  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.590528  6762 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:03:16.598471  6876 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.600484  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:16.694449  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 94088.4us
I20260709 13:03:16.695653  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:16.642827  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 191174us
I20260709 13:03:16.643311  6778 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:16.613688  6883 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:03:16.613008  6885 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:03:16.615675  6887 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:03:16.823559  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 128021us
I20260709 13:03:16.828794  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.669878  6713 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:16.597734  6893 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:16.856465  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 337970us
I20260709 13:03:16.865134  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.692584  6886 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:16.868729  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.820158  6778 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:16.600651  6846 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.256s
I20260709 13:03:16.907855  1642 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:16.907078  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.918337  1648 raft_consensus.cc:2376] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11. Re-sending same reply.
I20260709 13:03:16.919019  1642 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:16.921029  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 366812us
I20260709 13:03:16.599977  6879 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.267s
W20260709 13:03:16.921618  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.006s	sys 0.039s
W20260709 13:03:16.927407  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.006s	sys 0.039s
I20260709 13:03:16.885288  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:16.928285  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 665086us
W20260709 13:03:16.892982  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.620219  6786 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:16.878934  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:16.948976  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:16.991600  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:16.993431  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:16.992760  1645 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:16.873107  6886 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:16.992705  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:16.950781   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:03:17.004491  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75198.9us
I20260709 13:03:16.872200  6713 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:16.884236  6785 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.827308  6847 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:16.668584  6714 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.046864  1690 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:03:17.067838  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75222.3us
W20260709 13:03:17.069557  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 190742us
W20260709 13:03:17.077255  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.079247  1717 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.000s
I20260709 13:03:17.079653  1717 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:17.082849  1644 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:17.068759  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:16.878217  6905 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.183s
I20260709 13:03:17.027030  6806 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:17.089093  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1981) took 1003 ms. Trace:
I20260709 13:03:17.105480  1647 rpcz_store.cc:276] 0709 13:03:16.085518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.085902 (+   384us) service_pool.cc:224] Handling call
0709 13:03:16.086228 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.100529 (+ 14301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.100579 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.112702 (+ 12123us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.271985 (+159283us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:16.272048 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.272156 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.088777 (+816621us) raft_consensus.cc:1685] finished
0709 13:03:17.088807 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.089017 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6536,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":24257,"thread_start_us":134144,"threads_started":1,"wal-append.queue_time_us":277587}
W20260709 13:03:17.096284  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49554.6us
I20260709 13:03:17.110816  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:17.103272  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:17.113483  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:17.115028  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:17.094552  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:17.054718  6847 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:17.117360  6847 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.117590  6894 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:17.118230  6886 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.119565  6886 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.121053  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1993) took 1024 ms. Trace:
I20260709 13:03:17.041320  6832 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.915520  6888 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.144s
I20260709 13:03:17.121750  1646 rpcz_store.cc:276] 0709 13:03:16.096550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.157186 (+ 60636us) service_pool.cc:224] Handling call
0709 13:03:16.157570 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:16.196829 (+ 39259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.196858 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:16.215083 (+ 18225us) log.cc:844] Serialized 52 byte log entry
0709 13:03:16.295934 (+ 80851us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:16.353100 (+ 57166us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b48001c0158. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:03:16.583198 (+230098us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.676056 (+ 92858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.676140 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.095309 (+419169us) raft_consensus.cc:1685] finished
0709 13:03:17.120731 (+ 25422us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.120969 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95154,"prepare.queue_time_us":150450,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":137037,"raft.run_cpu_time_us":1386,"raft.run_wall_time_us":273061,"spinlock_wait_cycles":6194048,"thread_start_us":209845,"threads_started":2,"wal-append.queue_time_us":201613}
I20260709 13:03:17.121994  6901 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.109159  6785 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:17.147282  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:17.150805  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.152657  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.156926  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.160894  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:17.151647  6816 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:17.150367  6901 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:17.150111  6785 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:17.155049  6752 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:17.157150  6730 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:17.128811  6806 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:17.208833  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 172256us
I20260709 13:03:17.121080  6803 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.174726  6816 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:17.225046  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.199509  6901 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.084194  6893 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.226349  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:17.205668  6785 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:17.229553  6909 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:03:17.213757  6730 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:17.205969  6772 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.229801  6752 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.232672  6816 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:17.253338  6900 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:17.251334  6730 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:17.255817  6908 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:03:17.234094  6916 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.256875  6906 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:17.251497  6752 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:17.227916  6620 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:17.251762  6738 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.258809  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:17.257921  6896 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:03:17.228997  6821 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.245s
I20260709 13:03:17.252035  6893 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.286223  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.280375  6917 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:17.272909  6714 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:17.298784  6724 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.772s	user 0.010s	sys 0.000s
W20260709 13:03:17.299230  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 206) took 31501 ms (client timeout 15000 ms). Trace:
I20260709 13:03:17.272596  6900 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:17.192749  6913 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:17.310802  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 153963us
W20260709 13:03:17.311359  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 216887us
W20260709 13:03:17.311797  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 241519us
I20260709 13:03:17.266685  6772 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:17.270571  6752 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.278262  6738 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.272401  6906 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:17.336874  6724 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:17.341078  6714 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:17.344748  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.360091  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2726) took 1323 ms. Trace:
I20260709 13:03:17.365967  1568 rpcz_store.cc:276] 0709 13:03:16.037181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.126979 (+ 89798us) service_pool.cc:224] Handling call
0709 13:03:17.359908 (+1232929us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b54002a76d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:17.360025 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160621312}
I20260709 13:03:17.326092  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:17.269723  6916 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.318881  6620 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.344795  6900 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.382290  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.344655  6713 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:17.343690  6717 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:17.393441  1568 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 13:03:17.365031  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140060us
I20260709 13:03:17.399155  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:17.339468  6772 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.402992  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:17.407485  1574 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.332692  6917 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.364789  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:17.396730  6905 consensus_queue.cc:1243] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:17.400708  6821 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:17.401487  6714 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:17.435628  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 70920.3us
I20260709 13:03:17.445426  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:17.447819  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:17.451165  1574 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:17.341886  6913 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.430272  6806 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:17.454459  6905 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:17.431787  6926 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.464453  1541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.023s
W20260709 13:03:17.469383  1541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.013s	sys 0.024s
I20260709 13:03:17.452204  6923 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:17.453469  6886 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:17.399998  6900 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.460716  6845 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:17.465059  6714 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:17.472487  6905 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:17.504452  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 335577us
W20260709 13:03:17.487784  6912 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 13:03:17.551319  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:17.552873  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 208546us
W20260709 13:03:17.566365  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 154710us
I20260709 13:03:17.486370  6922 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:17.493777  6845 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:17.584066  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 128881us
I20260709 13:03:17.493884  6918 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:17.553838  6915 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.587558  2342 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:17.576900  6457 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:17.330775  1467 rpcz_store.cc:269] 0709 13:02:45.767422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.767671 (+   249us) service_pool.cc:224] Handling call
0709 13:02:46.563845 (+796174us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:03:17.265286 (+30701441us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:07.089971 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:07.090688 (+   717us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:07.090740 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:03:07.090776 (+    36us) tablet.cc:662] Decoding operations
0709 13:03:07.132164 (+ 41388us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:07.132254 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:07.132283 (+    29us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:03:07.163896 (+ 31613us) tablet.cc:701] Row locks acquired
0709 13:03:07.163953 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:03:07.211217 (+ 47264us) write_op.cc:270] Start()
0709 13:03:07.244697 (+ 33480us) write_op.cc:276] Timestamp: P: 1783602187186525 usec, L: 0
0709 13:03:07.250119 (+  5422us) op_driver.cc:348] REPLICATION: starting
0709 13:03:07.416619 (+166500us) log.cc:844] Serialized 9562 byte log entry
0709 13:03:07.669397 (+252778us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b
0709 13:03:09.177876 (+1508479us) op_driver.cc:464] REPLICATION: finished
0709 13:03:09.617564 (+439688us) write_op.cc:301] APPLY: starting
0709 13:03:09.636851 (+ 19287us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:09.726496 (+ 89645us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:09.726540 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:10.134999 (+408459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:10.135633 (+   634us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 13:03:10.135697 (+    64us) write_op.cc:312] APPLY: finished
0709 13:03:10.136575 (+   878us) log.cc:844] Serialized 212 byte log entry
0709 13:03:10.422086 (+285511us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b
0709 13:03:10.636958 (+214872us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:10.751825 (+114867us) write_op.cc:454] Released schema lock
0709 13:03:10.751934 (+   109us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:10.752656 (+   722us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:10.752696 (+    40us) write_op.cc:454] Released schema lock
0709 13:03:10.923182 (+170486us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
Metrics: {"mutex_wait_us":3712,"reported_tablets":52,"spinlock_wait_cycles":105851776,"child_traces":[["op",{"apply.queue_time_us":391600,"apply.run_cpu_time_us":75135,"apply.run_wall_time_us":1154571,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104591,"num_ops":24,"prepare.queue_time_us":119018,"prepare.run_cpu_time_us":119965,"prepare.run_wall_time_us":1667986,"raft.queue_time_us":342341,"raft.run_cpu_time_us":32687,"raft.run_wall_time_us":314070,"replication_time_us":1892026,"spinlock_wait_cycles":284403712,"thread_start_us":2231503,"threads_started":5,"wal-append.queue_time_us":1317645,"wal-append.run_cpu_time_us":68654,"wal-append.run_wall_time_us":3092490}]]}
W20260709 13:03:17.604364  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 205288us
I20260709 13:03:17.597973  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 712) took 1140 ms. Trace:
W20260709 13:03:17.580914  6544 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.114s	user 0.011s	sys 0.003s
I20260709 13:03:17.600874  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:17.475512  6806 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:03:17.607153  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:17.605463  1723 rpcz_store.cc:276] 0709 13:03:16.457357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.475495 (+ 18138us) service_pool.cc:224] Handling call
0709 13:03:16.509612 (+ 34117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.543864 (+ 34252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.543939 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.544563 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.027675 (+483112us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:17.027787 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.027936 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.597739 (+569803us) raft_consensus.cc:1685] finished
0709 13:03:17.597758 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.597912 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33847,"raft.queue_time_us":83837,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":19720,"thread_start_us":459838,"threads_started":1,"wal-append.queue_time_us":667250}
I20260709 13:03:17.605803  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:17.602476  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 588) took 1125 ms. Trace:
I20260709 13:03:17.609486  1570 rpcz_store.cc:276] 0709 13:03:16.477285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.477826 (+   541us) service_pool.cc:224] Handling call
0709 13:03:16.478191 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.479912 (+  1721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.479944 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.527432 (+ 47488us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.875047 (+347615us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:17.387465 (+512418us) log.cc:844] Serialized 14 byte log entry
0709 13:03:17.601639 (+214174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.601851 (+   212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.602276 (+   425us) raft_consensus.cc:1685] finished
0709 13:03:17.602288 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.602417 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46360,"prepare.queue_time_us":707901,"raft.queue_time_us":127021,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":64631,"thread_start_us":958062,"threads_started":3,"wal-append.queue_time_us":613011}
I20260709 13:03:17.608889  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:17.580623  6714 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:17.593703  6858 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.740s	user 0.017s	sys 0.014s
I20260709 13:03:17.581133  6717 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:17.474794  6926 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.455152  6906 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.598451  6918 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:17.631400  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60289us
I20260709 13:03:17.551736  6713 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.603250  6457 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:17.617851  6858 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.957s	user 0.017s	sys 0.015s
I20260709 13:03:17.550719  6928 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:03:17.637751  1723 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:17.653620  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2442) took 1105 ms. Trace:
I20260709 13:03:17.654167  1723 rpcz_store.cc:276] 0709 13:03:16.548245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.618244 (+1069999us) service_pool.cc:224] Handling call
0709 13:03:17.630719 (+ 12475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.653271 (+ 22552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.653533 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.656852  1575 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:17.661489  6878 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:17.667551  1567 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:17.585719  6921 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.256s
I20260709 13:03:17.613415  6821 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:17.677210  1575 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:17.679924  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:17.680934  1570 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:17.681461  1570 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:17.619103  6544 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.689818  6941 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:03:17.690637  6706 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:17.691222  6544 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.691864  6717 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.693013  6931 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:17.694048  6931 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.685726  1576 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:17.634564  6918 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:17.687708  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2443) took 1137 ms. Trace:
I20260709 13:03:17.688390  1575 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:17.654057  6916 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:17.699590  1723 rpcz_store.cc:276] 0709 13:03:16.550145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.655224 (+1105079us) service_pool.cc:224] Handling call
0709 13:03:17.687078 (+ 31854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.687288 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.687321 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:17.687363 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:17.687395 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.687458 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.687622 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.670907  6620 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.656507  6772 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.624445  6714 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.653206  6286 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:17.621446  6816 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.677712  6785 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.703436  6913 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:17.703845  6933 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:17.705384  6911 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:03:17.604386  6915 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.621981  6902 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.253s
I20260709 13:03:17.706913  1570 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.734764  1570 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:17.719480  1723 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:17.735811  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 716) took 1185 ms. Trace:
I20260709 13:03:17.736207  1723 rpcz_store.cc:276] 0709 13:03:16.550468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.701803 (+1151335us) service_pool.cc:224] Handling call
0709 13:03:17.718961 (+ 17158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.735535 (+ 16574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.735749 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.705859  6876 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.728482  6924 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:17.708205  6929 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:17.710608  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 159385us
I20260709 13:03:17.738867  1722 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:17.701901  6905 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.702807  6544 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:17.740293  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 152733us
I20260709 13:03:17.714394  6938 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:17.713783  6917 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.752264  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:17.752127  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:17.754814  1723 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:17.759647  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 717) took 1110 ms. Trace:
I20260709 13:03:17.760046  1723 rpcz_store.cc:276] 0709 13:03:16.649759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.737314 (+1087555us) service_pool.cc:224] Handling call
0709 13:03:17.754363 (+ 17049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.759373 (+  5010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.759593 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:17.737178  6913 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:17.724901  6457 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.750432  6876 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.766552  6807 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:17.751004  6847 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.633s	user 0.015s	sys 0.002s
I20260709 13:03:17.774233  6807 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:17.788676  6457 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.789674  6807 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.793262  6900 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:17.794013  6900 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:17.794188  6714 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.794436  6900 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:17.795563  6900 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.759945  6924 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:17.770080  6941 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:17.762593  6929 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:17.771575  6913 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:17.798441  6788 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.506s	user 0.013s	sys 0.000s
W20260709 13:03:17.811117  6936 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:03:17.793110  6876 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:17.866647  6924 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.817095  6902 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:17.849783  6913 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:17.842173  6929 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:17.737541  6916 consensus_queue.cc:1175] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd6d377efe514bc58ae49c3f4a107232" responder_term: 15 status { last_received { term: 13 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:17.785480  6847 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:17.930739  6926 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:17.936815  6892 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.269s	user 0.002s	sys 0.000s
I20260709 13:03:17.938323  6919 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.165s
I20260709 13:03:17.941826  6931 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:17.942098  6951 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.132s
W20260709 13:03:17.944865  6899 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.920s	user 0.005s	sys 0.000s
I20260709 13:03:17.946043  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 713) took 1488 ms. Trace:
I20260709 13:03:17.946307  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1980) took 1861 ms. Trace:
I20260709 13:03:17.946787  1649 rpcz_store.cc:276] 0709 13:03:16.085204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.087368 (+  2164us) service_pool.cc:224] Handling call
0709 13:03:16.087667 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.093972 (+  6305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.094003 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.094390 (+   387us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.441272 (+346882us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b1800268720. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:16.584859 (+143587us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:16.584903 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.584982 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:17.769685 (+1184703us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:17.946057 (+176372us) raft_consensus.cc:1685] finished
0709 13:03:17.946083 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.946231 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6327,"raft.queue_time_us":6599,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":25835,"spinlock_wait_cycles":16358656,"thread_start_us":143399,"threads_started":1,"wal-append.queue_time_us":724017}
I20260709 13:03:17.946529  1717 rpcz_store.cc:276] 0709 13:03:16.457603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.198557 (+740954us) service_pool.cc:224] Handling call
0709 13:03:17.231913 (+ 33356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.276815 (+ 44902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.276872 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:17.276929 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:17.276972 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.945723 (+668751us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b5400088254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:17.945775 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.945960 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102673280}
I20260709 13:03:17.865413  6788 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:17.960490  6939 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.962872  6847 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:17.951512  6926 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:17.943971  6902 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 13:03:17.997012  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.973436  6939 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:18.005520  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.984277  6926 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:18.008526  6919 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:17.998329  6785 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:18.007145  6893 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.004s
W20260709 13:03:18.016574  6778 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.013s	user 0.004s	sys 0.013s
I20260709 13:03:18.007483  6816 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:18.026077  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 409809us
I20260709 13:03:18.006040  6939 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.027679  6806 consensus_queue.cc:1243] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:18.029551  6915 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.022555  6935 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:18.076812  1717 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:18.046077  6893 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:18.033622  6778 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:18.090618  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 481812us
I20260709 13:03:18.086576  6941 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:18.098220  6941 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.088229  6325 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.115494  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:18.102581  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 280835us
I20260709 13:03:18.042188  6926 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.129434  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:18.130628  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.130466  2375 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 8.
I20260709 13:03:18.143390  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:18.092247  6919 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:18.089934  6893 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.151968  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.152905  2375 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:18.154019  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:18.154603  1575 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:18.127142  6788 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.158074  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.150851  6780 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.149912  6714 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:18.166059  6730 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.896s	user 0.008s	sys 0.005s
I20260709 13:03:18.166615  6956 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:18.168674  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:18.169493  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 718) took 1471 ms. Trace:
I20260709 13:03:18.174582  2375 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:18.159789  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:18.131489  6955 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.119578  6915 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:03:18.138692  6806 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.186581  6780 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.251346  6780 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.251955  6931 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.252534  6935 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.253232  6738 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.255566  6715 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.256222  6786 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.256115  6715 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.258381  6715 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.259303  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:18.201131  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:18.260601  6724 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.261402  6724 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.261826  6724 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.151225  6926 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.270082  6803 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.253247  6935 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:18.202404  6752 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.860s	user 0.014s	sys 0.000s
W20260709 13:03:18.205289  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.206790  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.221760  6957 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.222437  6893 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.155501  6816 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:18.289134  6816 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.291711  6816 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.127619  6325 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.293942  6325 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.223312  6917 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.294749  6917 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.224967  6714 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.229218  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:18.233887  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.236140  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:18.225539  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:18.185048  6902 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:03:18.185832  6960 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.150084  6913 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.245972  6955 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.314509  6913 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.315562  1569 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:18.299404  6944 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.249013  6806 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.188235  1723 rpcz_store.cc:276] 0709 13:03:16.698107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.761032 (+1062925us) service_pool.cc:224] Handling call
0709 13:03:17.769578 (+  8546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:17.797580 (+ 28002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:17.797609 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:17.798088 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.086287 (+288199us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:18.086346 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.086412 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.169253 (+ 82841us) raft_consensus.cc:1685] finished
0709 13:03:18.169274 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.169436 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27553,"raft.queue_time_us":53896,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":29556,"thread_start_us":287809,"threads_started":1,"wal-append.queue_time_us":368159}
I20260709 13:03:18.252741  6931 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.189543  1567 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.181809  6713 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.283596  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 122970us
I20260709 13:03:18.228540  6831 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:03:18.235011  6730 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.286083  6752 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.294062  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.333204  1725 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:18.226145  6971 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:18.160100  6717 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.302156  2380 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.228551  6778 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.340623  6717 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.336002  6713 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.349507  6713 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:18.303709  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 78818.2us
I20260709 13:03:18.274423  6923 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:18.309260  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80113us
I20260709 13:03:18.312604  6712 raft_consensus.cc:993] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:18.249439  1649 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:18.318038  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.318682  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 714) took 1860 ms. Trace:
I20260709 13:03:18.356067  1722 rpcz_store.cc:276] 0709 13:03:16.457828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.282573 (+824745us) service_pool.cc:224] Handling call
0709 13:03:17.320990 (+ 38417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.747293 (+426303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.318437 (+571144us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b5400087854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:18.318623 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":960652544}
I20260709 13:03:18.316913  6955 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.304409  6878 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:18.316715  6957 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:18.296303  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.303149  6951 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.334306  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.366816  1725 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:18.305589  6929 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:18.321202  6901 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.073s	user 0.010s	sys 0.006s
I20260709 13:03:18.336298  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.383410  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.383555  1723 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:03:18.384855  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.386050  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:18.386708  6806 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:18.388583  6730 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:18.389921  6785 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:18.390508  6785 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.391323  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:18.392239  6803 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:18.392673  6736 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:18.393677  6803 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:18.393980  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:18.358772  6724 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.354805  6923 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:18.394794  1649 raft_consensus.cc:2368] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:18.367946  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.395742  6923 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.397540  6946 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.398058  6923 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:18.334877  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.362325  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.394794  2373 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:18.400718  1721 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:18.327226  6944 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.371752  6939 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.372655  6712 raft_consensus.cc:1081] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:18.373131  1573 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:18.374430  6935 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:18.374884  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.416325  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.417752  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.417840  2380 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:18.376705  6714 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.372815  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.355427  6937 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:18.424703  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.425501  1648 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:18.426614  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.427330  6888 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.393348  6736 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.356894  6906 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.427866  6888 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:18.428550  6906 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.428800  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.429010  6906 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.429402  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:18.430018  6736 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.430207  6888 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.430365  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.431608  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.395371  1647 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:18.404749  1642 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:03:18.381191  6918 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.003s
I20260709 13:03:18.379069  6931 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.392725  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:18.412971  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:18.414238  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.442935  1645 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:18.422025  6499 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.319754  6960 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.422533  2375 raft_consensus.cc:2410] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:03:18.423494  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.424842  6937 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:18.427762  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.357295  6717 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:18.380430  6951 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.432140  1721 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:18.423743  6928 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.426501  6752 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:18.490442  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:18.412134  1567 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:18.405136  6901 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:18.480422  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 87772.6us
I20260709 13:03:18.480553  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.482331  1648 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:18.462958  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.487247  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.398126  6946 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.492987  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:18.499927  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.500218  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.479686  1649 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:18.505452  1720 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:18.513048  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.506631  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:18.515314  1721 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:18.516373  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:18.517123  1649 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.517864  1651 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:18.485358  6714 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:18.484653  6724 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.473730  6918 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.505255  6821 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:18.525395  1648 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:18.527093  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.491508  6785 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.481463  6499 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.514887  6752 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.499306  6931 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:03:18.543025  1575 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:03:18.523821  6447 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:18.544795  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:18.549806  1649 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:18.558151  1719 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:18.550010  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:18.551702  2378 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:18.551878  6944 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.553526  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2271) took 2482 ms. Trace:
I20260709 13:03:18.554292  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:18.494050  6928 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:18.558662  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:18.559748  1573 rpcz_store.cc:276] 0709 13:03:16.071503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.244295 (+172792us) service_pool.cc:224] Handling call
0709 13:03:16.244668 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.411601 (+2166933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.553230 (+141629us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b54001aa4d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:18.553457 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4894679424}
I20260709 13:03:18.532395  6821 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.569487  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:18.555135  1720 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10.
I20260709 13:03:18.575938  1688 leader_election.cc:400] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:18.535017  6917 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.523344  6714 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:18.560686  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:18.578171  1688 leader_election.cc:403] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:18.567102  6951 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:18.528491  6937 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:19.111555  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 535778us
I20260709 13:03:19.113209  1567 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:18.816345  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.122146  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.122802  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.123148  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.123521  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.123802  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:18.696699  2372 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:03:18.586875  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60116.1us
I20260709 13:03:19.113219  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:18.567245  6447 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:18.796554  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:18.545075  6752 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:19.192517  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53031.5us
W20260709 13:03:19.568614  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.003s	sys 0.005s
W20260709 13:03:19.907196  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.003s	sys 0.011s
W20260709 13:03:19.907907  1649 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.203s	user 0.002s	sys 0.019s
I20260709 13:03:19.266502  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:19.298542  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 615) took 1110 ms. Trace:
I20260709 13:03:19.139621  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:19.526974  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 608) took 1149 ms. Trace:
I20260709 13:03:19.182097  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:19.914079  1642 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:18.719748  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:19.527954  2378 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:19.568893  1724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.001s	sys 0.015s
W20260709 13:03:19.918872  1724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.002s	sys 0.020s
W20260709 13:03:19.908284  1649 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.002s	sys 0.019s
I20260709 13:03:18.558041  6913 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.906265  2376 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:19.909720  2375 rpcz_store.cc:276] 0709 13:03:18.192831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.205214 (+ 12383us) service_pool.cc:224] Handling call
0709 13:03:19.298153 (+1092939us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54006fed54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:19.298439 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317465984}
I20260709 13:03:19.916235  1573 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:19.973193  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2299) took 1584 ms (client timeout 1522 ms). Trace:
W20260709 13:03:19.974141  1573 rpcz_store.cc:269] 0709 13:03:18.388273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.697004 (+308731us) service_pool.cc:224] Handling call
0709 13:03:19.973096 (+1276092us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:19.526909  6884 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.416s	user 0.001s	sys 0.001s
W20260709 13:03:19.379741  6716 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.007s
I20260709 13:03:20.007356  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:18.581528  6902 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:19.981436  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2356) took 1265 ms (client timeout 1545 ms). Trace:
I20260709 13:03:19.234670  6917 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:19.911478  2377 rpcz_store.cc:276] 0709 13:03:18.377944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.378135 (+   191us) service_pool.cc:224] Handling call
0709 13:03:18.378526 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.379011 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:18.379044 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.379573 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.719314 (+339741us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:18.719374 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.719462 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.526567 (+807105us) raft_consensus.cc:1685] finished
0709 13:03:19.526600 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.526840 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3632,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"thread_start_us":339357,"threads_started":1,"wal-append.queue_time_us":1144797}
I20260709 13:03:19.653030  6778 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.621349  6355 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.016s	sys 0.044s
I20260709 13:03:18.577032  6821 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.581984  6803 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:19.273535  6980 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.433s	user 0.000s	sys 0.001s
I20260709 13:03:18.577831  6878 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:18.568154  6482 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:19.269042  6979 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.551s	user 0.001s	sys 0.000s
W20260709 13:03:19.291431  6926 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.027s	user 0.010s	sys 0.000s
W20260709 13:03:19.268885  6976 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.723s	user 0.001s	sys 0.000s
I20260709 13:03:19.181316  6730 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:18.877054  6736 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.797222  6959 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.866122  6888 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.174624  6929 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:18.629524  6785 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:19.132790  6714 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:20.014123  1642 rpcz_store.cc:269] 0709 13:03:18.716249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.181799 (+465550us) service_pool.cc:224] Handling call
0709 13:03:19.981313 (+799514us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:19.283735  6906 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.000s
I20260709 13:03:19.084873  6901 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:19.966392  6954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.008s	sys 0.011s
I20260709 13:03:19.299858  6930 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:19.002023  6957 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:18.692703  6713 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:18.558059  6806 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 1.422s
I20260709 13:03:19.899893  6931 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:19.969218  2376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 615) took 1484 ms (client timeout 1744 ms). Trace:
I20260709 13:03:19.916472  6815 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.023730  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:20.006716  6716 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.000s	sys 0.013s
I20260709 13:03:20.025274  6235 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:03:20.078063  2376 rpcz_store.cc:269] 0709 13:03:18.485203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.497574 (+ 12371us) service_pool.cc:224] Handling call
0709 13:03:19.969136 (+1471562us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.038312  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:20.028023  6355 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.018s	sys 0.055s
I20260709 13:03:20.046789  1717 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:20.023764  6986 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.002756  6913 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:20.054090  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.057240  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1998) took 1811 ms. Trace:
I20260709 13:03:20.034372  6917 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:20.042169  6878 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:20.066308  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:20.043993  6803 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:20.130916  6878 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.029181  6778 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.135099  6778 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.057067  6752 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:20.137056  6995 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:20.139797  6730 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:20.043121  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1986) took 3950 ms. Trace:
W20260709 13:03:20.069128  6954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.009s	sys 0.011s
I20260709 13:03:20.080703  6929 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.134924  6917 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.140942  6941 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:20.143357  6845 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.144272  6913 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.186816  6913 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.186865  6845 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.187463  6913 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.157994  6985 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:20.188557  6917 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.188555  6985 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.189636  6913 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.146716  6960 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:20.069043  6499 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:20.148056  6878 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.137101  6914 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:20.131639  6803 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.190775  6914 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:20.191202  6985 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.191792  6914 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.193193  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2676) took 2139 ms. Trace:
I20260709 13:03:20.060431  6959 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.046296  6482 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.200896  6914 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.058274  6736 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.109802  6235 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.063768  6888 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.110816  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 611) took 2011 ms. Trace:
I20260709 13:03:20.209961  2379 rpcz_store.cc:276] 0709 13:03:18.099748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.099979 (+   231us) service_pool.cc:224] Handling call
0709 13:03:18.100512 (+   533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.101089 (+   577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.101131 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.101715 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.556993 (+455278us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:18.557035 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.557098 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.110410 (+1553312us) raft_consensus.cc:1685] finished
0709 13:03:20.110441 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.110720 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":454933,"threads_started":1,"wal-append.queue_time_us":415514}
I20260709 13:03:20.081517  6901 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:20.110139  6923 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.697s	user 0.016s	sys 0.001s
I20260709 13:03:20.157876  6730 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:20.106359  6930 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.068673  6906 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.077934  6447 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.185091  6941 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:20.185720  6929 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.182317  6778 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.155460  6986 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.106070  6931 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.105818  6713 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.218267  6901 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.244994  6930 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.249668  6959 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.251585  6878 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.254698  6986 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:20.217284  6969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.368s	user 0.000s	sys 0.041s
W20260709 13:03:20.277197  6969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.441s	user 0.000s	sys 0.042s
W20260709 13:03:20.080112  1540 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.014s	sys 0.014s
W20260709 13:03:20.328758  1540 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.146s	user 0.016s	sys 0.014s
I20260709 13:03:20.329151  1572 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:03:20.081976  1573 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.147993  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2285) took 2005 ms. Trace:
W20260709 13:03:20.332492  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.012s	sys 0.045s
W20260709 13:03:20.333077  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.012s	sys 0.045s
I20260709 13:03:20.151952  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2343) took 3254 ms. Trace:
I20260709 13:03:20.151327  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:20.111572  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2287) took 1968 ms. Trace:
I20260709 13:03:20.348790  1575 rpcz_store.cc:276] 0709 13:03:18.143719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.155200 (+ 11481us) service_pool.cc:224] Handling call
0709 13:03:18.155436 (+   236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.543717 (+388281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.111217 (+1567500us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54003638d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:20.111502 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890691456}
I20260709 13:03:20.334393  1644 rpcz_store.cc:276] 0709 13:03:16.897169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.905573 (+  8404us) service_pool.cc:224] Handling call
0709 13:03:16.918408 (+ 12835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.083382 (+164974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.151678 (+3068296us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540038b3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:20.151884 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377144704}
W20260709 13:03:20.336297  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.004s	sys 0.021s
W20260709 13:03:20.353440  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.005s	sys 0.021s
W20260709 13:03:20.354133  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.24166e+06us
W20260709 13:03:20.355286  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.2316e+06us
I20260709 13:03:20.356155  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:20.356796  1642 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:20.129941  1646 rpcz_store.cc:276] 0709 13:03:18.245816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.249864 (+  4048us) service_pool.cc:224] Handling call
0709 13:03:18.250237 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.252085 (+  1848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.252120 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.252977 (+   857us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.520799 (+267822us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:18.520846 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.520922 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.056913 (+1535991us) raft_consensus.cc:1685] finished
0709 13:03:20.056938 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.057156 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1375,"raft.queue_time_us":1980,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":267451,"threads_started":1,"wal-append.queue_time_us":324209}
I20260709 13:03:20.359529  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:20.337863  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2010) took 1926 ms. Trace:
I20260709 13:03:20.176081  1650 rpcz_store.cc:276] 0709 13:03:16.086877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.127150 (+ 40273us) service_pool.cc:224] Handling call
0709 13:03:16.127488 (+   338us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:16.138736 (+ 11248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.138765 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:16.146921 (+  8156us) log.cc:844] Serialized 78 byte log entry
0709 13:03:16.291983 (+145062us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:16.317796 (+ 25813us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.318269 (+   473us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.318476 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.318587 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.066729 (+2748142us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b180046b940. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:20.036587 (+969858us) raft_consensus.cc:1685] finished
0709 13:03:20.036637 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.036949 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28796,"prepare.queue_time_us":349350,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":5338,"raft.queue_time_us":8309,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":166170,"spinlock_wait_cycles":13269376,"thread_start_us":111205,"threads_started":1,"wal-append.queue_time_us":278388}
W20260709 13:03:20.339128  2374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.501s	user 0.003s	sys 0.027s
I20260709 13:03:20.339303  1567 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:20.193780  1721 rpcz_store.cc:276] 0709 13:03:18.053760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.517609 (+463849us) service_pool.cc:224] Handling call
0709 13:03:18.530829 (+ 13220us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:18.531207 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.531241 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:18.536153 (+  4912us) log.cc:844] Serialized 52 byte log entry
0709 13:03:19.170933 (+634780us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:19.170986 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.171063 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:20.192895 (+1021832us) raft_consensus.cc:1685] finished
0709 13:03:20.192927 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.193119 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":6110,"thread_start_us":455301,"threads_started":1,"wal-append.queue_time_us":1599799}
I20260709 13:03:20.265803  6929 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.341609  1725 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:20.337056  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.013s	sys 0.018s
I20260709 13:03:20.360646  1576 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:20.360975  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2677) took 2306 ms. Trace:
I20260709 13:03:20.332842  1569 rpcz_store.cc:276] 0709 13:03:18.142573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.142730 (+   157us) service_pool.cc:224] Handling call
0709 13:03:18.164017 (+ 21287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.315953 (+151936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.147634 (+1831681us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:20.147913 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347962112}
W20260709 13:03:20.355370  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2354) took 1639 ms (client timeout 1579 ms). Trace:
I20260709 13:03:20.369726  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:20.369997  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2351) took 1654 ms (client timeout 1580 ms). Trace:
W20260709 13:03:20.390961  1644 rpcz_store.cc:269] 0709 13:03:18.715793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.354963 (+1639170us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.355257 (+   294us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.230832  6888 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.377942  6959 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.378561  6785 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:20.379581  6947 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.380297  6942 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:20.276755  6941 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:20.396279  2374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.558s	user 0.003s	sys 0.028s
I20260709 13:03:20.392004  1724 rpcz_store.cc:276] 0709 13:03:18.054047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.567687 (+513640us) service_pool.cc:224] Handling call
0709 13:03:18.568053 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.568281 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:18.702691 (+134410us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.350626 (+1647935us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.351420 (+   794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.351537 (+   117us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:20.351721 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.351986 (+   265us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.360572 (+  8586us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30808,"prepare.queue_time_us":134163,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":72,"raft.queue_time_us":172168,"raft.run_cpu_time_us":610,"raft.run_wall_time_us":6574,"thread_start_us":1646144,"threads_started":2,"wal-append.queue_time_us":1488320}
W20260709 13:03:20.391378  1646 rpcz_store.cc:269] 0709 13:03:18.715104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.369498 (+1654394us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.369724 (+   226us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.401820  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.014s	sys 0.018s
I20260709 13:03:20.389300  6985 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:20.406739  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2295) took 2029 ms (client timeout 1956 ms). Trace:
W20260709 13:03:20.407745  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2747) took 1827 ms (client timeout 1826 ms). Trace:
I20260709 13:03:20.408188  7013 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:20.409190  7017 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.407745  7012 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:20.409966  6482 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.411859  6713 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:20.413069  7001 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
I20260709 13:03:20.413296  6736 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.414716  6821 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.419167  1716 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:20.420889  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:20.423537  7000 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
I20260709 13:03:20.231657  6778 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.374738  1645 rpcz_store.cc:276] 0709 13:03:18.410688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.443610 (+ 32922us) service_pool.cc:224] Handling call
0709 13:03:18.443896 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.444100 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:18.756098 (+311998us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.336401 (+1580303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.336539 (+   138us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:20.336867 (+   328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.337049 (+   182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.337467 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15815,"prepare.queue_time_us":1079504,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":234426,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":1874676,"threads_started":3,"wal-append.queue_time_us":1478253}
W20260709 13:03:20.436398  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.517755  6941 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:20.445179  7004 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 13:03:20.474840  1569 rpcz_store.cc:269] 0709 13:03:18.580497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.407330 (+1826833us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.407601 (+   271us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.465806  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2350) took 1751 ms (client timeout 1584 ms). Trace:
W20260709 13:03:20.659857  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 300623us
I20260709 13:03:20.451875  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 607) took 2087 ms. Trace:
W20260709 13:03:20.443845  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74467.9us
W20260709 13:03:20.471504  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2348) took 1772 ms (client timeout 1627 ms). Trace:
W20260709 13:03:20.668217  1644 rpcz_store.cc:269] 0709 13:03:18.699666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.471190 (+1771524us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.471406 (+   216us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.453192  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.435467  6888 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.670521  6888 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:20.474551  1576 rpcz_store.cc:269] 0709 13:03:18.377626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.150900 (+1773274us) service_pool.cc:224] Handling call
0709 13:03:20.406480 (+255580us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:20.673588  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2741) took 2112 ms (client timeout 1999 ms). Trace:
I20260709 13:03:20.673902  6918 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:20.674368  1576 rpcz_store.cc:269] 0709 13:03:18.561143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.673269 (+2112126us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.673484 (+   215us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.446003  7011 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.676162  7011 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:20.438467  6942 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.474354  6985 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.450402  6235 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:20.677050  1575 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:20.680536  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.687047  6990 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.066s	user 0.014s	sys 0.009s
W20260709 13:03:20.687806  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.007s	sys 0.008s
W20260709 13:03:20.688303  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.007s	sys 0.008s
W20260709 13:03:20.681529  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2303) took 2113 ms (client timeout 2031 ms). Trace:
W20260709 13:03:20.692118  1576 rpcz_store.cc:269] 0709 13:03:18.568120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.681163 (+2113043us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.681426 (+   263us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.666392  1646 rpcz_store.cc:269] 0709 13:03:18.714818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.465465 (+1750647us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.465700 (+   235us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.428212  6931 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:20.694527  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2746) took 2115 ms (client timeout 1832 ms). Trace:
W20260709 13:03:20.709339  1575 rpcz_store.cc:269] 0709 13:03:18.569889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.391107 (+1821218us) service_pool.cc:224] Handling call
0709 13:03:20.685641 (+294534us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:20.677991  1722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.000s	sys 0.010s
I20260709 13:03:20.443495  7010 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.713295  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2357) took 1993 ms (client timeout 1726 ms). Trace:
W20260709 13:03:20.716022  1644 rpcz_store.cc:269] 0709 13:03:18.716504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.704719 (+1988215us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.709962 (+  5243us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.718290  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2358) took 2001 ms (client timeout 1860 ms). Trace:
W20260709 13:03:20.718811  1646 rpcz_store.cc:269] 0709 13:03:18.716850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.717751 (+2000901us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.717957 (+   206us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.719936  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 786) took 1745 ms (client timeout 1697 ms). Trace:
W20260709 13:03:20.722573  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.723068  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2444) took 3828 ms. Trace:
W20260709 13:03:20.723892  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.676884  7011 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.677752  6942 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:20.725293  1646 rpcz_store.cc:269] 0709 13:03:18.974878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.719680 (+1744802us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.719845 (+   165us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.676316  6778 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.680287  6985 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.677817  6901 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.723596  1716 rpcz_store.cc:276] 0709 13:03:16.885011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.938147 (+1053136us) service_pool.cc:224] Handling call
0709 13:03:17.938531 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.680611 (+2742080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.681548 (+   937us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:20.713612 (+ 32064us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5702811392}
W20260709 13:03:20.731985  1722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.001s	sys 0.010s
W20260709 13:03:20.738283  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 594) took 1610 ms (client timeout 1540 ms). Trace:
W20260709 13:03:20.744918  1569 rpcz_store.cc:269] 0709 13:03:19.122306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.724664 (+1602358us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.732679 (+  8015us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:20.745954  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:20.742161  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2735) took 2179 ms (client timeout 2121 ms). Trace:
I20260709 13:03:20.667009  2374 rpcz_store.cc:276] 0709 13:03:18.364061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.364520 (+   459us) service_pool.cc:224] Handling call
0709 13:03:18.364985 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.365304 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:18.651108 (+285804us) log.cc:844] Serialized 14 byte log entry
0709 13:03:18.837780 (+186672us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.450379 (+1612599us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.450638 (+   259us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:20.450899 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.451290 (+   391us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.451667 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":608443,"raft.queue_time_us":305185,"raft.run_cpu_time_us":487,"raft.run_wall_time_us":48697,"spinlock_wait_cycles":55994496,"thread_start_us":2038588,"threads_started":3,"wal-append.queue_time_us":1492589}
W20260709 13:03:20.743146  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2745) took 2167 ms (client timeout 2124 ms). Trace:
W20260709 13:03:20.748153  1576 rpcz_store.cc:269] 0709 13:03:18.556309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.728217 (+2171908us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.735921 (+  7704us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.438176  6947 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.476418  7012 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.475440  7013 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.437574  6785 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.435812  6959 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:20.748781  1575 rpcz_store.cc:269] 0709 13:03:18.569606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.726528 (+2156922us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.736819 (+ 10291us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.475929  7017 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.715472  6736 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:20.735431  6235 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.749708  6482 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:20.759130  1573 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:20.451678  6730 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.766309  2371 raft_consensus.cc:2368] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:20.749531  6918 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.766364  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 338964us
I20260709 13:03:20.756498  6821 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.776029  6736 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:20.776793  7006 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.777783  7006 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:20.779042  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.779325  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 720) took 2730 ms. Trace:
I20260709 13:03:20.780387  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:20.780630  6940 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.780917  6752 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:20.781459  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:20.782037  7005 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:20.782272  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2366) took 2024 ms (client timeout 2216 ms). Trace:
W20260709 13:03:20.783128  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2736) took 2223 ms (client timeout 1607 ms). Trace:
I20260709 13:03:20.782863  7005 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:20.784559  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2367) took 1939 ms (client timeout 1649 ms). Trace:
W20260709 13:03:20.785046  1650 rpcz_store.cc:269] 0709 13:03:18.758465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.690910 (+1932445us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:20.697710 (+  6800us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:03:20.785868  6959 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.786449  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:20.787627  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2359) took 2070 ms (client timeout 2277 ms). Trace:
W20260709 13:03:20.782886  1646 rpcz_store.cc:269] 0709 13:03:18.758047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.743649 (+1985602us) service_pool.cc:224] Handling call
0709 13:03:20.782172 (+ 38523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.789672  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:20.779362  6736 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:20.788153  1650 rpcz_store.cc:269] 0709 13:03:18.717136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.785990 (+2068854us) service_pool.cc:224] Handling call
0709 13:03:20.787512 (+  1522us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.783902  6893 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:20.805004  6918 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:20.783629  1573 rpcz_store.cc:269] 0709 13:03:18.559752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.975545 (+1415793us) service_pool.cc:224] Handling call
0709 13:03:20.782984 (+807439us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:20.791880  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2673) took 2738 ms (client timeout 1687 ms). Trace:
I20260709 13:03:20.800294  6235 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.763595  6482 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:20.782321  6752 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:20.793538  2344 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.005s	sys 0.005s
I20260709 13:03:20.769599  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:20.821022  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:20.823221  6939 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.823738  7012 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.779954  1718 rpcz_store.cc:276] 0709 13:03:18.048566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.050949 (+  2383us) service_pool.cc:224] Handling call
0709 13:03:18.051347 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.230640 (+179293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.230698 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:18.230782 (+    84us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:18.230829 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.778755 (+2547926us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54003dc7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:20.778856 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.779120 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411648256}
W20260709 13:03:20.808835  1720 rpcz_store.cc:269] 0709 13:03:18.052947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.200881 (+147934us) service_pool.cc:224] Handling call
0709 13:03:20.791129 (+2590248us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400223b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:20.791438 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563200384}
I20260709 13:03:20.888820  1716 raft_consensus.cc:1217] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Deduplicated request from leader. Original: 11.12->[12.13-12.13]   Dedup: 12.13->[]
I20260709 13:03:20.824227  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:20.908802  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 100227us
I20260709 13:03:20.886477  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:20.910939  1716 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:03:20.876796  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:20.924590  2344 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.010s	sys 0.010s
I20260709 13:03:20.932070  1569 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
W20260709 13:03:20.932691  1576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 597) took 1809 ms (client timeout 1770 ms). Trace:
I20260709 13:03:20.885649  6764 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:20.933384  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 596) took 1810 ms (client timeout 1697 ms). Trace:
W20260709 13:03:20.934000  1569 rpcz_store.cc:269] 0709 13:03:19.123089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.755094 (+1632005us) service_pool.cc:224] Handling call
0709 13:03:20.933257 (+178163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.857916  7010 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.935626  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:20.933416  1576 rpcz_store.cc:269] 0709 13:03:19.123416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.764517 (+1641101us) service_pool.cc:224] Handling call
0709 13:03:20.932574 (+168057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.888142  6235 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:20.942569  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2445) took 2802 ms. Trace:
I20260709 13:03:20.943094  1716 rpcz_store.cc:276] 0709 13:03:18.123831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.753234 (+2629403us) service_pool.cc:224] Handling call
0709 13:03:20.856985 (+103751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.919910 (+ 62925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.925733 (+  5823us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.936512  6888 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.933832  6764 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:20.946883  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:20.947959  1571 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.893417  6918 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:20.949034  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2681) took 2649 ms. Trace:
I20260709 13:03:20.956802  1720 rpcz_store.cc:276] 0709 13:03:18.300043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.947184 (+2647141us) service_pool.cc:224] Handling call
0709 13:03:20.947848 (+   664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.948469 (+   621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.948515 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.948602 (+    87us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:20.948658 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.948729 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.948904 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:20.925762  6803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.014s	sys 0.009s
I20260709 13:03:20.965046  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:20.963102  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:20.884697  6959 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.781545  6940 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:20.806175  6893 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:20.963129  1644 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:20.910504  6482 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:20.807904  6730 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:20.982523  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 257914us
W20260709 13:03:20.986550  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64555.7us
I20260709 13:03:20.985152  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:20.977435  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 915829us
W20260709 13:03:20.983472  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2352) took 2263 ms (client timeout 2183 ms). Trace:
I20260709 13:03:20.978586  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 724) took 2871 ms. Trace:
I20260709 13:03:20.988242  1724 rpcz_store.cc:276] 0709 13:03:18.096374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.454956 (+2358582us) service_pool.cc:224] Handling call
0709 13:03:20.455578 (+   622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.456173 (+   595us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:20.762142 (+305969us) log.cc:844] Serialized 14 byte log entry
0709 13:03:20.967386 (+205244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:20.967459 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:20.967510 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.967600 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:20.967788 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":521597,"raft.queue_time_us":11921,"thread_start_us":135434,"threads_started":2}
W20260709 13:03:20.987766  1644 rpcz_store.cc:269] 0709 13:03:18.715355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.739727 (+2024372us) service_pool.cc:224] Handling call
0709 13:03:20.979014 (+239287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:20.806291  6903 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.999756  6893 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.001489  7005 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.003224  1718 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:21.006378  1569 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.973508  7006 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:21.008319  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2360) took 2286 ms (client timeout 2887 ms). Trace:
W20260709 13:03:21.009407  1650 rpcz_store.cc:269] 0709 13:03:18.717445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.926774 (+2209329us) service_pool.cc:224] Handling call
0709 13:03:21.004040 (+ 77266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.009574  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2365) took 2252 ms. Trace:
I20260709 13:03:20.948698  6764 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.011453  1646 rpcz_store.cc:276] 0709 13:03:18.757706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.925845 (+2168139us) service_pool.cc:224] Handling call
0709 13:03:21.009489 (+ 83644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.002614  6893 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.010808  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2447) took 2863 ms. Trace:
I20260709 13:03:21.017503  1573 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:21.017350  1718 rpcz_store.cc:276] 0709 13:03:18.142573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.937021 (+2794448us) service_pool.cc:224] Handling call
0709 13:03:20.985220 (+ 48199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.003893 (+ 18673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.005972 (+  2079us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:21.018486  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2749) took 2179 ms (client timeout 2150 ms). Trace:
W20260709 13:03:21.018976  1573 rpcz_store.cc:269] 0709 13:03:18.838696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.814486 (+1975790us) service_pool.cc:224] Handling call
0709 13:03:21.018110 (+203624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.021819  6720 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:21.022843  6720 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:21.024313  6900 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.052067  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 723) took 2977 ms. Trace:
W20260709 13:03:21.053421  6914 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.852s	user 0.021s	sys 0.006s
I20260709 13:03:21.053774  7015 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:21.055157  7015 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.042373  6947 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:21.058897  1650 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: [6.4-12.7]
I20260709 13:03:21.059859  6940 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.041021  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:21.065073  6720 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:21.062705  6982 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.442s	user 0.013s	sys 0.011s
I20260709 13:03:21.061100  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:21.074676  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2682) took 2767 ms. Trace:
I20260709 13:03:21.076808  6947 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:21.059894  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2015) took 2616 ms. Trace:
I20260709 13:03:21.044633  1575 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:21.064851  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.016819  6959 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.067247  7010 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:20.990854  7004 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:21.075141  6764 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:21.085312  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2455) took 2807 ms (client timeout 1726 ms). Trace:
I20260709 13:03:21.106630  1571 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:21.052681  1721 rpcz_store.cc:276] 0709 13:03:18.074760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.415414 (+2340654us) service_pool.cc:224] Handling call
0709 13:03:20.416183 (+   769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.718382 (+302199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:20.718436 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:20.719780 (+  1344us) log.cc:844] Serialized 26 byte log entry
0709 13:03:20.920006 (+200226us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:20.920153 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.920259 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:21.051573 (+131314us) raft_consensus.cc:1685] finished
0709 13:03:21.051604 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.051793 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20660,"raft.queue_time_us":43840,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":306,"thread_start_us":173553,"threads_started":1,"wal-append.queue_time_us":328473}
I20260709 13:03:21.114060  1650 rpcz_store.cc:276] 0709 13:03:18.443861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.057616 (+2613755us) service_pool.cc:224] Handling call
0709 13:03:21.058479 (+   863us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:21.059387 (+   908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.059779 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.112056  1720 rpcz_store.cc:276] 0709 13:03:18.307336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.963946 (+2656610us) service_pool.cc:224] Handling call
0709 13:03:21.073623 (+109677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.074251 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.074278 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.074329 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:21.074367 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.074433 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.074594 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:21.117238  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.124396  1725 rpcz_store.cc:269] 0709 13:03:18.278089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.367650 (+ 89561us) service_pool.cc:224] Handling call
0709 13:03:21.084887 (+2717237us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b5400222c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:21.085179 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4470817664}
W20260709 13:03:21.113353  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 593) took 1991 ms (client timeout 1777 ms). Trace:
W20260709 13:03:21.128753  1575 rpcz_store.cc:269] 0709 13:03:19.122033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.770276 (+1648243us) service_pool.cc:224] Handling call
0709 13:03:21.113281 (+343005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.038023  7017 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.133124  7025 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.134387  7025 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.122008  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 595) took 1999 ms. Trace:
I20260709 13:03:21.121850  7004 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.136826  7004 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.140615  1576 rpcz_store.cc:276] 0709 13:03:19.122577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.962148 (+1839571us) service_pool.cc:224] Handling call
0709 13:03:21.121932 (+159784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.142414  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.143750  1576 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:21.150127  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2752) took 1115 ms. Trace:
I20260709 13:03:21.146071  6807 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.150964  1571 rpcz_store.cc:276] 0709 13:03:20.011587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.936668 (+925081us) service_pool.cc:224] Handling call
0709 13:03:21.126396 (+189728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.157297  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 727) took 2780 ms. Trace:
I20260709 13:03:21.154299  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:21.157842  1721 rpcz_store.cc:276] 0709 13:03:18.381121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.146893 (+2765772us) service_pool.cc:224] Handling call
0709 13:03:21.147606 (+   713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.148073 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.148122 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.148227 (+   105us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:21.148265 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.148338 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.150739 (+  2401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.159335  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:21.163775  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 123222us
I20260709 13:03:21.164923  1646 raft_consensus.cc:2410] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:03:21.175374  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.176841  6738 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.118593  6903 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.177664  6738 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:21.181388  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.179812  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.184598  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2751) took 1930 ms. Trace:
I20260709 13:03:21.185204  1573 rpcz_store.cc:276] 0709 13:03:19.249820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.124835 (+1875015us) service_pool.cc:224] Handling call
0709 13:03:21.180133 (+ 55298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.188210  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.182977  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2369) took 1013 ms. Trace:
I20260709 13:03:21.189531  1646 rpcz_store.cc:276] 0709 13:03:20.169835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.111727 (+941892us) service_pool.cc:224] Handling call
0709 13:03:21.182877 (+ 71150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.196959  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:21.196331  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:21.200404  1644 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:21.201678  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:21.202076  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.203194  1650 raft_consensus.cc:2368] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:03:21.159444  6720 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:21.204698  6447 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.205163  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.205430  6447 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.210263  6738 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:21.214565  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.221385  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.226292  1575 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:21.223331  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2678) took 3169 ms. Trace:
I20260709 13:03:21.233872  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.208262  6447 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:21.241245  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:21.242362  1646 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:21.246176  6903 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.248741  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:21.249727  1650 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:21.251495  1650 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:21.251396  1573 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:21.236760  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:21.262534  6941 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.551s	user 0.003s	sys 0.014s
I20260709 13:03:21.262862  6720 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:21.265511  1644 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:21.266033  1644 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:21.266314  6325 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:21.267271  6325 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:21.268962  1690 connection.cc:256] RPC call timeout handler was delayed by -1.3321901969999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:03:21.269450  1690 connection.cc:256] RPC call timeout handler was delayed by -1.3072538000000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 13:03:21.269748  1690 connection.cc:256] RPC call timeout handler was delayed by -1.1278901969999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 13:03:21.271454  6325 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:21.271804  1690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.517s (SENT) [suppressed 14 similar messages]
I20260709 13:03:21.273036  1571 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:21.274885  6928 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:21.275789  6928 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:21.227278  1722 rpcz_store.cc:276] 0709 13:03:18.054308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.300948 (+1246640us) service_pool.cc:224] Handling call
0709 13:03:19.910938 (+609990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.911222 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:20.003166 (+ 91944us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.125467 (+1122301us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.156942 (+ 31475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.164178 (+  7236us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:21.190924 (+ 26746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.195888 (+  4964us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.223238 (+ 27350us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4889,"prepare.queue_time_us":223792,"prepare.run_cpu_time_us":146,"prepare.run_wall_time_us":150,"raft.queue_time_us":50224,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":16993,"thread_start_us":940991,"threads_started":2,"wal-append.queue_time_us":675072}
I20260709 13:03:21.276904  6928 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:21.279553  6928 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:21.280499  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2349) took 2552 ms. Trace:
I20260709 13:03:21.280958  1650 rpcz_store.cc:276] 0709 13:03:18.699990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.250520 (+2550530us) service_pool.cc:224] Handling call
0709 13:03:21.250985 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.252067 (+  1082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.252317 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.282853  1650 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:21.283458  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2362) took 2565 ms. Trace:
I20260709 13:03:21.283798  1650 rpcz_store.cc:276] 0709 13:03:18.718023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.281931 (+2563908us) service_pool.cc:224] Handling call
0709 13:03:21.282423 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.283184 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.283399 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.267539  1644 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.293309  2374 raft_consensus.cc:2410] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:21.303886  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.306489  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.300271  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:21.314489  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.265913  6720 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.336388  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:21.336925  6946 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.337396  2379 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:21.338898  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.298877  6982 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:03:21.343262  1646 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
W20260709 13:03:21.354795  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.442914  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.537585  7010 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.557355  7010 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.559170  6720 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:21.559991  6720 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:21.562860  6720 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:21.414469  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.354760  1569 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:03:21.381348  7023 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:03:21.362402  6482 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:21.540746  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 105895us
W20260709 13:03:21.402191  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.362913  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.434341  1716 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:03:21.569646  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2300) took 3179 ms (client timeout 3166 ms). Trace:
W20260709 13:03:21.657637  1569 rpcz_store.cc:269] 0709 13:03:18.390244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.934923 (+2544679us) service_pool.cc:224] Handling call
0709 13:03:21.569565 (+634642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.568713  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.360002  7004 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.410533  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2005) took 3006 ms. Trace:
W20260709 13:03:21.691633  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.658089  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 104808us
I20260709 13:03:21.695329  1573 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:21.684566  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.658474  7025 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:21.512598  6982 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:03:21.566290  7010 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:21.723275  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.579061  1575 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:03:21.706624  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.361778  6902 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.162s	user 0.014s	sys 0.013s
I20260709 13:03:21.735775  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:21.738355  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.693481  1651 rpcz_store.cc:276] 0709 13:03:18.403685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.403832 (+   147us) service_pool.cc:224] Handling call
0709 13:03:18.404211 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.519160 (+114949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.409618 (+2890458us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000f55d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:21.410155 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260947200}
W20260709 13:03:21.638561  2371 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 613) took 3176 ms (client timeout 1517 ms). Trace:
I20260709 13:03:21.735450  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:21.749209  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.016s	sys 0.043s
W20260709 13:03:21.771584  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.017s	sys 0.044s
W20260709 13:03:21.640762  6764 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.006s
I20260709 13:03:21.750612  1644 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.355046  6903 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:21.754192  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.548014  6717 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:21.606359  6937 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.636395  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2674) took 3583 ms. Trace:
W20260709 13:03:21.635931  1648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2016) took 3191 ms (client timeout 1522 ms). Trace:
W20260709 13:03:21.743286  6730 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.738s	user 0.014s	sys 0.000s
I20260709 13:03:21.361495  7015 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.757193  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.426033  6715 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:21.757931  2379 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:21.735052  1576 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:21.741659  6482 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.762761  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.763151  1575 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:21.763680  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.764707  6846 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.765563  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.360996  6752 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:21.770402  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:21.771313  2371 rpcz_store.cc:269] 0709 13:03:18.462239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.462568 (+   329us) service_pool.cc:224] Handling call
0709 13:03:21.638224 (+3175656us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54005c30d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:21.638416 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5035468544}
W20260709 13:03:21.771895  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 586768us
W20260709 13:03:21.785655  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.577504  6785 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:21.808321  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.782611  1719 rpcz_store.cc:276] 0709 13:03:18.053228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.517458 (+464230us) service_pool.cc:224] Handling call
0709 13:03:18.520576 (+  3118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.558769 (+ 38193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.635870 (+3077101us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b540010e5d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:21.636191 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85923712}
W20260709 13:03:21.783201  1648 rpcz_store.cc:269] 0709 13:03:18.444122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.499996 (+ 55874us) service_pool.cc:224] Handling call
0709 13:03:21.635517 (+3135521us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b54001bc6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:21.635703 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29936512}
I20260709 13:03:21.799157  1573 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:03:21.813450  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.779023  6903 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.806039  6785 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.805751  6846 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:21.815809  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 781) took 3056 ms. Trace:
I20260709 13:03:21.816852  1568 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:21.780103  6947 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:21.817343  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.824831  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.817816  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2291) took 3430 ms. Trace:
I20260709 13:03:21.857952  1575 rpcz_store.cc:276] 0709 13:03:18.368264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.762297 (+3394033us) service_pool.cc:224] Handling call
0709 13:03:21.762857 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.798590 (+ 35733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.798863 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21376}
I20260709 13:03:21.834152  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.836443  6986 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:21.837087  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:21.837751  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2013) took 3321 ms. Trace:
I20260709 13:03:21.805773  6715 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:21.842770  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2353) took 3127 ms (client timeout 1807 ms). Trace:
W20260709 13:03:21.844688  1690 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.014s	sys 0.026s
W20260709 13:03:21.862555  1690 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.014s	sys 0.026s
W20260709 13:03:21.862476  1645 rpcz_store.cc:269] 0709 13:03:18.715578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.842493 (+3126915us) service_pool.cc:211] Skipping call since client already timed out
0709 13:03:21.842686 (+   193us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:03:21.862946  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 591252us
I20260709 13:03:21.851080  1573 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:21.854771  1651 rpcz_store.cc:276] 0709 13:03:18.759064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.814784 (+3055720us) service_pool.cc:224] Handling call
0709 13:03:21.815181 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.815447 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.815481 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.815539 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:21.815586 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.815635 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.815736 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.820627  6959 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.869453  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.864099  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2292) took 3487 ms. Trace:
I20260709 13:03:21.812755  7015 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.874260  6876 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:21.864880  6947 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:21.876276  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2487886120000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 13:03:21.877161  1569 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:21.797767  6902 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:21.861605  6903 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:21.913837  6947 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:21.871699  1573 rpcz_store.cc:276] 0709 13:03:18.376696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.832563 (+3455867us) service_pool.cc:224] Handling call
0709 13:03:21.850851 (+ 18288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.863767 (+ 12916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.864031 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.877708  1723 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:21.877388  7015 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:21.860025  1649 rpcz_store.cc:276] 0709 13:03:18.439407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.558709 (+119302us) service_pool.cc:224] Handling call
0709 13:03:18.600302 (+ 41593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.600620 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:19.922356 (+1321736us) log.cc:844] Serialized 14 byte log entry
0709 13:03:21.173312 (+1250956us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b180034d7a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:03:21.537896 (+364584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:21.693015 (+155119us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:21.737301 (+ 44286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.751153 (+ 13852us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:21.761060 (+  9907us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":923197,"prepare.run_cpu_time_us":7985,"prepare.run_wall_time_us":385644,"raft.queue_time_us":38779,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":143933,"spinlock_wait_cycles":40186112,"thread_start_us":1476506,"threads_started":2,"wal-append.queue_time_us":1902031}
I20260709 13:03:21.880415  6986 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:21.882558  6846 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:21.882987  1570 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:03:21.883600  2376 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:21.887511  6942 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.129s	user 0.010s	sys 0.004s
W20260709 13:03:21.895941  2342 connection.cc:256] RPC call timeout handler was delayed by -2.313216648999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:03:21.922864  2342 connection.cc:256] RPC call timeout handler was delayed by -1.7337794709999912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 13:03:21.923275  2342 connection.cc:256] RPC call timeout handler was delayed by -1.6680794709999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260709 13:03:21.911136  6876 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:21.912119  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2487886120000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 13:03:21.921329  1570 pending_rounds.cc:85] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:03:21.877745  2371 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:21.904810  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.910434  1568 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:21.923764  2342 connection.cc:256] RPC call timeout handler was delayed by -1.6680794709999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 13:03:21.949520  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.951086  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2297) took 3564 ms. Trace:
I20260709 13:03:21.872570  6482 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.863001  6717 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:21.968564  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.951658  1568 rpcz_store.cc:276] 0709 13:03:18.386352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.909784 (+3523432us) service_pool.cc:224] Handling call
0709 13:03:21.910218 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.950201 (+ 39983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.950896 (+   695us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.928140  1570 pending_rounds.cc:107] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:03:21.979406  1570 raft_consensus.cc:2889] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:21.983450  1573 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:03:21.970155  1576 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:21.972916  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2352886120000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:03:21.984732  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2181886120000058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:03:21.985101  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2172886120000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 13:03:21.985379  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2136886120000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:03:21.985652  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1749886120000497s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:03:21.986050  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1164886120000119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 13:03:21.986363  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0858886120000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
W20260709 13:03:21.986932  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0858886120000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
W20260709 13:03:21.987073  1718 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.026s	sys 0.075s
W20260709 13:03:21.987713  1718 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.026s	sys 0.075s
I20260709 13:03:21.989625  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2298) took 3597 ms. Trace:
I20260709 13:03:21.990240  1573 rpcz_store.cc:276] 0709 13:03:18.387966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.919161 (+3531195us) service_pool.cc:224] Handling call
0709 13:03:21.983211 (+ 64050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.984024 (+   813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.984316 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:21.992449  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:21.998181  1651 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:21.988664  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2355) took 3261 ms (client timeout 1560 ms). Trace:
W20260709 13:03:22.003088  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0129886120000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 13:03:22.003548  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0129886120000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 13:03:21.999372  1642 rpcz_store.cc:269] 0709 13:03:18.716012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.065856 (+1349844us) service_pool.cc:224] Handling call
0709 13:03:21.969548 (+1903692us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54003fd754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:21.977832 (+  8284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471870208}
I20260709 13:03:21.996649  1569 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:22.005120  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2301) took 3606 ms. Trace:
I20260709 13:03:22.005402  1569 rpcz_store.cc:276] 0709 13:03:18.398374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.915324 (+3516950us) service_pool.cc:224] Handling call
0709 13:03:21.985501 (+ 70177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.004881 (+ 19380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.005070 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.013569  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.001s	sys 0.040s
W20260709 13:03:22.014015  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.001s	sys 0.040s
I20260709 13:03:22.014827  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 784) took 3035 ms. Trace:
I20260709 13:03:22.015234  1651 rpcz_store.cc:276] 0709 13:03:18.974086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.870733 (+2896647us) service_pool.cc:224] Handling call
0709 13:03:21.954013 (+ 83280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.009600 (+ 55587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.009802 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.018237  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.949895  6942 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.018672  1644 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:22.023162  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2361) took 3301 ms. Trace:
W20260709 13:03:22.027520  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49902.4us
I20260709 13:03:22.027319  1644 rpcz_store.cc:276] 0709 13:03:18.717724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.264781 (+2547057us) service_pool.cc:224] Handling call
0709 13:03:21.265272 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.019106 (+753834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.019293 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.033324  1724 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:21.997112  6482 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.038468  1644 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:22.040961  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2368) took 1876 ms. Trace:
I20260709 13:03:22.041509  1568 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:22.041528  1644 rpcz_store.cc:276] 0709 13:03:20.164697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.028802 (+1864105us) service_pool.cc:224] Handling call
0709 13:03:22.037075 (+  8273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.040588 (+  3513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.040887 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.023797  7004 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.045851  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2732) took 3490 ms. Trace:
W20260709 13:03:22.026554  6879 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:03:22.057585  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 789) took 2002 ms. Trace:
I20260709 13:03:22.051514  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2733) took 3495 ms. Trace:
I20260709 13:03:22.055124  1568 rpcz_store.cc:276] 0709 13:03:18.555454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.013037 (+3457583us) service_pool.cc:224] Handling call
0709 13:03:22.041285 (+ 28248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.045629 (+  4344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.045802 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.060778  1569 rpcz_store.cc:276] 0709 13:03:18.555716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.006165 (+3450449us) service_pool.cc:224] Handling call
0709 13:03:22.045206 (+ 39041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.051036 (+  5830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.051080 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.051145 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:22.051201 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.051286 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.051440 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:22.061604  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 176012us
I20260709 13:03:22.048532  1573 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:22.054567  1576 raft_consensus.cc:1217] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Deduplicated request from leader. Original: 8.5->[10.6-12.7]   Dedup: 10.6->[12.7-12.7]
I20260709 13:03:22.062711  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2731) took 3507 ms. Trace:
I20260709 13:03:22.044260  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2273) took 4388 ms. Trace:
I20260709 13:03:22.063067  1576 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:03:22.062104  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 788) took 2046 ms. Trace:
I20260709 13:03:22.060660  1651 rpcz_store.cc:276] 0709 13:03:20.046337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.015970 (+1969633us) service_pool.cc:224] Handling call
0709 13:03:22.042407 (+ 26437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.047625 (+  5218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.047650 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.047694 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:22.047725 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.047778 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.047874 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.063063  1573 rpcz_store.cc:276] 0709 13:03:18.555155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.009552 (+3454397us) service_pool.cc:224] Handling call
0709 13:03:22.029571 (+ 20019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.062472 (+ 32901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.062664 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.064131  1642 rpcz_store.cc:276] 0709 13:03:20.016205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.013118 (+1996913us) service_pool.cc:224] Handling call
0709 13:03:22.036547 (+ 23429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.058318 (+ 21771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.058360 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.061792 (+  3432us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:22.061838 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.061926 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.062041 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.066293  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2302) took 3534 ms. Trace:
I20260709 13:03:22.066646  1576 rpcz_store.cc:276] 0709 13:03:18.532263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.989552 (+3457289us) service_pool.cc:224] Handling call
0709 13:03:22.024404 (+ 34852us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:22.066056 (+ 41652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.066222 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.063203  1570 rpcz_store.cc:276] 0709 13:03:17.647824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.648064 (+   240us) service_pool.cc:224] Handling call
0709 13:03:17.680803 (+ 32739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.786276 (+4105473us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b5400251820. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d70d87 00007fc833d71470 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:03:22.020440 (+234164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.036190 (+ 15750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124202880}
I20260709 13:03:22.071305  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.072018  1569 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:22.075649  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.066089  7015 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.078876  1644 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:22.079742  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.077695  6847 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.085034  1651 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:22.082140  7020 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:22.084394  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2370) took 1739 ms. Trace:
I20260709 13:03:22.086552  1644 rpcz_store.cc:276] 0709 13:03:20.345717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.050391 (+1704674us) service_pool.cc:224] Handling call
0709 13:03:22.059698 (+  9307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.083930 (+ 24232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.084148 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.087195  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:22.079252  7028 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.229s	user 0.001s	sys 0.001s
I20260709 13:03:22.093744  1573 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.077688  6876 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.102581  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.098050  1568 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:22.105052  1573 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.117203  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 618) took 3537 ms. Trace:
I20260709 13:03:22.117887  6764 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:22.120669  7013 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.117702  2378 rpcz_store.cc:276] 0709 13:03:18.577836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.578102 (+   266us) service_pool.cc:224] Handling call
0709 13:03:19.527622 (+949520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.928362 (+400740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.106386 (+2178024us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54005fee50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:22.115262 (+  8876us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917770624}
I20260709 13:03:22.086315  7020 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:22.125562  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:22.112453  7029 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.162s	user 0.000s	sys 0.008s
I20260709 13:03:22.176367  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2279) took 4526 ms. Trace:
I20260709 13:03:22.176815  1572 rpcz_store.cc:276] 0709 13:03:17.649778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.651137 (+  1359us) service_pool.cc:224] Handling call
0709 13:03:17.676325 (+ 25188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:20.330429 (+2654104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.176020 (+1845591us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b540028afd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.176278 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6097669504}
I20260709 13:03:22.177239  7020 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.178350  7020 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.168045  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2734) took 3612 ms. Trace:
I20260709 13:03:22.185333  1571 rpcz_store.cc:276] 0709 13:03:18.555978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.166790 (+3610812us) service_pool.cc:224] Handling call
0709 13:03:22.167284 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.167543 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.167587 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.167668 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:22.167712 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.167785 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.167942 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.097417  6785 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.130087  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:22.191227  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 61267.6us
I20260709 13:03:22.151930  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2454) took 3911 ms. Trace:
I20260709 13:03:22.192133  1716 rpcz_store.cc:276] 0709 13:03:18.240516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.961014 (+2720498us) service_pool.cc:224] Handling call
0709 13:03:21.025584 (+ 64570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.636723 (+611139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.151561 (+514838us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400213754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.151850 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899300096}
I20260709 13:03:22.130927  7031 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:22.193909  7031 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:22.194273  7031 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.194629  7031 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:22.194998  1569 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:22.195642  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2739) took 3635 ms. Trace:
I20260709 13:03:22.131271  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 780) took 3372 ms. Trace:
I20260709 13:03:22.196501  1646 rpcz_store.cc:276] 0709 13:03:18.758809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.374300 (+2615491us) service_pool.cc:224] Handling call
0709 13:03:21.748423 (+374123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.763578 (+ 15155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.763607 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.764153 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.029504 (+265351us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:22.029540 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.029614 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.131047 (+101433us) raft_consensus.cc:1685] finished
0709 13:03:22.131067 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.131219 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3674,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":11206,"thread_start_us":264882,"threads_started":1,"wal-append.queue_time_us":297405}
I20260709 13:03:22.198066  6902 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.198505  6902 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:22.199398  6902 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.062736  6717 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.201208  6715 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.137704  6659 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:22.139142  6289 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:22.202354  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.144475  6714 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 2.017s	user 0.012s	sys 0.012s
I20260709 13:03:22.145004  6482 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.145119  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.206403  1568 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.145617  6764 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:22.148269  6986 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.145231  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:22.153975  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2294) took 3776 ms (client timeout 1744 ms). Trace:
I20260709 13:03:22.156607  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:22.159164  6847 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.125681  6942 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.165513  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:22.171116  7011 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.415s	user 0.009s	sys 0.007s
I20260709 13:03:22.175145  6906 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.195920  1569 rpcz_store.cc:276] 0709 13:03:18.560691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.194398 (+3633707us) service_pool.cc:224] Handling call
0709 13:03:22.194783 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.195399 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.195596 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.206904  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:22.207545  1567 rpcz_store.cc:269] 0709 13:03:18.377341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.139254 (+761913us) service_pool.cc:224] Handling call
0709 13:03:22.153496 (+3014242us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400041ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:22.153685 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982307072}
W20260709 13:03:22.132184  6893 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.031s	user 0.009s	sys 0.005s
I20260709 13:03:22.215250  1574 raft_consensus.cc:2410] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:22.211859  6715 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.215782  7011 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.222357  6715 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.225517  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 1999) took 3893 ms. Trace:
I20260709 13:03:22.225898  1647 rpcz_store.cc:276] 0709 13:03:18.332526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.332713 (+   187us) service_pool.cc:224] Handling call
0709 13:03:18.333038 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:18.432660 (+ 99622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.225089 (+3792429us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b540038aed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.225341 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142699904}
W20260709 13:03:22.227188  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 722) took 4174 ms (client timeout 1540 ms). Trace:
W20260709 13:03:22.227591  1717 rpcz_store.cc:269] 0709 13:03:18.052193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.092966 (+ 40773us) service_pool.cc:224] Handling call
0709 13:03:22.226884 (+4133918us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b540004ef54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:22.227021 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4431001344}
I20260709 13:03:22.228482  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.230010  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.231338  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.233224  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 782) took 3260 ms. Trace:
I20260709 13:03:22.232913  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.234236  1574 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:22.233749  1648 rpcz_store.cc:276] 0709 13:03:18.973154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.814949 (+2841795us) service_pool.cc:224] Handling call
0709 13:03:21.815253 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.815682 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.815708 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.816238 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.059421 (+243183us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:22.059470 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.059544 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.232851 (+173307us) raft_consensus.cc:1685] finished
0709 13:03:22.232870 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.232994 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4729,"spinlock_wait_cycles":20441984,"thread_start_us":242670,"threads_started":1,"wal-append.queue_time_us":285717}
I20260709 13:03:22.237703  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:22.240525  6447 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:22.241236  6447 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.241593  6951 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:22.242233  6951 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:22.242755  6951 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.243414  6447 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.244616  6893 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:22.245325  6893 consensus_queue.cc:1243] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:22.244200  6730 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:03:22.247421  6730 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.247759  6730 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:22.238742  6900 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:03:22.248582  6730 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.249620  6659 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.249925  6730 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.250207  6659 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:22.252095  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.251556  6659 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.253376  6447 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:22.254902  6447 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:22.256839  6447 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.263943  6447 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:22.263002  6893 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:22.231235  6815 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.265733  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.266474  6482 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:22.279295  6902 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.214219  6714 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.306644  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.307529  1572 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.307876  6714 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.308440  6815 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:22.273519  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.273608  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.309803  1568 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:22.313161  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.313443  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.313748  1568 raft_consensus.cc:2376] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14. Re-sending same reply.
I20260709 13:03:22.313700  1647 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:22.314608  1646 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:03:22.314955  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.315264  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:22.315688  1568 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:22.316275  1648 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16.
I20260709 13:03:22.314965  1646 pending_rounds.cc:85] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 10
I20260709 13:03:22.316920  1646 pending_rounds.cc:107] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:03:22.317160  1646 raft_consensus.cc:2889] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:22.318979  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.271651  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2000) took 3932 ms. Trace:
I20260709 13:03:22.264173  1573 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
W20260709 13:03:22.279523  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65376.8us
I20260709 13:03:22.280433  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.262342  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:22.224164  7011 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.284628  6715 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:22.286497  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.289258  6903 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.290071  6924 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.276644  6879 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.293874  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.294476  1570 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.281209  2377 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:22.335217  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:22.298642  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 61247us
I20260709 13:03:22.300089  1574 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:22.267530  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 725) took 3909 ms. Trace:
I20260709 13:03:22.337266  1724 rpcz_store.cc:276] 0709 13:03:18.357731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.007605 (+2649874us) service_pool.cc:224] Handling call
0709 13:03:21.073202 (+ 65597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.033831 (+960629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.267191 (+233360us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b540010dbd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.267458 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2203694464}
I20260709 13:03:22.304229  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 620) took 1184 ms. Trace:
I20260709 13:03:22.338438  2374 rpcz_store.cc:276] 0709 13:03:21.161329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.174742 (+ 13413us) service_pool.cc:224] Handling call
0709 13:03:22.303980 (+1129238us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54005e2754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:22.304134 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253925120}
I20260709 13:03:22.304425  6986 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:22.304765  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 246563us
I20260709 13:03:22.308676  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.308861  1571 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:22.283774   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:22.313796  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:22.321077  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58801.6us
I20260709 13:03:22.319635  1568 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:22.326270  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:22.303853  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97096.1us
I20260709 13:03:22.339768  1572 raft_consensus.cc:2376] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13. Re-sending same reply.
I20260709 13:03:22.333995  6846 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:22.320250  1643 rpcz_store.cc:276] 0709 13:03:18.339144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.339405 (+   261us) service_pool.cc:224] Handling call
0709 13:03:18.340256 (+   851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.016378 (+2676122us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:21.257850 (+241472us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.025989 (+768139us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.269907 (+243918us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b18003ac300. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:22.270094 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.270149 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:22.270198 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.271181 (+   983us) spinlock_profiling.cc:243] Waited 2.62 s on lock 0x7b54000f5ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.271228 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.271433 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238029,"prepare.queue_time_us":713550,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":3804,"raft.queue_time_us":261735,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":4800,"spinlock_wait_cycles":6032244224,"thread_start_us":917924,"threads_started":2,"wal-append.queue_time_us":709537}
I20260709 13:03:22.343353  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.343823  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.343892  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2296) took 3952 ms. Trace:
I20260709 13:03:22.343981  1572 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:03:22.344664  1691 connection.cc:256] RPC call timeout handler was delayed by -2.7793099549999738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:22.335127  1567 raft_consensus.cc:2376] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 15. Re-sending same reply.
I20260709 13:03:22.348227  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.349500  1573 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.352830  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.355608  1646 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:22.332991  6942 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:22.358450  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.358592  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.359143  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.325640  6764 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:22.345180  1575 rpcz_store.cc:276] 0709 13:03:18.386032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.858921 (+3472889us) service_pool.cc:224] Handling call
0709 13:03:21.976280 (+117359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.001347 (+ 25067us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:22.104079 (+102732us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.333707 (+229628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.333776 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:22.333841 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.337681 (+  3840us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.337883 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":104738,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":1344,"thread_start_us":212826,"threads_started":2}
I20260709 13:03:22.343367  6846 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.361100  7025 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:22.361835  7025 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.364565  7030 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:22.368706  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 785) took 3394 ms. Trace:
I20260709 13:03:22.368840  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.369395  7011 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.369570  1720 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:22.369985  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.370195  7011 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.370896  1716 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:22.370937  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.369057  1717 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:22.371562  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.371109  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.348968  6879 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:22.368187  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.382558  1574 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.373255  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.372813  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.369110  1649 rpcz_store.cc:276] 0709 13:03:18.974586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.977556 (+3002970us) service_pool.cc:224] Handling call
0709 13:03:22.015019 (+ 37463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.029810 (+ 14791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.029843 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.044568 (+ 14725us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.213437 (+168869us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:22.213493 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.213679 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.368444 (+154765us) raft_consensus.cc:1685] finished
0709 13:03:22.368469 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.368643 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214877,"raft.queue_time_us":4943,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":18886,"thread_start_us":131898,"threads_started":1,"wal-append.queue_time_us":240120}
I20260709 13:03:22.373602  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.372267  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.397493  1572 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.345218  6986 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.393787  1719 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:22.396061  1570 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:22.398393  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.373231  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.372427  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.399963  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:22.400269  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43707.3us
I20260709 13:03:22.398203  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2737) took 3799 ms. Trace:
I20260709 13:03:22.401475  1567 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:22.403584  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:22.403759  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51234.8us
I20260709 13:03:22.404161  1569 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:22.401928  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:22.348482  6900 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:22.373488  6831 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.579s	user 0.007s	sys 0.023s
I20260709 13:03:22.408290  1568 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:22.378400  6942 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.425382  6942 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.412497  1717 raft_consensus.cc:2376] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 10. Re-sending same reply.
I20260709 13:03:22.373256  6955 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:03:22.403506  6986 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.426371  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.416838  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.428304  1572 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.429136  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.429615  1572 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.420116  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 726) took 4050 ms. Trace:
I20260709 13:03:22.430595  1718 rpcz_store.cc:276] 0709 13:03:18.369578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.101630 (+2732052us) service_pool.cc:224] Handling call
0709 13:03:21.199153 (+ 97523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.988159 (+789006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.988209 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.016521 (+ 28312us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.291761 (+275240us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:22.291920 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.292046 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.419889 (+127843us) raft_consensus.cc:1685] finished
0709 13:03:22.419908 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.420057 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36313,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":18513,"thread_start_us":982983,"threads_started":2,"wal-append.queue_time_us":317446}
I20260709 13:03:22.411659  6923 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.432375  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.432590  6923 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.433053  6923 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.433568  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.434326  1718 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:22.434170  6847 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.435292  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.436420  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.437137  1718 raft_consensus.cc:2410] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:22.437896  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.438521  1718 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:22.439054  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.439190  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:22.421389  7043 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.112s	user 0.004s	sys 0.000s
I20260709 13:03:22.422626  6815 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:22.408540  6831 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.688s	user 0.007s	sys 0.027s
I20260709 13:03:22.446118  6919 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:22.447489  1574 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:22.448185  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2740) took 3887 ms. Trace:
I20260709 13:03:22.448807  6715 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.448721  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.449359  6715 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:22.449784  1649 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:22.450649  6715 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.453169  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.453361  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 783) took 3479 ms. Trace:
I20260709 13:03:22.453722  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.454689  1719 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:22.453770  1645 rpcz_store.cc:276] 0709 13:03:18.973767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.869489 (+2895722us) service_pool.cc:224] Handling call
0709 13:03:21.885056 (+ 15567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.920227 (+ 35171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.920274 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.993339 (+ 73065us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.292327 (+298988us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:22.292398 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.292478 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.453123 (+160645us) raft_consensus.cc:1685] finished
0709 13:03:22.453146 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.453299 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":760,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":261216,"threads_started":1,"wal-append.queue_time_us":338039}
W20260709 13:03:22.424458  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 2679) took 4369 ms (client timeout 2121 ms). Trace:
I20260709 13:03:22.423925  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2742) took 3855 ms. Trace:
W20260709 13:03:22.455760  1723 rpcz_store.cc:269] 0709 13:03:18.055298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.385818 (+330520us) service_pool.cc:224] Handling call
0709 13:03:22.424192 (+4038374us) spinlock_profiling.cc:243] Waited 3.01 s on lock 0x7b54003abe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:22.424398 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6922934016}
I20260709 13:03:22.456524  1567 rpcz_store.cc:276] 0709 13:03:18.568565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.405076 (+3836511us) service_pool.cc:224] Handling call
0709 13:03:22.423301 (+ 18225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.423602 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.423625 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.423665 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:22.423693 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.423747 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.423842 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.457114  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.457767  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.458259  1723 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:22.458678  7025 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.459277  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.459498  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:22.424923  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.425010  6764 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.459858  1723 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:22.421077  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.439253  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2738) took 3878 ms. Trace:
I20260709 13:03:22.424700  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.460552  1571 rpcz_store.cc:276] 0709 13:03:18.560460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.186685 (+3626225us) service_pool.cc:224] Handling call
0709 13:03:22.187089 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.340313 (+153224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.439041 (+ 98728us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003101d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.439200 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279635328}
I20260709 13:03:22.461550  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.461661  6764 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.462024  7031 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:22.462610  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.462998  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.427583  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.463747  1717 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:22.446440  6937 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:22.461709  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.460103  1568 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:22.423316  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 55093.7us
I20260709 13:03:22.422523  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:22.381902  7011 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.455888  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.354617  6924 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.464118  7031 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.568946  1716 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
W20260709 13:03:22.465521  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.401829  1576 rpcz_store.cc:276] 0709 13:03:18.560133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.186335 (+3626202us) service_pool.cc:224] Handling call
0709 13:03:22.186778 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.187011 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:22.187325 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.359626 (+172301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.359678 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:22.359725 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.359787 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.359908 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":684,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":7423,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":164550,"threads_started":1}
I20260709 13:03:22.502954  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2364) took 3784 ms. Trace:
I20260709 13:03:22.452059  6715 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.579924  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.580953  1568 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.582259  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.583870  1568 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:22.458323  1567 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:22.474825  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.575800  1650 rpcz_store.cc:276] 0709 13:03:18.718617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.285329 (+2566712us) service_pool.cc:224] Handling call
0709 13:03:21.755744 (+470415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:21.811948 (+ 56204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:21.812001 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:21.873308 (+ 61307us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.280960 (+407652us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:22.281014 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.281159 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.502740 (+221581us) raft_consensus.cc:1685] finished
0709 13:03:22.502758 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.502895 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10717,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":33219,"thread_start_us":290855,"threads_started":1,"wal-append.queue_time_us":575643}
I20260709 13:03:22.466044  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.468780  1644 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:22.522956  1572 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:22.572268  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 149812us
W20260709 13:03:22.555614  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.552981  1573 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:22.569713  1472 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 116) took 61773 ms (client timeout 15000 ms). Trace:
I20260709 13:03:22.579208  7020 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:22.579661  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.522449  6659 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.470760  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.575350  1718 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:22.496543  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.448527  1574 rpcz_store.cc:276] 0709 13:03:18.560918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.404953 (+3844035us) service_pool.cc:224] Handling call
0709 13:03:22.447295 (+ 42342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.447920 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.448130 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.514786  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:22.585839  1721 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:03:22.579180  1719 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:22.514108  6846 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.572882  6937 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:22.602041  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.540s (SENT)
I20260709 13:03:22.590706  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.602294  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.602679  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.602846  1571 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.603134  1572 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:22.603240  1573 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:22.459877  6879 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:22.604530  1569 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:22.473553  6903 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.614476  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.615586  1568 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:22.616592  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.617862  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.618971  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.620198  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.621197  6937 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:22.607517  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.623804  6845 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.622200  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.611166  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "740ac262ab184918992607d66446b8f3"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44746
I20260709 13:03:22.626355  6951 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.627341  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:22.596647  1472 rpcz_store.cc:269] 0709 13:02:20.809887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.820236 (+ 10349us) service_pool.cc:224] Handling call
0709 13:02:21.058430 (+238194us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:03:22.569433 (+61511003us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:14.620866 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:14.621658 (+   792us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:14.621716 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:03:14.621755 (+    39us) tablet.cc:662] Decoding operations
0709 13:03:14.796454 (+174699us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:14.887341 (+ 90887us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:14.888428 (+  1087us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:15.035382 (+146954us) tablet.cc:701] Row locks acquired
0709 13:03:15.035459 (+    77us) write_op.cc:260] PREPARE: finished
0709 13:03:15.035790 (+   331us) write_op.cc:270] Start()
0709 13:03:15.036018 (+   228us) write_op.cc:276] Timestamp: P: 1783602195035698 usec, L: 0
0709 13:03:15.036045 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:03:15.176875 (+140830us) log.cc:844] Serialized 919 byte log entry
0709 13:03:15.553101 (+376226us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc830c640ac 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b
0709 13:03:15.755156 (+202055us) op_driver.cc:464] REPLICATION: finished
0709 13:03:15.830624 (+ 75468us) write_op.cc:301] APPLY: starting
0709 13:03:15.850793 (+ 20169us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:15.886121 (+ 35328us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:15.886164 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:15.889798 (+  3634us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:15.890017 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:03:15.890077 (+    60us) write_op.cc:312] APPLY: finished
0709 13:03:15.890297 (+   220us) log.cc:844] Serialized 33 byte log entry
0709 13:03:15.891306 (+  1009us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:15.891410 (+   104us) write_op.cc:454] Released schema lock
0709 13:03:15.891452 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:15.891889 (+   437us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:15.891920 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3077369,"reported_tablets":23,"spinlock_wait_cycles":74182912,"child_traces":[["op",{"apply.queue_time_us":21055,"apply.run_cpu_time_us":5299,"apply.run_wall_time_us":70771,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12499,"num_ops":2,"prepare.queue_time_us":123055,"prepare.run_cpu_time_us":51496,"prepare.run_wall_time_us":1105332,"raft.queue_time_us":118688,"raft.run_cpu_time_us":3500,"raft.run_wall_time_us":12050,"replication_time_us":718901,"spinlock_wait_cycles":60229504,"thread_start_us":522080,"threads_started":4,"wal-append.queue_time_us":445402,"wal-append.run_cpu_time_us":32451,"wal-append.run_wall_time_us":1257510}]]}
I20260709 13:03:22.621853  6831 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:22.641260  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:22.605614  1575 raft_consensus.cc:2368] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:22.624966  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.642426  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2743) took 4073 ms. Trace:
I20260709 13:03:22.410436  6900 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.639034  1724 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:22.623271  6937 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.629077  1569 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:22.643129  1570 rpcz_store.cc:276] 0709 13:03:18.568989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.409405 (+3840416us) service_pool.cc:224] Handling call
0709 13:03:22.597052 (+187647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.601312 (+  4260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.604169 (+  2857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.604205 (+    36us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:22.604230 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.604272 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.642163 (+ 37891us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b5400041ad0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:22.642344 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92430720}
I20260709 13:03:22.629796  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2753) took 2611 ms. Trace:
I20260709 13:03:22.624212  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.632751  1718 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:03:22.601344  7020 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:22.612614  1574 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:22.636221  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:22.605880  1645 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:22.620452  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:22.635027  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 190063us
I20260709 13:03:22.624393  6845 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.645138  1569 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.620169  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:22.647529  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:22.642475  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 407584us
I20260709 13:03:22.633299  6923 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:22.651467  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.653443  1575 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:22.654820  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.655125  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2744) took 4085 ms. Trace:
I20260709 13:03:22.658136  1567 raft_consensus.cc:2376] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11. Re-sending same reply.
I20260709 13:03:22.631011  6986 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.645932  1568 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:22.645673  1571 rpcz_store.cc:276] 0709 13:03:20.018722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.607117 (+2588395us) service_pool.cc:224] Handling call
0709 13:03:22.629169 (+ 22052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.629410 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.629435 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:22.629505 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:22.629548 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.629603 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.629727 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.640326  6730 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.665516  1572 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:22.657363  1569 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.640879  6831 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:22.655598  1574 rpcz_store.cc:276] 0709 13:03:18.569314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.605406 (+4036092us) service_pool.cc:224] Handling call
0709 13:03:22.612456 (+  7050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.654861 (+ 42405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.655056 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.741449  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2748) took 4036 ms. Trace:
I20260709 13:03:22.742479  1570 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:22.743022  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.743084  1572 rpcz_store.cc:276] 0709 13:03:18.706709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.606294 (+3899585us) service_pool.cc:224] Handling call
0709 13:03:22.665063 (+ 58769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.741152 (+ 76089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.741385 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:22.684723  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.687705  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:22.733863  1720 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:22.746307  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110145us
I20260709 13:03:22.747169  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:22.744784  1718 raft_consensus.cc:2376] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15. Re-sending same reply.
I20260709 13:03:22.689464  1725 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:22.743904  1716 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:22.748202  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:22.742758  1688 leader_election.cc:400] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:22.757369  1688 leader_election.cc:403] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:22.748123  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:22.761250  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.761870  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.653791  7020 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.764418  1571 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:22.766537  1570 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:22.739717  6752 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:03:22.766647  6831 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.771704  6923 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.684764  6935 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:03:22.772842  6924 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.739276  6736 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:03:22.774688  1724 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:22.775228  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.802403  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.776782  1571 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:22.795524  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.778589  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.779973  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:22.791594  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 104043us
I20260709 13:03:22.730377  7031 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:22.794449  1569 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:22.794636  1572 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:22.796999  1568 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:22.799579  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 85494.5us
I20260709 13:03:22.684612  6900 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.687263  6845 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:22.813437  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.813792  1571 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:03:22.814169  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.816059  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44746 (request call id 2750) took 3777 ms. Trace:
I20260709 13:03:22.817431  1569 rpcz_store.cc:276] 0709 13:03:19.044920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.606681 (+3561761us) service_pool.cc:224] Handling call
0709 13:03:22.804807 (+198126us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":215}
I20260709 13:03:22.761747  6951 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.774369  6902 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.821074  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.825142  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.823277  1718 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.823406  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.823309  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:22.828666  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87578.5us
I20260709 13:03:22.826998  1717 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:22.830605  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.829558  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:22.828142  6815 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:22.882611  6815 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:22.883116  6815 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.758420  6730 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.884626  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:22.780007  6942 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:22.886373  6730 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:22.887408  6942 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:22.887732  7050 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:03:22.893280  1722 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:22.896278  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 106606us
I20260709 13:03:22.882966  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.899746  1718 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:22.896062  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.893344  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:22.817723  7030 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:23.024941  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 195475us
I20260709 13:03:22.684837  1465 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:22.945520  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:23.014444  1716 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.988217  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 592) took 3866 ms. Trace:
W20260709 13:03:22.912726  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 259345us
W20260709 13:03:22.916018  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 136189us
W20260709 13:03:23.030557  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.975083  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:22.977514  1723 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:22.793737  6831 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:22.824470  1570 pending_rounds.cc:85] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 10
I20260709 13:03:23.033842  1570 pending_rounds.cc:107] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:03:23.034504  1570 raft_consensus.cc:2889] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:22.932637  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:22.854897  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.902271  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:23.035591  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.035921  1720 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:22.916647  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 84427.1us
I20260709 13:03:22.799386  6846 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:23.036299  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:22.910880  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:23.031248  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:23.030493  1573 rpcz_store.cc:276] 0709 13:03:19.121726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.607017 (+3485291us) service_pool.cc:224] Handling call
0709 13:03:22.612534 (+  5517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:22.612917 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:22.612943 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:22.613451 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:03:22.719451 (+106000us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:22.811636 (+ 92185us) log.cc:844] Serialized 14 byte log entry
0709 13:03:22.812809 (+  1173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.813069 (+   260us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:22.985414 (+172345us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:22.987792 (+  2378us) raft_consensus.cc:1685] finished
0709 13:03:22.987918 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:22.988130 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":176613,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":10858,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":6661,"spinlock_wait_cycles":3602816,"thread_start_us":171509,"threads_started":2,"wal-append.queue_time_us":219314}
W20260709 13:03:23.043227  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.915233  6764 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:22.943212  6924 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:23.044559  7030 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:22.951967  6922 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:22.817600  6986 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.045872  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:23.046288  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:23.047812  1718 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:22.825397  6902 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.048509  2342 leader_election.cc:400] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16.
I20260709 13:03:22.992491  6894 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.982187  6903 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:23.057137  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:23.053857  2342 leader_election.cc:403] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:23.049796  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:22.975714  7020 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.068920  1720 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:23.041402  6923 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:23.044998  6924 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:23.063506  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:23.063992  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 89021.1us
I20260709 13:03:23.046650  6659 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:23.042929  6846 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.090291  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:23.054219  1690 leader_election.cc:400] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:03:23.092926  1643 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:23.053030  6752 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:23.054978  6902 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.054811  6894 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.105672  6894 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.106805  6894 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:23.108944  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64770.2us
I20260709 13:03:23.056725  6903 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.102845  7057 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.103555  7020 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:23.103902  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74182.7us
I20260709 13:03:23.104518  6736 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:23.102159  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58681.8us
I20260709 13:03:23.085764  6923 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.111598  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:23.111161  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:23.121414  1716 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:23.089270  6924 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.126755  1718 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.128018  1717 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:23.128019  1716 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:23.098520  6846 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.134210  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:23.128736  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.084971  6922 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.097378  6659 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.135016  1719 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:23.132916  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:23.140022  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:23.143931  1717 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:23.143294  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:23.123961  6906 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:23.148931  6906 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.149298  7055 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:03:23.151693  6906 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:23.128481  6951 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:23.141880  6659 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.165743  6951 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.168041  6951 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:23.129027  6923 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.064886  7030 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.160465  1717 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:23.182516  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:23.183246  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:23.163534  6235 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:23.145826  1465 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 13, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.187137  7011 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:23.186765  1465 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 12, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.187707  7011 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:23.188673  7011 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.192842  7063 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.193614  7063 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:23.194924  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.198410  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.199093  1569 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.200842  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.142292  6447 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:03:23.204464  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:23.204923  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:23.146674  7047 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:23.205942  7047 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:23.206362  7047 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.206852  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:23.214069  6937 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
I20260709 13:03:23.214249  6986 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:23.214675  7063 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.216773  6986 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:23.217181  6986 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.218015  6986 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.219496  6986 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:23.146119  6736 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.220989  6736 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.169037  1720 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:23.153445  6815 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.225226  1724 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:23.225414  1720 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:23.174476  6752 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:23.226421  6815 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:23.226616  6752 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:23.227016  6752 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.147737  7020 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:23.228271  7020 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.228685  7020 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.231089  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:23.260205  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 168915us
W20260709 13:03:23.260874  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97690.9us
I20260709 13:03:23.262531  1721 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:23.264837  1575 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:23.275002  1718 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:23.277756  1539 leader_election.cc:400] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
I20260709 13:03:23.281180  1574 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:23.281705  1539 leader_election.cc:403] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:23.278810  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:23.287032  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:23.311029  1569 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:23.175205  7025 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:03:23.315006  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.267122  6845 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:23.297165  7064 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:23.348757  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:23.174086  6903 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:23.365036  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.367865  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.371106  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.374320  1721 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:23.316956  6764 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.386217  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:23.325817  7013 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:23.312461  6935 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:03:23.319453  6845 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:23.407594  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:23.411090   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:23.417429  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:23.446033  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:23.409004  6845 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.447971  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:23.317183  6922 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.384157  6715 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:03:23.385274  6764 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:23.468858  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.424108  6903 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.438225  7013 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:23.486068  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:23.487146  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:23.488314  7025 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:23.499271  6659 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.500551  6785 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.501063  6785 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:23.501511  6785 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.508597  7013 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.509809  7013 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.488963  6815 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:23.513559  6815 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:23.514598  6815 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.489102  6937 raft_consensus.cc:2764] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 13:03:23.523361  7071 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:23.507704  6846 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.524720  1573 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:23.525741  6846 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:23.530417  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.531802  1719 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:23.512950  7073 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.504107  7075 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.475677  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:23.477382  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 215719us
I20260709 13:03:23.437893  7055 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:23.546506  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 129199us
I20260709 13:03:23.467623  7054 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:23.598485  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.561285  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.556572  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.697s (SENT)
I20260709 13:03:23.609220  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:23.611675  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:23.548017  6831 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.002s
W20260709 13:03:23.618194  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 142869us
I20260709 13:03:23.625578  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:23.611714  7054 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.544289  6923 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.534212  7025 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:03:23.623021  7057 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16.
I20260709 13:03:23.635699  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:23.624301  6736 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:23.642421  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 293759us
W20260709 13:03:23.642747  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.650182  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.663000  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 106507us
I20260709 13:03:23.632841  7064 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.659065  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:23.628909  1722 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:03:23.672528  6923 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.673569  6858 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:23.508482  1465 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:23.676558  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 50889.2us
W20260709 13:03:23.678628  6894 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.571s	user 0.015s	sys 0.007s
I20260709 13:03:23.678380  7064 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.673784  6923 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:23.681068  6923 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:23.681588  6923 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.683465  7080 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:23.684886  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.716976  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58030.6us
I20260709 13:03:23.719238  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:23.723476  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 114441us
I20260709 13:03:23.656522  7054 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.667582  6986 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:23.656895  6752 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.747295  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:23.740985  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:23.739152  7078 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:23.714035  6894 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:03:23.745548  6906 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:23.747629  7076 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:23.784504  6986 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.784934  7078 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:23.773617  6924 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.006s
W20260709 13:03:23.790448  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 112286us
I20260709 13:03:23.823470  1571 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:23.835209  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.740406  7073 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:03:23.844973  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2475) took 1203 ms. Trace:
I20260709 13:03:23.856876  1722 rpcz_store.cc:276] 0709 13:03:22.641249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.895761 (+254512us) service_pool.cc:224] Handling call
0709 13:03:23.844746 (+948985us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400232154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:23.844905 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409093888}
I20260709 13:03:23.858783  1573 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:23.857838  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 117145us
I20260709 13:03:23.790194  6906 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:23.675985  1465 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 10, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.856226  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.862151  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.859606  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.806107  6986 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.860247  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:23.854417  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.812721  7078 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.794222  7076 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:23.863613  1576 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:23.792447  7055 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:23.829455  6715 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:03:23.867920  1571 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:03:23.840355  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:23.873672  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:23.845170  6764 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:23.876858  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:23.860713  6894 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:23.885854  7083 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:23.888922  7090 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:03:23.870258  6986 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:23.899915  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73488.6us
I20260709 13:03:23.871423  7076 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:23.879967  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:23.821866  6924 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.861683  6846 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11.
W20260709 13:03:23.906096  6902 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.776s	user 0.013s	sys 0.003s
I20260709 13:03:23.924676  6902 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.926074  1725 raft_consensus.cc:2368] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14.
I20260709 13:03:23.925870  6845 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.927098  6902 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:23.883685  6894 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:23.871554  7095 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:23.904680  6935 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.928539  6894 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.929849  6845 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.923862  6785 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.932974  6832 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:23.933727  6832 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:23.905125  1571 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:23.919072  6752 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.920499  1465 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 12, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.931284  1567 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:23.912937  6906 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:23.941932  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62033.4us
I20260709 13:03:23.943691  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:23.947515  1568 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:23.912957  7081 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.948987  7091 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:23.949527  6752 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.950390  6752 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.950466  7091 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:03:23.951192  7047 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.744s	user 0.015s	sys 0.001s
I20260709 13:03:23.951719  7047 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.954224  7047 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:23.936812  7020 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.004s
I20260709 13:03:23.949211  6785 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.955664  7020 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.956111  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:23.957765  2377 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:23.949278  6906 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.958753  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:23.959161  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:23.958376  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:23.951228  7091 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:23.963753  7091 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.966548  1465 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:23.949296  7081 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.964208  6924 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:23.958811  7055 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.960525  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:23.929934  6894 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:23.974473  6923 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.974529  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:23.976819  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:23.977250  2374 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:23.978365  7100 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.978955  7100 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.979316  7076 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:23.972967  7091 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.982071  7100 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:23.981258  1465 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:23.987722  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:23.973621  6832 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:23.990525  7100 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.984211  1465 catalog_manager.cc:5697] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 12, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:23.958272  7020 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:23.998042  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.010349  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.006294  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 145489us
W20260709 13:03:24.027750  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.985234  7081 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.991190  7100 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:24.057893  1725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 737) took 1253 ms (client timeout 1540 ms). Trace:
W20260709 13:03:24.058739  1725 rpcz_store.cc:269] 0709 13:03:22.791165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.800657 (+  9492us) service_pool.cc:224] Handling call
0709 13:03:24.043734 (+1243077us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540004ef54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:24.044056 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2529062144}
I20260709 13:03:23.984849  6923 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:24.046096  1719 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:23.989826  6832 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:24.054716  1645 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:24.053854  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:24.074174  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.081228  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:24.082023  1650 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:24.085377  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.770s (SENT)
W20260709 13:03:24.109565  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.777s (SENT)
I20260709 13:03:24.086994  6986 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:24.114733  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.084100  6924 raft_consensus.cc:2764] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:03:24.117170  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81788.6us
I20260709 13:03:24.033949  7078 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:24.119830  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.121322  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.162272  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:24.163048  1568 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:24.122094  7076 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:24.123932  7067 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.644s	user 0.003s	sys 0.000s
I20260709 13:03:24.126201  7048 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:24.197233  1645 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:24.131194  7011 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:24.134469  7068 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.642s	user 0.000s	sys 0.004s
I20260709 13:03:24.137257  6659 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:24.032227  1465 catalog_manager.cc:5697] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 12, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:24.143988  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.216892  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.151523  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97147.4us
W20260709 13:03:24.219197  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 275715us
W20260709 13:03:24.219645  2342 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.540s (SENT)
I20260709 13:03:24.220184  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:24.220369  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:24.222064  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.166898  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:24.223222  1568 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:24.223472  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.130394  6235 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.202809  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:24.117040  6923 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.224524  1574 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.222412  1572 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:24.202644  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:24.226168  1576 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:24.187268  7063 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:24.233332  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.244830  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:24.196969  7048 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:24.248878  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.250418  1644 raft_consensus.cc:2368] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14.
I20260709 13:03:24.132193  7097 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:03:24.256443  1648 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:03:24.271056  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 165133us
I20260709 13:03:24.233826  6715 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.297919  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:24.303866  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:24.306967  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62240.7us
I20260709 13:03:24.306172  1651 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:24.307979  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:24.315420  1645 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:24.221287  6924 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:03:24.261749  7047 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:24.262416  6922 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:24.238711  6902 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:03:24.321762  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:24.325691  6903 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.807s	user 0.014s	sys 0.000s
I20260709 13:03:24.358546  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.239704  6923 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.360869  6923 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:24.361737  6986 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:24.285559  7080 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 13:03:24.326934  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.328055  6736 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.636s	user 0.013s	sys 0.000s
I20260709 13:03:24.219455  7011 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.239456  6235 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.380465  1650 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:24.319381  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:24.355964  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.356913  1574 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:24.289531  7095 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:03:24.365074  1646 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:24.393738  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2328) took 1974 ms. Trace:
I20260709 13:03:24.394455  1570 rpcz_store.cc:276] 0709 13:03:22.419259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.780963 (+361704us) service_pool.cc:224] Handling call
0709 13:03:22.818223 (+ 37260us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:23.061659 (+243436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.061705 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:23.085544 (+ 23839us) log.cc:844] Serialized 52 byte log entry
0709 13:03:23.315339 (+229795us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:23.315408 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.315517 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:24.123260 (+807743us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:24.392113 (+268853us) raft_consensus.cc:1685] finished
0709 13:03:24.392137 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.393634 (+  1497us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3611,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":63756032,"thread_start_us":193295,"threads_started":1,"wal-append.queue_time_us":304633}
I20260709 13:03:24.397023  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.399545  7011 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:24.372697  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.198262  7054 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:24.381776  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 62503.6us
W20260709 13:03:24.384789  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 155317us
I20260709 13:03:24.269778  6845 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:03:24.263208  6659 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:03:24.391084  2378 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:03:24.364408  6935 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:24.402166  7064 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:24.289281  7083 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.086s
W20260709 13:03:24.365156  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57317.5us
I20260709 13:03:24.309562  6894 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:24.382474  7030 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.004s	sys 0.008s
I20260709 13:03:24.460290  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:24.460798  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:24.401933  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.391849  6879 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:24.451784  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:24.317338  6715 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:24.380913  6736 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:24.453521  1651 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:24.419243  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:24.391026  7075 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.779s	user 0.015s	sys 0.000s
I20260709 13:03:24.471706  1651 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:24.472142  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:24.476781  1648 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:24.467141  6879 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:24.460625  7030 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.004s	sys 0.009s
I20260709 13:03:24.485445  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:24.461959  7064 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:24.500445  2371 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:24.467640  7100 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:24.499878  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:24.505795  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.508014  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:24.481988  7075 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:24.509238  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90097.4us
I20260709 13:03:24.501827  7064 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.490690  6879 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.511475  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:24.500030  6986 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:24.518379  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.493191  6894 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:24.519569  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:24.522066  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:24.508615  6923 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:24.452584  1465 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 10, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:24.520533  7125 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.099s	user 0.003s	sys 0.000s
I20260709 13:03:24.531891  1648 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:24.550908  2377 raft_consensus.cc:2410] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:24.552659  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:24.533244  7107 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:03:24.520804  7078 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:24.518026  7119 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:24.568598  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:24.575235  7108 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.452s	user 0.004s	sys 0.005s
I20260709 13:03:24.543102  6715 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:24.585234  6235 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:24.592536  2376 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:24.569893  7091 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.001s
W20260709 13:03:24.593495  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 71532.4us
I20260709 13:03:24.530453  7115 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:03:24.612239  7075 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:24.612751  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:24.613286  7075 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:24.613999  7115 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.615763  7075 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.618433  7075 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:24.619184  1539 connection.cc:256] RPC call timeout handler was delayed by -3.6098227389999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30110000000000003s for a response
I20260709 13:03:24.623818  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:24.631610  1651 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:24.587173  7119 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.609222  7101 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:24.609293  6922 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.647823  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:24.650585  7048 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:24.651350  1539 connection.cc:256] RPC call timeout handler was delayed by -3.2469245560000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2887s for a response
W20260709 13:03:24.683267  1539 connection.cc:256] RPC call timeout handler was delayed by -2.9661245560000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3199s for a response
W20260709 13:03:24.683734  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1909341269999345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260709 13:03:24.684011  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1846341269999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:03:24.684281  1539 connection.cc:256] RPC call timeout handler was delayed by -2.00823412699998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 13:03:24.684580  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9479341269999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 13:03:24.684847  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9479341269999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260709 13:03:24.685089  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9065341269999294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
W20260709 13:03:24.685334  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6950341269999285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 13:03:24.685585  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6950341269999285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 13:03:24.685920  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6662341269999388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
W20260709 13:03:24.686187  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6581341269999825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
W20260709 13:03:24.686440  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6581341269999825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
W20260709 13:03:24.657521  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 134485us
W20260709 13:03:24.658419  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45850.2us
I20260709 13:03:24.688664  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:24.687486  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:24.686695  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1145341269999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2851s for a response
W20260709 13:03:24.691219  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0794341269999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28900000000000003s for a response
W20260709 13:03:24.691526  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0794341269999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28900000000000003s for a response
I20260709 13:03:24.692958  1645 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:24.695788  1465 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.699004  1465 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:24.709112  6879 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.721323  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.718297  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:24.714798  1651 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:24.730016  2378 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:24.658365  7101 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.643061  7115 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:24.738781  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.643442  7119 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:24.746083  1576 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 13:03:24.731884  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:24.742059  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:24.746876  1576 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:24.745113  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:24.718748  6986 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:24.717341  6951 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:24.750095  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:24.712265  6879 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:24.766949  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.742910  6736 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:24.745287  7048 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:24.755618  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.742220  7054 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:24.753756  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:24.744450  6903 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:03:24.743391  7101 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:24.777992  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.694958  6922 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:24.781483  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.786893  1575 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:24.681387  6894 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:24.782531  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46096.8us
I20260709 13:03:24.751175  6951 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.750164  6986 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:24.776712  6831 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:24.751075  7109 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:24.818010  6831 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:24.775590  7089 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:24.776168  7076 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:24.789954  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:24.789173  6942 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:24.819859  1575 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:24.820245  6942 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:24.820433  7078 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:24.819545  7076 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:24.821653  6942 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:24.807816  6951 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.712483  1465 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 11, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:24.787132  1649 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:24.841220  1574 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:24.809698  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.817732  7109 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:24.793850  7123 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:03:24.791740  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:24.845186  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:24.845917  1567 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:24.802459  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:24.846906  1569 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:24.843189  1648 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:03:24.822587  7076 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.819507  6831 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.813014  7083 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:03:24.844586  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:24.855470  6906 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:24.833076  7129 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.230s	user 0.000s	sys 0.002s
W20260709 13:03:24.869676  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 82046.3us
I20260709 13:03:24.808722  6764 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:24.832537  6986 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:24.876724  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43787.5us
W20260709 13:03:24.877959  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.877070  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:24.865926  6906 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:24.915298  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:24.905565  1650 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:24.822038  7128 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:24.847642  6879 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:24.854439  1465 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 9, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:24.887382  6815 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:24.931000  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:24.896478  7030 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:03:24.907181  6235 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:24.945111  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.945885  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.956882  7109 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:24.961447  7130 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:03:24.963615  6942 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.964046  6906 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:24.975059  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2787) took 1110 ms. Trace:
I20260709 13:03:24.928689  7078 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:24.975710  1572 rpcz_store.cc:276] 0709 13:03:23.837591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.839659 (+  2068us) service_pool.cc:224] Handling call
0709 13:03:24.947760 (+1108101us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54003106d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:24.947968 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826785536}
W20260709 13:03:24.978266  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63053.3us
I20260709 13:03:24.941233  6815 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:24.942159  7081 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.858s	user 0.014s	sys 0.004s
I20260709 13:03:24.984768  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:24.942620  7025 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.988180  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:24.989326  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.948835  6235 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.906494  6764 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:25.007413  7104 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.884s	user 0.007s	sys 0.000s
I20260709 13:03:24.984140  6937 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.008370  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.007200  1465 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 10 to 12, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.008635  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:25.009757  7110 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.881s	user 0.000s	sys 0.004s
I20260709 13:03:25.004719  6235 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.992717  6894 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:24.990011  6815 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.008230  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.034427  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.033483  1643 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:25.051075  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:25.024811  7083 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:24.982826  6942 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.078382  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.061753  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.038194  6937 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:25.089797  7064 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.006s
I20260709 13:03:25.090425  7064 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.091097  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.091804  1647 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:25.095165  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.096391  1647 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:25.096836  1647 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:25.069274  6879 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:25.098222  2376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 628) took 1113 ms. Trace:
I20260709 13:03:24.993588  7081 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.099036  2376 rpcz_store.cc:276] 0709 13:03:23.984849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.987437 (+  2588us) service_pool.cc:224] Handling call
0709 13:03:25.097924 (+1110487us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b540057e4d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:25.098127 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109977984}
I20260709 13:03:25.099997  1644 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:25.102202  7081 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:25.073081  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.102699  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 624) took 1135 ms. Trace:
W20260709 13:03:25.109229  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.277s (SENT)
I20260709 13:03:25.112645  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:25.114440  2378 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:25.118084  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:25.035473  6906 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.001933  7025 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.123608  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.085521  7097 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:25.060964  7134 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260709 13:03:25.091681  1651 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:25.093688  1649 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:03:25.043151  6894 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.098425  1647 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.131728  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.132829  1720 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.119395  1642 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 10.
I20260709 13:03:25.121625  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.057458  1465 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 12, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 12 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:25.124426  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:25.125233  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74264.5us
I20260709 13:03:25.136777  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:25.135548  1723 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:03:25.083151  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:25.064177  6923 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 13:03:25.136425  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.138598  2378 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:25.112955  7119 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.143594  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.144824  7064 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.145697  7119 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:25.147373  7075 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.000s
I20260709 13:03:25.127153  7100 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.148432  7100 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.149369  6942 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.150398  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.151021  7100 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.122484  7128 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:25.151603  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45900.5us
I20260709 13:03:25.152088  7128 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.153576  1647 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:25.153963  6764 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.154543  7128 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.155396  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.156898  7054 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:25.157377  7054 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:25.157768  7054 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.158521  2377 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:25.159696  1721 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:25.160627  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.161226  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.162588  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.163403  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:25.103322  2372 rpcz_store.cc:276] 0709 13:03:23.966858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.969702 (+  2844us) service_pool.cc:224] Handling call
0709 13:03:23.990529 (+ 20827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:23.990835 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:23.991251 (+   416us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.416449 (+425198us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.417004 (+   555us) log.cc:844] Serialized 14 byte log entry
0709 13:03:24.652284 (+235280us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b18003dbf40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:25.008947 (+356663us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:25.101903 (+ 92956us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b18003dbf40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:25.102171 (+   268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.102240 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:25.102308 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.102422 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.102607 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":740658,"prepare.queue_time_us":99682,"prepare.run_cpu_time_us":181,"prepare.run_wall_time_us":8389,"raft.queue_time_us":771699,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":7976,"spinlock_wait_cycles":217670144,"thread_start_us":333164,"threads_started":3,"wal-append.queue_time_us":136431}
I20260709 13:03:25.127853  7097 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:25.180189  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.198437  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.196805  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.202606  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:25.203792  1725 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:25.213897  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:25.162554  6923 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:25.217002  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.218464  1644 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:25.219815  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.220644  1723 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.224145  1719 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:25.226733  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.227669  1643 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:25.164482  7055 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:25.233959  1568 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:25.237407  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.237596  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.215853  6922 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:25.166165  7025 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:25.240902  1575 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:25.241122  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.196159  6879 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:25.245911  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.247923  1572 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:25.246081  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.248955  1716 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
W20260709 13:03:25.249866  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.252523  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54441.8us
I20260709 13:03:25.252696  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.237996  1720 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:25.255395  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.242085  1649 raft_consensus.cc:2376] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16. Re-sending same reply.
W20260709 13:03:25.255858  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.255051  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.257473  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.258796  1643 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:25.257289  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.246774  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:25.259325  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.260090  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.263137  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.263978  1576 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:25.269102  1725 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:03:25.272706  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 70224.5us
W20260709 13:03:25.276947  7120 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.819s	user 0.003s	sys 0.002s
I20260709 13:03:25.260103  7055 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:25.280941  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.281803  1646 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:25.281509  7097 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.282985  7097 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.283385  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.284766  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.285643  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.286041  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.286985  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.287211  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:25.287634  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.787s (SENT)
I20260709 13:03:25.284588  7128 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:25.288288  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.292873  6815 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.289822  7128 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:25.245466  7083 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:25.258579  6894 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:25.294203  7101 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.284641  6951 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.281652  7055 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.296770  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:25.317451  2372 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:25.290169  1574 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:25.282976  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:25.303220  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.937s (SENT)
I20260709 13:03:25.318535  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:25.319231  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.311056  1647 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:25.312623  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.312783  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.313675  2374 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:25.315354  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.302593  7128 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:25.322808  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.327956  1576 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:25.324827  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.325981  1650 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:25.325170  7101 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.327780  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.332854  1646 raft_consensus.cc:2368] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:25.325428  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.330547  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.334216  1716 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:25.334976  1720 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:25.336872  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.338151  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:25.339735  1646 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:25.340502  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.343132  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.343878  1642 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:25.290081  6986 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:25.350419  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.352660  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.364938  1646 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.357971  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.367916  1642 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:25.368289  1718 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:25.358134  7128 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.368965  1722 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:25.370723  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.375113  1643 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 9.
I20260709 13:03:25.375860  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:25.376792  1723 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:25.378675  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.363911  1647 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:25.327216  6879 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:03:25.379668  1723 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:25.383337  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.410676  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.386993  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.389026  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.411885  1647 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:25.398905  1648 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:25.374786  6831 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.383296  1572 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:25.400754  1716 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:25.362991  7083 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:25.408910  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:25.402413  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.417217  1643 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:25.418316  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.419106  1643 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:25.419325  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.419505  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.422060  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.422853  1648 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.425295  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:25.415948  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.415714  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.428179  1569 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.412497  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.430002  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.400995  7128 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.434172  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.423794  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.437418  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.438201  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.438454  1642 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.438926  1647 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:25.440989  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.441821  1722 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.437126  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.443506  1568 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:25.445101  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.446074  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.405869  7089 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:25.447273  1572 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.448671  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.450116  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.398083  7147 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:25.297777  1465 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 12, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 12 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:25.415930  7091 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.471458  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:25.472610  1643 raft_consensus.cc:2410] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:25.473658  1569 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.473917  1642 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:25.476115  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.478384  1647 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:25.477528  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.479113  1648 raft_consensus.cc:2376] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12. Re-sending same reply.
I20260709 13:03:25.468518  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:25.443534  7083 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:25.481078  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.484046  1642 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:25.484560  1645 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.412415  6235 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.486485  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.487800  1569 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.452294  6764 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:25.490688  1722 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:25.491979  1642 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:25.492710  2374 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:25.495340  1642 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:25.498698  6815 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.502159  1648 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:03:25.504949  7083 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.506045  1645 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:25.493260  7142 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.509040  1643 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:25.509433  7142 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.510447  6986 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.511088  1643 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:03:25.511565  1643 pending_rounds.cc:85] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 7
I20260709 13:03:25.511838  1643 pending_rounds.cc:107] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 12.8
W20260709 13:03:25.515431  7145 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:03:25.433231  6831 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.516597  6942 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.517279  6942 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.518625  1724 raft_consensus.cc:2368] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:25.519486  6942 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:25.482337  7091 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.521816  7091 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.519518  1719 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:25.525315  6447 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:25.525998  6447 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.497659  6951 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.531327  7025 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:25.533747  1716 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:25.534194  6235 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.536175  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.537787  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 632) took 1081 ms. Trace:
I20260709 13:03:25.538239  2379 rpcz_store.cc:276] 0709 13:03:24.456533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.456830 (+   297us) service_pool.cc:224] Handling call
0709 13:03:24.457317 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.104870 (+647553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.104928 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:25.105794 (+   866us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.298538 (+192744us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:25.298607 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.298717 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.537538 (+238821us) raft_consensus.cc:1685] finished
0709 13:03:25.537562 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.537693 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":646963,"raft.queue_time_us":649402,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":192132,"threads_started":1,"wal-append.queue_time_us":243655}
I20260709 13:03:25.540987  1718 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:25.512094  1643 raft_consensus.cc:2889] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:25.544451  1642 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:25.545666  1647 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:25.548272  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.546478  7142 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.551174  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:25.553128  2379 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:25.557740  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:25.518620  6831 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.561343  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:25.567655  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.580745  6447 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:25.542856  6715 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:25.581833  6715 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.584470  6906 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.585112  6906 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.586457  1572 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:25.587494  6906 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.588100  7109 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:25.564041  1570 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:25.588855  1465 catalog_manager.cc:5697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:25.592278  7148 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:03:25.592816  7057 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:25.611953  7057 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:25.613402  7054 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.614085  7057 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.615587  1718 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:25.617038  6937 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:25.593538  7109 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.618835  2378 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:25.592664  7091 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.620612  6715 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.605927  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:25.626468  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 176971us
I20260709 13:03:25.606158  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.629362  1717 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.607985  1725 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:25.591351  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.680s (SENT)
I20260709 13:03:25.631243  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:25.602392  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:25.594699  6832 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:03:25.595860  1465 catalog_manager.cc:5697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.620978  7091 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.598169  1723 raft_consensus.cc:2410] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:03:25.619992  7109 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:25.641558  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75720.2us
I20260709 13:03:25.635564  1721 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:25.637821  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:25.655423  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64144.4us
I20260709 13:03:25.657698  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:25.662433  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.665539  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:25.621110  7054 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.644070  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:25.672842  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:25.660246  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:25.662665  1716 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:25.688573  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:25.688707  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 44732us
I20260709 13:03:25.681203  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:25.697654  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:25.713406  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:25.670258  7122 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:25.669610  7081 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:25.678414  7101 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:25.696281  6894 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:25.696480  7128 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:25.713829  6922 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:25.674640  6986 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:03:25.754621  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:25.744719  6736 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:25.757071  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:25.759163  1465 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:25.702024  6923 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:03:25.729246  7101 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.749092  7128 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.752741  6894 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:25.777383  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:25.785470  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:25.796018  1644 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:25.786706  1465 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 12, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.763536  7083 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:25.794936  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:25.799638  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.810153  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:25.761868  7152 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:25.766324  7101 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.772856  6736 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:25.774147  7128 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.773851  6922 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:25.809779  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:25.801465  6937 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:25.822996  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 165752us
I20260709 13:03:25.822894  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:25.836890  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 139267us
I20260709 13:03:25.816756  6447 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:25.827265  6831 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:25.818689  7079 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:25.872157  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:25.827491  6937 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.838610  7100 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:25.834980  6879 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:25.878836  6447 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.913982  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.917685  1571 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:25.914233  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.877s (SENT)
I20260709 13:03:25.892403  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.909466  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.918762  1575 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.904165  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.908665  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.919936  1572 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.920001  1569 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:25.911230  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:25.921170  1574 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:25.890623  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:25.909993  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:25.921752  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.322s (SENT)
I20260709 13:03:25.922088  1567 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.888501  7063 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:25.922364  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:25.926568  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:25.930770  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:25.840749  7151 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:03:25.891338  7025 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:25.940659  7025 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.940701  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:25.941105  7025 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.880739  7142 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:03:25.892599  6937 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:25.902485  7157 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:03:25.892720  7091 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.893759  6831 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:25.891434  7100 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.952962  7100 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.953197  6831 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.928359  6922 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.885095  6894 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.960886  7147 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:25.929535  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:25.963923  7097 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:25.964475  7097 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.964887  7097 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:25.967428  7101 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.969771  7151 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:25.970235  7151 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:25.971113  7151 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.976521  7151 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:25.978874  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49451.6us
I20260709 13:03:25.981981  7128 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:25.984501  7128 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:25.931110  1576 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.926872  1570 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:03:26.002812  7169 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:03:26.003515  7160 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.064s	user 0.003s	sys 0.001s
I20260709 13:03:25.911725  6736 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.005995  6937 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:25.936950  7156 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:25.917060  1465 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:25.925863  7079 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.926131  6447 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.938084  7109 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:25.938522  7165 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:25.993022  7081 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:25.995855  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:25.997081  7100 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:25.994107  1717 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:26.045419  2377 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:25.999191  7101 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:25.927726  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:26.047739  2379 raft_consensus.cc:2376] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
W20260709 13:03:26.012431  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 81749.4us
W20260709 13:03:26.019052  1688 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.557s (SENT)
W20260709 13:03:26.051383  1688 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.575s (SENT)
I20260709 13:03:26.051865  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.034358  1576 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:26.034247  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:26.036206  1567 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:26.038048  1575 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:25.977576  7097 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.043603   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:26.047080  7081 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:26.044905  7079 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:26.034475  6923 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:26.061169  1614 leader_election.cc:400] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:26.096015  1614 leader_election.cc:403] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:03:26.096553  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 47057us
I20260709 13:03:26.062847  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.097941  7101 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.098634  7101 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.099387  7165 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:26.099772  7165 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:26.102542  6923 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:26.097970  1723 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:26.021723  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:26.074240  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.075789  2377 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:26.092689  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.771s (SENT)
I20260709 13:03:26.042378  7109 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:26.045114  6986 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.056859  7025 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:26.108475  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 86893.1us
W20260709 13:03:26.108322  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 351356us
I20260709 13:03:26.111377  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:26.078994  6951 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:26.112924  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:26.125386  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.027s (SENT)
I20260709 13:03:26.109166  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:26.117002  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:26.127431  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:26.133136  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.135430  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.136231  1575 raft_consensus.cc:2376] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 15. Re-sending same reply.
I20260709 13:03:26.140193  1717 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:03:26.107906  7109 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.150704  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.152493  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.153512  1572 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:26.153455  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.153885  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.154673  1571 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 9: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:26.155617  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.156839  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.158507  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.160182  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.160377  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:26.160683  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.161479  1568 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:26.162642  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.162734  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.165125  1572 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:26.165680  1575 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:26.167219  1575 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:26.100606  7165 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.175314  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.130117  7142 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.123644  7089 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.176805  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.198695  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:26.094053  6942 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:03:26.133960  7057 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:26.179020  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.113423  6986 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.193185  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.199770  2378 raft_consensus.cc:2410] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:03:26.194519  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:26.142836  6879 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.196820  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:26.199862  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 102397us
I20260709 13:03:26.199363  1574 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16.
W20260709 13:03:26.179411  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 145288us
I20260709 13:03:26.210480  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.202244  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.204704  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:26.206602  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.897s (SENT)
I20260709 13:03:26.213255  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:26.213497  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.213893  1721 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.213296  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.218652  1725 raft_consensus.cc:2368] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:26.219034  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.221221  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.192871  7142 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.246212  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:26.215408  7158 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:26.204071  7057 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.248246  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:26.235476  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.261044  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:26.225728  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.224786  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.205552  6951 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.178357  6922 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:03:26.257694  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 130399us
I20260709 13:03:26.259383  1721 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:26.216454  7165 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.237839  6831 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:26.222669  7122 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:26.223435  6894 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:03:26.251557  7142 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.262140  6764 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:26.281773  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 172677us
I20260709 13:03:26.264529  6951 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.264730  7057 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.288936  7025 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:26.295365  7166 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:03:26.296267  7100 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:26.200843  7081 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:26.301349  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.221299  7147 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:03:26.303122  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.304286  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.305755  1722 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:26.266081  6736 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.308254  1724 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:03:26.310894  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 88786.9us
I20260709 13:03:26.277274  7179 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:26.219120  6923 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.323361  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.282233  6937 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:26.285262  6831 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:26.328550  1718 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:03:26.328595  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.325428  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.302001  6951 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.284435  6764 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:26.372368  1720 raft_consensus.cc:2376] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14. Re-sending same reply.
I20260709 13:03:26.284711  7178 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:26.396615  6764 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:26.395865  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 75040us
W20260709 13:03:26.389073  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49583.8us
W20260709 13:03:26.389547  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 61072.5us
I20260709 13:03:26.390067  1724 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:26.399444  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 178334us
I20260709 13:03:26.400862  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.306488  7089 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:26.388089  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.111s (SENT)
I20260709 13:03:26.406754  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.369174  7171 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.327746  7170 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.331553  7151 raft_consensus.cc:2764] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:26.413527  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:26.414271  1646 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:26.324095  6447 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.331462  7097 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:26.394881  7179 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:26.397779  7078 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:26.398409  7048 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:26.421933  1645 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:26.424834  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:26.427062  1642 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:26.401248  6922 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.403378  7178 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:26.435503  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.436720  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:26.437568  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.438621  1645 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:26.439051  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.440682  1645 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:26.410882  7147 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:26.446696  2377 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:26.395455  6736 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:03:26.446657  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:26.447063  1644 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:26.447165  2378 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:26.419956  6764 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.411371  7079 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:26.455456  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54702.9us
W20260709 13:03:26.458889  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 59260.8us
I20260709 13:03:26.458441  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.460767  1724 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
W20260709 13:03:26.460949  7180 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:26.460059  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:26.470168  7176 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:03:26.471854  7177 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:03:26.473004  6986 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:26.473678  7185 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:03:26.433050  7190 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:03:26.481799  7179 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.484056  7109 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:26.486454  7109 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:26.487178  1646 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:26.488615  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49633.2us
I20260709 13:03:26.489969  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.491189  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.491957  7170 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.492404  7170 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:26.492828  7170 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.417308  7171 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.496610  7078 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:26.497217  7078 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:26.498412  7171 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:26.498193  7078 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.432523  7048 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:26.459162  6935 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:26.500613  6935 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.501040  6935 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:26.458550  7173 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:03:26.513824  6447 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:26.447654  7083 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:26.514842  7083 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:26.515836  7083 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.413215  7089 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:26.454668  7091 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:03:26.456398  7152 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:26.523020  7178 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.500895  7048 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:26.528399  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.492182  1539 leader_election.cc:400] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:26.536042  1539 leader_election.cc:403] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:26.537078  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45033.8us
I20260709 13:03:26.538044  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:26.539506  1642 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:26.541368  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 83207.1us
W20260709 13:03:26.541770  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81784.3us
I20260709 13:03:26.546037  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:26.546566  1721 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:26.548688  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:26.556396  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:26.556548  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 270251us
I20260709 13:03:26.558276  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:26.559276  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:26.563333  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 289359us
I20260709 13:03:26.530347  7081 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.566989  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:26.573024  1720 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:26.575248  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:26.572189  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:26.539381  7152 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
W20260709 13:03:26.583482  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 3.061s (SENT) [suppressed 79 similar messages]
I20260709 13:03:26.584862  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:26.581216  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:26.590777  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:26.591410  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:26.591442  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 43723.8us
I20260709 13:03:26.593477  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.597455  7181 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:26.598163  7181 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:26.599249  7181 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.561538  7151 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:26.602905  7151 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:26.549713  7185 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:26.604589  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:26.569114  7048 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.562811  7089 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:26.611271  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.578341  6922 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.454696  6831 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:26.624648  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52538.1us
I20260709 13:03:26.620421  6986 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:03:26.565997  6235 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:03:26.624010  7189 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:26.630846  6447 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.631583  7193 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.591284  7097 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:26.714064  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 80873.4us
I20260709 13:03:26.633038  7089 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:26.613807  7171 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.613618  6906 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:26.727162  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 111724us
W20260709 13:03:26.755831  1688 leader_election.cc:336] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.633s (SENT)
I20260709 13:03:26.756327  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.612589  7057 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:26.620687  6715 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.127s
I20260709 13:03:26.617719  7079 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.130s
I20260709 13:03:26.603854  7151 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:26.748818  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:26.723889  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:26.777393  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:26.777972  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54178.8us
I20260709 13:03:26.779286  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:26.746783  7168 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:26.782511  6906 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:26.716001  6922 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.748363  6447 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.791119  7179 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:26.791880  7063 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:26.792567  6447 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:26.792694  7192 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:26.793663  7192 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:26.794057  7192 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:26.794355  7057 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:26.794814  7192 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.796046  7192 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:26.798044  7165 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.525s	user 0.006s	sys 0.004s
I20260709 13:03:26.800750  7170 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.793226  6906 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.806471  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.810917  1567 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.807318  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.807530  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.811846  1576 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:26.811794  1572 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:26.811200  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.813284  1571 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.774827  6923 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.758140  7097 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:26.819885  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 208679us
I20260709 13:03:26.781616  6942 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.828315  1567 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:26.829159  7170 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:26.830186  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 70486us
W20260709 13:03:26.829744  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50540.1us
I20260709 13:03:26.838209  1616 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:26.840453  1616 leader_election.cc:403] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:26.844477  7168 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.860181  1567 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:26.822897  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.791942  7179 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:26.867038  1569 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:26.867236  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44450.9us
I20260709 13:03:26.778617  7089 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:26.810406  7142 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.005s
I20260709 13:03:26.867923  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:26.873015  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.748518  7189 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.871403  1572 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:26.882524  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.883320  1576 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
W20260709 13:03:26.884210  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.785148  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1acf73e529394004842219637cff7cfc"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:26.886085  1720 raft_consensus.cc:606] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:26.894199  7054 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:26.894869  7054 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:26.914283  1722 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:03:26.916445  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59588.1us
W20260709 13:03:26.890167  6879 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.004s
I20260709 13:03:26.872938  7170 raft_consensus.cc:2764] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:03:26.885499  7101 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.000s
W20260709 13:03:26.926959  1688 leader_election.cc:336] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.693s (SENT)
I20260709 13:03:26.927425  1688 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:26.887581  7100 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.889672  7078 raft_consensus.cc:2764] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:26.930540  1571 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:26.867470  6923 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.921352  7171 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:03:26.931912  6879 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.938505  7101 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:26.928390  7185 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:26.889504  7142 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:26.976548  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.938279  6942 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:27.038816  1570 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:26.983726  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 367834us
I20260709 13:03:26.947929  7168 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
W20260709 13:03:26.967900  7196 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:26.922581  7152 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:03:27.056454  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:27.064412  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 154037us
I20260709 13:03:27.065042  1571 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:27.059805  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:26.969512  6922 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.066417  1721 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.987989  6715 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.074966  7179 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.077128  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 121367us
I20260709 13:03:27.080432  1723 raft_consensus.cc:2368] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
W20260709 13:03:27.039089  6951 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.000s
I20260709 13:03:26.940804  7190 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 13:03:26.971747  7188 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:03:27.097236  1572 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:27.125361  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:27.097879  1575 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:27.097910  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:26.956063  7170 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:03:27.094666  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:27.119269  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:27.130944  1716 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:26.971355  6447 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:03:27.075708  7179 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.066671  7183 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.964730  7207 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.170s
I20260709 13:03:27.123937  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:27.090211  6879 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:26.956207  7054 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.134501  1569 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:27.151612  1721 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:26.974226  7097 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.055658  7182 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 13:03:27.156127  1567 raft_consensus.cc:2368] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15.
I20260709 13:03:27.064584  7078 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:03:26.963967  7089 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.037185  7186 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.168s
I20260709 13:03:27.118000  6951 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.142984  7205 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:27.155256  6922 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.165508  6764 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.641s	user 0.007s	sys 0.004s
I20260709 13:03:27.164059  7210 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.166772  7170 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:27.192631  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74033.4us
I20260709 13:03:27.157572  6879 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:27.120784  6715 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.210207  1644 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:03:27.199160  7199 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:03:27.203177  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:27.212904  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:27.230260  1570 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:27.297276  1571 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:27.169008  7216 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:27.185500  7186 consensus_queue.cc:1243] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:27.307605  1719 raft_consensus.cc:2368] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:27.290848  1646 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:27.310878  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 98059.2us
I20260709 13:03:27.211050  7170 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:27.526032  7170 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.527387  7170 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.159144  7183 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:27.300565  6935 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.799s	user 0.011s	sys 0.000s
I20260709 13:03:27.528873  6935 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.529536  1690 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:27.530160  7183 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:27.207784  6922 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.290149  7189 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.289503  7152 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
W20260709 13:03:27.395963  7204 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 13:03:27.413412  1721 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:27.424074  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:27.184249  7185 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.372423  7187 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:27.207562  7205 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.241173  7210 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.475826  7215 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.292s
I20260709 13:03:27.339090  7216 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:27.491770  7192 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.670s	user 0.004s	sys 0.003s
I20260709 13:03:27.494167  7226 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:27.502700  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:27.455700  6894 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.531200  1724 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:27.396461  6879 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:27.592592  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.629s (SENT)
W20260709 13:03:27.593552  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64088.7us
W20260709 13:03:27.596062  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.478305  7142 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:27.608706  1725 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:27.531134  6935 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.618996  1646 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:27.656119  7205 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:27.628110  7171 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:27.621721  7193 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:27.624655  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 122021us
I20260709 13:03:27.664963  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:27.611547  7183 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:27.636232  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:27.636576  7201 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:27.636839  7219 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 13:03:27.636982  7079 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:27.637229  7203 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 13:03:27.637492  6879 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.637590  7224 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.622287  7226 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:27.639936  7216 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:27.633104  7220 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:03:27.632648  7184 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.694981  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:27.695802  1647 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.625062  7230 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.626160  7223 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:27.628266  7091 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:27.653719  7228 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:27.612690  6923 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.665580  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:27.711581  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46110.3us
I20260709 13:03:27.712518  7216 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:27.704224  7223 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:27.714210  7223 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.666533  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:27.715652  7223 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.671077  1720 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:27.715968  7201 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:27.716888  7201 consensus_queue.cc:1243] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:27.664019  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:27.676820  1643 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:27.678738  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:27.626747  6951 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:27.634512  7179 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.721796  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:27.722628  1570 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:27.662636  7193 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:27.677083  7217 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:03:27.663830  6764 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:03:27.681409  7128 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:27.628569  6894 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.706549  7230 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.703123  7184 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.681951  7183 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:27.724502  1567 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:27.725658  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:27.779218  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 143399us
I20260709 13:03:27.726591  1646 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.727927  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:27.780294  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:27.715010  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:27.729751  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.735041  1720 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:27.736003  1574 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:27.725067  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:27.737417  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75280us
I20260709 13:03:27.786948  1569 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:27.740200  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2506) took 1437 ms. Trace:
I20260709 13:03:27.748966  7179 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:27.764737  1648 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:27.766418  6715 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.765702  7226 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.778573  1647 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:27.685781  7079 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.787009  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:27.692998  7224 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:27.925444  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 315193us
I20260709 13:03:27.981496  7055 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:27.911446  7201 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.013381  7055 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:28.014029  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:28.014290  7079 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:28.014775  1716 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 9.
I20260709 13:03:27.899456  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:27.997304  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 282356us
I20260709 13:03:28.017004  7224 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:27.750872  7229 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:03:28.017151  1716 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:03:27.988855  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2412) took 1536 ms. Trace:
I20260709 13:03:27.891772  1575 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:28.018146  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:27.974136  7235 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:03:28.019172  1720 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:27.744359  7228 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:27.953122  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.021109  1642 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:28.022107  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:27.752485  6906 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:28.022661  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2411) took 1571 ms. Trace:
I20260709 13:03:28.023046  1642 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:28.023216  6906 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:28.023357  1644 rpcz_store.cc:276] 0709 13:03:26.451636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.451761 (+   125us) service_pool.cc:224] Handling call
0709 13:03:26.509636 (+ 57875us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.232415 (+722779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.022367 (+789952us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54000b9854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:28.022588 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610484608}
I20260709 13:03:28.024261  6906 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.025568  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:28.026373  1723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2496) took 1866 ms (client timeout 1887 ms). Trace:
I20260709 13:03:28.026510  1721 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:28.026762  1644 raft_consensus.cc:2376] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14. Re-sending same reply.
I20260709 13:03:28.025655  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:28.027832  1716 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:28.024121  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:27.837149  7048 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.215s	user 0.006s	sys 0.006s
I20260709 13:03:28.027904  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:28.029193  1723 rpcz_store.cc:269] 0709 13:03:26.159961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.160126 (+   165us) service_pool.cc:224] Handling call
0709 13:03:28.026134 (+1866008us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400233054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:28.026309 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113216256}
I20260709 13:03:28.029986  2378 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:03:27.917344  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 132102us
I20260709 13:03:28.030509  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.032209  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:27.739426  7091 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.032930  1723 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:28.036232  7091 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:27.945617  7193 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:27.749929  7128 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:28.054311  7128 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.055392  7128 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:28.012425  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 225480us
I20260709 13:03:28.058226  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:28.059043  7054 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.038136  7193 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.040832  7152 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:28.017125  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.060699  7152 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:28.061404  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44354.2us
I20260709 13:03:28.061633  7152 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.062156  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.042912  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 648) took 1458 ms. Trace:
I20260709 13:03:28.062933  6879 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:28.063094  2378 rpcz_store.cc:276] 0709 13:03:26.584384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.584548 (+   164us) service_pool.cc:224] Handling call
0709 13:03:28.042612 (+1458064us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54005c30d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:28.042828 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3311849216}
I20260709 13:03:28.063503  6879 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:28.063666  7089 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:28.062949  7152 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:28.064430  6879 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.067014  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.067967  1648 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 10: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:28.018208  1651 rpcz_store.cc:276] 0709 13:03:26.452058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.452184 (+   126us) service_pool.cc:224] Handling call
0709 13:03:26.460722 (+  8538us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:03:26.541471 (+ 80749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.541499 (+    28us) raft_consensus.cc:1539] Triggering prepare for 5 ops
0709 13:03:26.562492 (+ 20993us) log.cc:844] Serialized 130 byte log entry
0709 13:03:26.817453 (+254961us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:27.029273 (+211820us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.240710 (+211437us) log.cc:844] Serialized 14 byte log entry
0709 13:03:27.312453 (+ 71743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.312536 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:27.988431 (+675895us) raft_consensus.cc:1685] finished
0709 13:03:27.988464 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.988761 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":241624,"prepare.queue_time_us":660416,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":359894,"raft.run_cpu_time_us":479,"raft.run_wall_time_us":50580,"thread_start_us":464216,"threads_started":3,"wal-append.queue_time_us":575384}
W20260709 13:03:27.859665  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.069185  7210 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:27.985821  7183 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:27.871791  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67033.1us
I20260709 13:03:27.793797  1646 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:28.068941  6951 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:28.011729  1719 rpcz_store.cc:276] 0709 13:03:26.302733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.302857 (+   124us) service_pool.cc:224] Handling call
0709 13:03:27.739832 (+1436975us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400224554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:27.740095 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149441792}
W20260709 13:03:28.069949  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.017511  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.047637  2377 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:28.071904  6951 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:28.073607  1643 raft_consensus.cc:2410] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:28.074059  6951 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.074820  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.075139  6951 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:28.076117  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.049963  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:27.804412  7081 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.225s	user 0.014s	sys 0.000s
I20260709 13:03:27.947789  7240 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:28.059360  6942 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:28.080516  6942 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:28.080380  7240 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.081657  6942 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.077373  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.049718  7226 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:28.082873  1646 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:28.084076  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:28.084635  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.084751  1646 raft_consensus.cc:2376] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15. Re-sending same reply.
I20260709 13:03:28.012833  7097 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:28.093314  7097 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:28.094065  7128 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:28.094300  7097 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.094689  7128 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.068953  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.096980  7128 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:28.097460  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.098162  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.103127  1648 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:28.069151  1539 leader_election.cc:400] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:28.105752  1539 leader_election.cc:403] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.045770  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.096118  1642 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:28.107400  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.098271  1651 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:28.100050  1643 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:28.131480  1644 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:28.188508  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.419428  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.360206  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:28.426604  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.412838  1688 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.598s (SENT)
I20260709 13:03:28.435365  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.140631  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:28.431447  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 362203us
I20260709 13:03:28.435660  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.439311  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.434612  1643 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.441704  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.447754  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.456233  1644 raft_consensus.cc:2376] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15. Re-sending same reply.
I20260709 13:03:28.468058  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.475183  1644 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:28.425714  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2767) took 1009 ms. Trace:
I20260709 13:03:28.477686  1724 rpcz_store.cc:276] 0709 13:03:27.416178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.416382 (+   204us) service_pool.cc:224] Handling call
0709 13:03:27.455477 (+ 39095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.623792 (+168315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.425428 (+801636us) spinlock_profiling.cc:243] Waited 75.3 ms on lock 0x7b5400171bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:28.425629 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173283712}
I20260709 13:03:28.398360  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:28.468058  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.026492  7179 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:28.484510  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.491416  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52163.9us
I20260709 13:03:28.494380  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.494573  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.496166  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.492105  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:28.173399  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.492726  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 79902.8us
W20260709 13:03:28.524967  1688 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.608s (SENT)
W20260709 13:03:28.525988  1688 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.704s (SENT)
W20260709 13:03:28.421191  7223 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.705s	user 0.007s	sys 0.004s
I20260709 13:03:28.510558  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.526412  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.109232  7232 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.428117  6235 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.507360  7224 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.507823  6736 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.360623  7128 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.205s
I20260709 13:03:28.500805  7165 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.609747  7165 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.610960  7128 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.427553  6986 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 13:03:28.504210  7181 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.117514  7079 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.520834  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 831) took 1319 ms. Trace:
I20260709 13:03:28.522457  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:28.522756  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.422811  7230 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:28.360857  7151 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:28.526559  1648 raft_consensus.cc:2376] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16. Re-sending same reply.
I20260709 13:03:28.490585  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:28.527205  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 455040us
I20260709 13:03:28.328374  6922 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:28.367091  6764 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:28.491114  7190 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.533008  1651 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:28.505388  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.328199  6715 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.328187  7091 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:28.545898  1643 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:28.499821  7216 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:28.550312  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.568471  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.569782  7185 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.010s	sys 0.023s
W20260709 13:03:28.575013  1723 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.020s	sys 0.026s
W20260709 13:03:28.684816  1723 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.020s	sys 0.027s
W20260709 13:03:28.578470  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 69853.7us
I20260709 13:03:28.512631  7089 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:28.613639  1724 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:03:28.618007  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92003.7us
I20260709 13:03:28.622418  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:28.630738  1647 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:28.633077  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.628412  2378 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 8.
I20260709 13:03:28.626318  1649 rpcz_store.cc:276] 0709 13:03:27.201506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.201629 (+   123us) service_pool.cc:224] Handling call
0709 13:03:27.240454 (+ 38825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.306963 (+ 66509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.307008 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.312568 (+  5560us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.706008 (+393440us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:27.706065 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.706162 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.520519 (+814357us) raft_consensus.cc:1685] finished
0709 13:03:28.520542 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.520750 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58239,"raft.queue_time_us":158596,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":15288,"spinlock_wait_cycles":896384,"thread_start_us":274751,"threads_started":1,"wal-append.queue_time_us":1117210}
W20260709 13:03:28.635069  1576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.007s	sys 0.015s
W20260709 13:03:28.691205  1576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.007s	sys 0.016s
I20260709 13:03:28.641728  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:28.641885  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.691524  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2081) took 1015 ms. Trace:
I20260709 13:03:28.692631  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.693442  1646 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:28.693960  1721 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:28.690201  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.710s (SENT)
W20260709 13:03:28.692111  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50489.5us
I20260709 13:03:28.694856  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.420328  7055 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.691849  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:28.650713  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.696374  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:28.697144  1648 raft_consensus.cc:2368] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16.
I20260709 13:03:28.651638  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:28.625064  7152 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 13:03:28.681504  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.687538  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:28.632557  6879 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:28.697842  1719 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
W20260709 13:03:28.695104  1688 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.712s (SENT)
I20260709 13:03:28.545337  7081 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:03:28.695961  1646 raft_consensus.cc:2368] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:28.697593  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:28.689285  7185 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.011s	sys 0.023s
I20260709 13:03:28.698493  7241 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.705931  7151 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.691939  1644 rpcz_store.cc:276] 0709 13:03:27.676590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.505042 (+828452us) service_pool.cc:224] Handling call
0709 13:03:28.691470 (+186428us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:28.700152  1688 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.786s (SENT)
I20260709 13:03:28.613607  6235 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.710342  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.704097  1616 leader_election.cc:400] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15.
I20260709 13:03:28.652262  7165 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:28.713124  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:28.713721  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:28.629740  7223 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:03:28.713770  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.620962  6736 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.709610  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.729921  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2098) took 1016 ms. Trace:
I20260709 13:03:28.733160  1649 rpcz_store.cc:276] 0709 13:03:27.713045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.691306 (+978261us) service_pool.cc:224] Handling call
0709 13:03:28.729835 (+ 38529us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:28.733819  1688 leader_election.cc:336] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.799s (SENT)
W20260709 13:03:28.734999  1688 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.880s (SENT)
I20260709 13:03:28.735479  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:28.742506  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.614111  7232 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.745170  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:28.746042  1539 leader_election.cc:400] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:28.746382  1539 leader_election.cc:403] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:03:28.747299  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:28.710758  7210 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:28.719866  7251 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:03:28.688201  7190 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.781435  7210 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.642814  7079 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.679088  6764 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:28.764045  1688 leader_election.cc:336] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.890s (SENT)
I20260709 13:03:28.817255  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:28.766930  7170 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.239s	user 0.016s	sys 0.000s
I20260709 13:03:28.711262  7089 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:28.775775  7193 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.002s
I20260709 13:03:28.697319  7250 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:28.751573  7225 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.776921  7241 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.816545  7210 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.777622  1539 leader_election.cc:400] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:28.779018  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:28.843554  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64602.8us
I20260709 13:03:28.845606  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:28.779523  7223 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:28.779759  7081 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.726326  7224 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:28.753094  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.865103  1644 raft_consensus.cc:2376] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15. Re-sending same reply.
I20260709 13:03:28.703614  7185 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:28.787001  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.762849  1651 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:28.812399  1647 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:28.818617  1572 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:28.867288  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2106) took 1151 ms. Trace:
I20260709 13:03:28.867444  1572 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:28.867897  1651 rpcz_store.cc:276] 0709 13:03:27.715489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.434794 (+719305us) service_pool.cc:224] Handling call
0709 13:03:28.867215 (+432421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.821023  1648 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:03:28.868940  1648 pending_rounds.cc:85] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 7
I20260709 13:03:28.869287  1648 pending_rounds.cc:107] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:03:28.869575  1648 raft_consensus.cc:2889] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:28.706660  7192 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:03:28.825688  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:28.877624  1724 raft_consensus.cc:606] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:28.693197  6715 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:28.832952  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68913.2us
W20260709 13:03:28.881045  1688 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.897s (SENT)
W20260709 13:03:28.882277  1688 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.912s (SENT)
I20260709 13:03:28.882783  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.796161  7183 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.832542  7264 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:28.834620  1643 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:03:28.777033  7231 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.856518  7081 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.815461  7228 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:28.835347  7250 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.751816  7165 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
W20260709 13:03:28.866626  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89149.6us
I20260709 13:03:28.924952  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:28.870663  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2109) took 1154 ms. Trace:
I20260709 13:03:28.925997  1647 rpcz_store.cc:276] 0709 13:03:27.716238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.698358 (+982120us) service_pool.cc:224] Handling call
0709 13:03:28.870575 (+172217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:28.927111  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:28.823894  7249 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:28.831845  7181 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.853374  7089 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.697522  7091 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.867228  7101 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:28.881666  1572 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.834797  6942 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.713974  6879 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:28.904218  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.918049  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.932560  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:28.904709  7231 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.941427  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:28.876362  6986 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:03:28.876672  6922 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:03:28.841853  7225 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.912853  7081 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.957672  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:28.972853  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:28.992060  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 146452us
I20260709 13:03:28.875830  7241 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:28.950021  7228 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.953189  7185 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:29.015429  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.055135  2378 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:29.037292  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 98846us
I20260709 13:03:28.950398  7048 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.042191  1647 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:29.042062  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:29.057492  2374 raft_consensus.cc:2410] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:29.018251  1572 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.056172  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:29.090847  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.007627  6715 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:29.125062  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 83697.9us
W20260709 13:03:29.069080  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.949064  7181 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.044366  7205 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.016875  7231 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:29.136844  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.748s (SENT)
I20260709 13:03:29.157044  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:29.158267  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.611s (SENT)
I20260709 13:03:29.137775  2374 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:29.062378  7192 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:29.211926  7192 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:29.213626  7192 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.139701  7190 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.215597  7192 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:29.227010  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87840.4us
W20260709 13:03:29.147244  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3732610200000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 13:03:29.022154  6235 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:28.946390  7249 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:28.945150  7258 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.186s
W20260709 13:03:29.149551  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 130019us
I20260709 13:03:29.139492  7101 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:29.162231  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:29.162842  7267 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:03:29.162967  1572 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:03:29.233290  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:29.239033  1572 pending_rounds.cc:85] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:03:29.239543  1572 pending_rounds.cc:107] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:03:29.239888  1572 raft_consensus.cc:2889] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:29.163821  7253 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:03:29.164711  7256 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
W20260709 13:03:29.244975  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.170327  7248 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:03:29.173794  7285 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:29.175354  7255 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:03:29.191471  6832 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.192852  7207 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:29.193783  6879 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.062160  7228 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.195788  7225 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.199919  7241 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.200788  6764 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:29.201573  7276 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:29.225576  7205 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:29.204324  6951 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.129s	user 0.015s	sys 0.000s
I20260709 13:03:29.149940  7165 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:29.155659  7224 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.061936  7185 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:29.234948  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2796610199999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 13:03:29.530675  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2409610199999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
I20260709 13:03:29.531749  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:29.169265  7260 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:03:29.534196  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:29.200212  7054 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.141s	user 0.012s	sys 0.000s
I20260709 13:03:29.249312  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2818) took 2143 ms. Trace:
W20260709 13:03:29.233278  7201 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.220s	user 0.009s	sys 0.005s
W20260709 13:03:29.246096  7226 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.157s	user 0.007s	sys 0.007s
W20260709 13:03:29.296511  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.246356  6922 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.202s
I20260709 13:03:29.144867  7048 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:29.408056  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 149287us
I20260709 13:03:29.536444  7201 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:29.432993  1647 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:29.245170  7170 raft_consensus.cc:476] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 13:03:29.244932  7151 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.237136  7101 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:29.440523  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2501) took 3225 ms (client timeout 1692 ms). Trace:
I20260709 13:03:29.460114  2374 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:03:29.250671  7122 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:29.449705  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 220295us
I20260709 13:03:29.495122  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2107) took 1779 ms. Trace:
I20260709 13:03:29.576151  1648 rpcz_store.cc:276] 0709 13:03:27.715739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.706762 (+991023us) service_pool.cc:224] Handling call
0709 13:03:28.713193 (+  6431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.870135 (+156942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.494658 (+624523us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400133b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:29.495018 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246720896}
I20260709 13:03:29.491873  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.533790  7185 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.535430  1572 rpcz_store.cc:276] 0709 13:03:27.098055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.126377 (+ 28322us) service_pool.cc:224] Handling call
0709 13:03:27.126724 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.818495 (+1691771us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b5400255e20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d701c0 00007fc833d71231 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:03:29.240572 (+422077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.240902 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44756864}
I20260709 13:03:29.536092  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:29.578606  1650 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:29.538183  1617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.151s	sys 0.196s
I20260709 13:03:29.251283  7216 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.250928  7181 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:29.250888  7078 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:03:29.544704  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.545588  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 766) took 1216 ms. Trace:
I20260709 13:03:29.254415  7207 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:29.528262  7248 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.268s	user 0.000s	sys 0.000s
I20260709 13:03:29.240703  7083 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.406502  7091 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:29.563562  1717 rpcz_store.cc:269] 0709 13:03:26.215045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.215287 (+   242us) service_pool.cc:224] Handling call
0709 13:03:29.440168 (+3224881us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400213c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:29.440412 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114865536}
I20260709 13:03:29.480315  7240 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:29.579435  1617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.151s	sys 0.196s
I20260709 13:03:29.529863  7165 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.478461  7179 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.581389  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.592909  2374 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:29.556161  7055 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:29.584075  2378 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:29.537192  7269 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.585997  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.584223  2379 rpcz_store.cc:276] 0709 13:03:28.329081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.366488 (+ 37407us) service_pool.cc:224] Handling call
0709 13:03:28.470550 (+104062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.470832 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:28.730854 (+260022us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.471162 (+740308us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b1800441e80. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a
0709 13:03:29.544812 (+ 73650us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.545051 (+   239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.545123 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:29.545201 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.545302 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.545509 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72626,"prepare.queue_time_us":335045,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":30082,"raft.queue_time_us":21042,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11985,"spinlock_wait_cycles":74112000,"thread_start_us":307699,"threads_started":2,"wal-append.queue_time_us":283702}
I20260709 13:03:29.589339  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.303210  6832 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.598899  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.583036  7170 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.600515  7274 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.600907  7170 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.601312  7274 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.601620  7207 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.602754  7207 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.589850  7216 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:29.613566  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 92414us
I20260709 13:03:29.560786  7226 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:29.606035  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44417.9us
W20260709 13:03:29.590997  7183 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.692s	user 0.013s	sys 0.001s
I20260709 13:03:29.547416  7224 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.643844  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.574607  7244 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:29.649925  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.654711  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.600558  7165 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.722151  6832 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.709178  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:29.603420  7083 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.723830  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.617610  7122 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:29.725554  7282 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:29.725338  7122 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.726182  7282 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.606693  7201 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:29.714787  7240 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.727509  7240 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.642530  1650 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:29.725340  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.603309  7170 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.587365  7151 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.729700  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:29.616516  7190 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.621287  7179 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:29.741066  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 118590us
W20260709 13:03:29.742458  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 210736us
I20260709 13:03:29.642729  7183 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:29.744822  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:29.749473  1650 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:03:29.752830  1649 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:29.754634  1648 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:29.716905  7281 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:29.764566  7262 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:29.742020  7170 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.765156  7262 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:29.765563  7170 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.766264  7262 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.767627  7262 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.768997  7170 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.773931  7290 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:29.772060  7222 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 2.289s	user 0.003s	sys 0.021s
I20260709 13:03:29.774865  7290 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:29.776516  7170 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:29.776883  1572 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:29.776957  7170 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:29.776139  7290 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:29.778053  7170 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.779491  7170 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.777100  1568 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:03:29.779958  1568 pending_rounds.cc:85] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:03:29.725558  7083 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.780192  1568 pending_rounds.cc:107] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:03:29.780465  1568 raft_consensus.cc:2889] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:29.780898  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.781692  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.781978  1567 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:29.782469  1574 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:29.621624  7269 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.785386  2372 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:29.786237  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:29.786866  1574 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.730103  7284 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:03:29.793754  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.794514  1568 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:29.795423  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:29.797861  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2122) took 1042 ms. Trace:
I20260709 13:03:29.798436  1649 rpcz_store.cc:276] 0709 13:03:28.755006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.733198 (+978192us) service_pool.cc:224] Handling call
0709 13:03:29.747637 (+ 14439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.760718 (+ 13081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.797804 (+ 37086us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:29.799459  7269 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.803885  1573 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:29.718156  7274 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.806998  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.819929  1571 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:29.809672  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.821015  2379 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:29.810328  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.811954  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.814153  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.822532  2374 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:29.822566  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.825544  1574 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
W20260709 13:03:29.818647  1539 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Timed out: RequestConsensusVote RPC to 127.0.107.195:44281 timed out after 1.692s (SENT)
I20260709 13:03:29.816063  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:29.827071  2377 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:29.824308  1568 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:29.824447  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.829173  1567 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:29.825454  1572 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
W20260709 13:03:29.831311  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.640456  7226 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.757845  7179 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.845609  7281 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.849733  7224 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.737401  7244 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.853220  7244 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.854763  7205 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.856886  7048 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.857983  7249 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:29.745529  7215 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.716176  7101 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.843441  7226 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:29.873584  1690 connection.cc:256] RPC call timeout handler was delayed by -1.0475184549999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 13:03:29.892359  7215 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.893615  7184 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:29.882910  7179 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.897606  7151 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:29.901986  7210 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.026s	user 0.013s	sys 0.002s
I20260709 13:03:29.903477  7210 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:29.849817  7294 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.819178  7216 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.918883  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:29.744445  7190 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.834728  7282 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:29.946878  7224 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.947539  7226 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:29.962725  1569 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:29.904745  7294 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.949396  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:29.952785  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:29.955157  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.965011  1573 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:29.966114  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.966905  1573 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:29.968454  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:29.969991  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:29.919919  7101 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.948770  1572 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:29.970229  1573 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:29.963979  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:29.957340  1616 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:29.944963  7216 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.994251  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.015s	sys 0.014s
W20260709 13:03:29.998342  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.016s	sys 0.014s
I20260709 13:03:29.996215  1569 raft_consensus.cc:2376] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15. Re-sending same reply.
I20260709 13:03:29.945294  7274 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.989787  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 365796us
I20260709 13:03:29.944242  7215 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.008838  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.002957  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.013381  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.017583  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.018056  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.988930  7281 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.019734  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2118) took 1265 ms. Trace:
I20260709 13:03:29.977800  7083 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.021095  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:29.945052  7179 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:03:30.020743  1647 rpcz_store.cc:276] 0709 13:03:28.753923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.560805 (+806882us) service_pool.cc:224] Handling call
0709 13:03:29.561210 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:29.561672 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.561721 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:29.562680 (+   959us) log.cc:844] Serialized 52 byte log entry
0709 13:03:29.730517 (+167837us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:29.744468 (+ 13951us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.744808 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 13:03:29.744980 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.745048 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.019375 (+274327us) raft_consensus.cc:1685] finished
0709 13:03:30.019418 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.019648 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59567,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":87,"raft.queue_time_us":22330,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7151,"spinlock_wait_cycles":28705920,"thread_start_us":133666,"threads_started":1,"wal-append.queue_time_us":363008}
I20260709 13:03:30.024462  1574 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:30.025529  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.026330  1574 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:30.018682  1573 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:30.029510  1572 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:29.986743  7290 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.034334  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:29.963910  7232 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.036674  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:30.037688  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80351.5us
W20260709 13:03:30.031889  7231 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.818s	user 0.014s	sys 0.000s
W20260709 13:03:30.023085  7081 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.966s	user 0.015s	sys 0.000s
I20260709 13:03:30.053416  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.069420  1647 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:29.975478  7226 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:29.965667  7224 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.065712  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:30.035231  7215 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.998241  7216 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.076206  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:30.085718  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:30.062593  7151 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:30.096737  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.028856  7274 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.028270  7269 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:03:30.033492  7211 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:03:30.098589  1571 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:30.079299  7234 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:30.087221  7048 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:30.107268  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.075928  7081 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.076222  7231 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.114739  7297 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.117460  7277 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:30.118135  7277 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.118584  7277 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:30.118986  1651 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:30.119966  7286 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:30.120533  7259 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:30.088785  7215 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.120743  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.126001  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.126722  1649 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:30.102155  7151 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:30.122553  7223 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.255s	user 0.018s	sys 0.001s
I20260709 13:03:30.138934  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.120937  7303 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.124464  7286 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.116802  7048 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:30.139660  1647 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:30.120739  7081 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.141249  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.142802  7215 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.144060  7224 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.122294  7210 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.122340  7259 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:30.151985  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.154930  1647 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:30.158043  1570 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:30.137921  7190 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.156605  7259 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:30.164222  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.28122e+06us
W20260709 13:03:30.179822  1688 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.938s (SENT)
I20260709 13:03:30.180430  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:30.192593  6906 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.192839  7312 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.193790  7312 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:30.194352  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.195061  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.195195  7226 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.195611  7025 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.196240  7025 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.198177  7294 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.198899  7025 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.205706  7231 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.207518  7274 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.209578  7048 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:30.210807  7259 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:30.139173  7223 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:30.228322  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:30.243809  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:30.244709  1647 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
W20260709 13:03:30.199642  7304 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.093s	user 0.001s	sys 0.002s
W20260709 13:03:30.281158  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.277674  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49501.4us
I20260709 13:03:30.279464  1647 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:30.280679  1644 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:30.150121  7286 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.157379  7079 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:30.244271  7192 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.017s	user 0.020s	sys 0.000s
I20260709 13:03:30.297997  7151 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.279289  7312 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.280443  6942 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:30.327976  1647 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:30.289887  7142 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:30.368116  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 84950.6us
W20260709 13:03:30.376392  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.382158  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2086) took 2698 ms (client timeout 1704 ms). Trace:
W20260709 13:03:30.382841  1642 rpcz_store.cc:269] 0709 13:03:27.683363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.023851 (+340488us) service_pool.cc:224] Handling call
0709 13:03:30.381775 (+2357924us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b5400306954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:30.382020 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132985216}
W20260709 13:03:30.330263  7311 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:03:30.404963  6815 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:30.407748  7083 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.408723  7170 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:30.409590  6447 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.553535  7170 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.414366  7128 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:30.409740  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 229877us
W20260709 13:03:30.554973  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.971s (SENT)
W20260709 13:03:30.556221  1688 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.031s (SENT)
I20260709 13:03:30.394133  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:30.446818  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.556743  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:30.365198  7312 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.371244  7223 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.373442  7142 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.419648  7185 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.000s
I20260709 13:03:30.466295  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.466595  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.468521  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.468768  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.562655  1723 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:30.470168  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.499545  1567 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:30.507676  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.563592  1716 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:30.529808  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.157306  7303 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:30.507689  7264 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.564509  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.553835  1574 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:30.432104  7165 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.006s
W20260709 13:03:30.561353  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 167232us
I20260709 13:03:30.367369  6942 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.562377  1724 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:03:30.572995  1688 leader_election.cc:336] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.047s (SENT)
I20260709 13:03:30.563035  1721 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:30.577592  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:30.579026  1721 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:30.563809  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.563809  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.563954  1717 raft_consensus.cc:2410] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:03:30.571586  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.577534  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.571995  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:30.579996  1688 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.083s (SENT)
I20260709 13:03:30.584645  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:30.588721  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.576186  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.591158  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:30.592547  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.564808  7142 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:30.593762  1723 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:30.597535  1722 raft_consensus.cc:2410] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:03:30.600056  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:30.568912  6447 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:30.570557  7151 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.605674  6831 raft_consensus.cc:993] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:30.605407  7151 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.606369  6831 raft_consensus.cc:1081] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:30.574723  6764 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.616717  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.601141  7025 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:30.604753  7207 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:30.623617  1722 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:30.609663  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.625078  1722 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:30.586956  7294 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:30.607990  7151 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.565052  7223 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.561312  7128 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.621305  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:30.562142  7312 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.635382  1722 pending_rounds.cc:85] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 6
I20260709 13:03:30.581442  7165 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.581764  7264 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.645562  1722 pending_rounds.cc:107] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:03:30.651819  1722 raft_consensus.cc:2889] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:30.649073  1688 leader_election.cc:336] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.114s (SENT)
I20260709 13:03:30.571797  7185 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:30.626106  7025 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.656976  1688 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.082s (SENT)
I20260709 13:03:30.622931  1723 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:30.623657  7226 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:30.631209  6831 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:03:30.704120  1718 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:30.679880  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.729519  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:30.650255  7297 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.005s
I20260709 13:03:30.644770  7259 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.647768  7207 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:30.728176  7025 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:30.708514  7170 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:30.687819  7317 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:03:30.681774  7303 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.638012  7294 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.730896  7312 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:30.703207  7277 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.007s
I20260709 13:03:30.658314  6764 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:03:30.744216  7297 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.751669  7207 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:30.734127  7226 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:30.757074  7319 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:03:30.763136  7101 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.760s	user 0.014s	sys 0.000s
I20260709 13:03:30.761052  7128 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.764842  7240 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.758359  7264 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.759735  7165 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.757180  7185 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.757229  7223 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.756304  7277 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:30.801832  7286 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:30.803689  7286 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.806198  7294 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.689347  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 828) took 3489 ms. Trace:
I20260709 13:03:30.860630  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.863199  1724 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:30.865252  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:30.738281  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 95854.2us
I20260709 13:03:30.867276  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.866259  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.859113  1644 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:30.869555  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.859072  1645 rpcz_store.cc:276] 0709 13:03:27.200436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.209390 (+  8954us) service_pool.cc:224] Handling call
0709 13:03:27.209758 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.624836 (+415078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.624864 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.624903 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:27.624942 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.688966 (+3064024us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54001bc1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:30.689038 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.689250 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954169728}
I20260709 13:03:30.869534  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.758360  6894 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.757215  6942 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.895560  7294 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.897012  7250 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.872010  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:30.897358  7294 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:30.889521  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.888314  7226 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.890904  7207 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.896119  7128 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:30.730897  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:30.775665  7055 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:30.901198  7128 consensus_queue.cc:1243] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:30.901302  7055 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:30.903667  7277 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.916594  6942 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:30.917793  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 260554us
I20260709 13:03:30.769737  7025 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:30.928644  1575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2826) took 3200 ms (client timeout 2217 ms). Trace:
W20260709 13:03:30.929318  1575 rpcz_store.cc:269] 0709 13:03:27.727561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.727672 (+   111us) service_pool.cc:224] Handling call
0709 13:03:30.927928 (+3200256us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54001257d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:30.928272 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256407936}
I20260709 13:03:30.933116  1722 raft_consensus.cc:2368] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
W20260709 13:03:30.934449  1688 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.199s (SENT)
I20260709 13:03:30.936398  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:30.937485  7025 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:30.938180  6942 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:30.961208  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:30.962669  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2113) took 3243 ms (client timeout 2205 ms). Trace:
W20260709 13:03:30.963297  1643 rpcz_store.cc:269] 0709 13:03:27.718909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.681096 (+962187us) service_pool.cc:224] Handling call
0709 13:03:30.961574 (+2280478us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540005d2d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:30.962267 (+   693us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307050880}
I20260709 13:03:30.967237  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 840) took 1373 ms. Trace:
W20260709 13:03:30.971125  1688 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.205s (SENT)
I20260709 13:03:30.963430  1575 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:30.968091  1648 rpcz_store.cc:276] 0709 13:03:29.593658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.770918 (+177260us) service_pool.cc:224] Handling call
0709 13:03:29.771198 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.851131 (+ 79933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.851179 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:29.851237 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:29.851280 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.966490 (+1115210us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b54001bb7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:30.966600 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.966943 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183110400}
I20260709 13:03:30.980918  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.970263  1716 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:30.967412  7142 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:30.993495  7228 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:30.995085  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.967119  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2420) took 1341 ms. Trace:
I20260709 13:03:30.996318  7142 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:30.996070  1651 rpcz_store.cc:276] 0709 13:03:29.625146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.724742 (+ 99596us) service_pool.cc:224] Handling call
0709 13:03:30.966444 (+1241702us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400411ed4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:30.966790 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894670080}
I20260709 13:03:31.004746  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.004805  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.006582  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.008811  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:31.009636  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:30.983011  7250 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:31.014137  7250 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.021514  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.023471  2373 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:31.032599  2379 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:31.034593  7250 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:31.038617  6447 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:31.039179  6736 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.039635  6447 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.032550  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2425) took 1072 ms. Trace:
I20260709 13:03:31.003434  7055 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.054071  2376 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:31.041260  7207 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.052909  7294 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.065706  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.067116  2378 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:31.065737  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:31.070676  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 1.0269e+06us
I20260709 13:03:31.071170  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 838) took 1477 ms. Trace:
I20260709 13:03:31.075294  7187 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:31.073266  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:31.077790  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.078476  7187 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.071787  1650 rpcz_store.cc:276] 0709 13:03:29.593179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.750651 (+157472us) service_pool.cc:224] Handling call
0709 13:03:29.750987 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.751221 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:29.959000 (+207779us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.041278 (+ 82278us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.069773 (+1028495us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b18001fb160. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:31.070318 (+   545us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.070424 (+   106us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:31.070554 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.070716 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.070932 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15170,"prepare.queue_time_us":44397,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":115,"raft.queue_time_us":1055547,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":15089,"spinlock_wait_cycles":35597440,"thread_start_us":261061,"threads_started":2,"wal-append.queue_time_us":132768}
I20260709 13:03:31.084461  7185 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:31.087108  7185 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.090837  7185 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.092170  7182 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.093400  7182 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.097327  1575 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:31.103523  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.106828  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.108881  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2418) took 1513 ms. Trace:
I20260709 13:03:31.113263  1649 rpcz_store.cc:276] 0709 13:03:29.594937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.799364 (+204427us) service_pool.cc:224] Handling call
0709 13:03:29.799680 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.127913 (+328233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.107992 (+980079us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54003fc854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:31.108596 (+   604us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751333120}
I20260709 13:03:31.114521  1573 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:31.121203  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2845) took 1033 ms. Trace:
I20260709 13:03:31.121757  1575 rpcz_store.cc:276] 0709 13:03:30.077132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.042974 (+965842us) service_pool.cc:224] Handling call
0709 13:03:31.110411 (+ 67437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.115784  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 649) took 1105 ms. Trace:
I20260709 13:03:31.123529  1573 rpcz_store.cc:276] 0709 13:03:30.010594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.102903 (+1092309us) service_pool.cc:224] Handling call
0709 13:03:31.115698 (+ 12795us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.127800  7181 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.128954  7181 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.132972  1569 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:31.132860  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.134389  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2843) took 1069 ms. Trace:
I20260709 13:03:31.135195  1569 rpcz_store.cc:276] 0709 13:03:30.064168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.941722 (+877554us) service_pool.cc:224] Handling call
0709 13:03:31.106405 (+164683us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:31.133857 (+ 27452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812160}
I20260709 13:03:31.136606  7142 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:31.137841  7142 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.140631  1650 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:31.098408  7055 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:31.143137  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2428) took 1127 ms. Trace:
I20260709 13:03:31.151597  7291 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.154227  7291 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.041292  1568 rpcz_store.cc:276] 0709 13:03:29.946676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.950089 (+  3413us) service_pool.cc:224] Handling call
0709 13:03:29.950419 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.950833 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:30.067296 (+116463us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.926578 (+859282us) spinlock_profiling.cc:243] Waited 887 us on lock 0x7b18001aa940. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:03:30.945699 (+ 19121us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.946943 (+  1244us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.947127 (+   184us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:30.998713 (+ 51586us) log.cc:844] Serialized 14 byte log entry
0709 13:03:30.999333 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.999505 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.001990 (+  2485us) raft_consensus.cc:1685] finished
0709 13:03:31.002015 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.019365 (+ 17350us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":205677,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":126,"raft.queue_time_us":1252,"raft.run_cpu_time_us":703,"raft.run_wall_time_us":2337,"spinlock_wait_cycles":2040576,"thread_start_us":183952,"threads_started":3,"wal-append.queue_time_us":853195}
I20260709 13:03:31.154824  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2841) took 1118 ms. Trace:
I20260709 13:03:31.158588  1574 rpcz_store.cc:276] 0709 13:03:30.036273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.036413 (+   140us) service_pool.cc:224] Handling call
0709 13:03:31.154380 (+1117967us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400041ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:31.154573 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1172913536}
I20260709 13:03:31.160642  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:31.158352  2373 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:03:31.167994  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.143720  1644 rpcz_store.cc:276] 0709 13:03:30.019256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.287657 (+268401us) service_pool.cc:224] Handling call
0709 13:03:30.346692 (+ 59035us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.879113 (+532421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.142550 (+263437us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b540038a9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:31.143029 (+   479us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173383040}
I20260709 13:03:31.172709  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:31.177493  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71623.4us
I20260709 13:03:31.172701  1575 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:31.191066  6986 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:31.194280  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.196610  7081 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.188411  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:31.168025  7181 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:31.171916  7182 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:31.207790  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.204519  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.221204  1569 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.242789  7291 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.258016  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.260597  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.262116  7142 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:31.263911  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.202559  7081 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.221433  6447 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.285099  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:31.247058  7309 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.119s	user 0.007s	sys 0.009s
I20260709 13:03:31.244122  7216 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.290287  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.308233  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:31.312115  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2432) took 1346 ms. Trace:
I20260709 13:03:31.312893  1571 rpcz_store.cc:276] 0709 13:03:29.958641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.969492 (+ 10851us) service_pool.cc:224] Handling call
0709 13:03:29.985877 (+ 16385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.104513 (+118636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.285472 (+1180959us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002a76d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:31.304480 (+ 19008us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258305920}
W20260709 13:03:31.309603  1688 connection.cc:256] RPC call timeout handler was delayed by -2.6797154459999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 13:03:31.331437  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2429) took 1225 ms. Trace:
I20260709 13:03:31.333864  6447 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:31.332739  1647 rpcz_store.cc:276] 0709 13:03:30.020825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.330516 (+309691us) service_pool.cc:224] Handling call
0709 13:03:30.429697 (+ 99181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.430013 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:30.560619 (+130606us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.129740 (+569121us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.188676 (+ 58936us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.203607 (+ 14931us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.208436 (+  4829us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:31.217449 (+  9013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.217657 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.228038 (+ 10381us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b48001c0158. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:03:31.245019 (+ 16981us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":545310,"prepare.queue_time_us":212710,"prepare.run_cpu_time_us":186,"prepare.run_wall_time_us":4951,"raft.queue_time_us":571652,"raft.run_cpu_time_us":3699,"raft.run_wall_time_us":144292,"spinlock_wait_cycles":43310336,"thread_start_us":206565,"threads_started":3,"wal-append.queue_time_us":57478}
I20260709 13:03:31.343281  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:31.342613  1573 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:31.347608  6879 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:31.348802  6879 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.352768  7334 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.342842  7216 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:31.354314  7334 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.356952  6906 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.361649  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:31.364811  7294 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.356200  7216 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.366703  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:31.369287  2372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 773) took 1780 ms (client timeout 1880 ms). Trace:
W20260709 13:03:31.369967  2372 rpcz_store.cc:269] 0709 13:03:29.588812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.588999 (+   187us) service_pool.cc:224] Handling call
0709 13:03:31.368667 (+1779668us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54000a87d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:31.368996 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435172096}
I20260709 13:03:31.372459  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.373826  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2829) took 1349 ms. Trace:
I20260709 13:03:31.374673  1568 rpcz_store.cc:276] 0709 13:03:30.024767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.158015 (+1133248us) service_pool.cc:224] Handling call
0709 13:03:31.372173 (+214158us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:31.373728 (+  1555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15237376}
I20260709 13:03:31.376766  7333 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.367949  1574 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:31.369065  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.371933  1575 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:03:31.363657  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.381773  1575 pending_rounds.cc:85] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:03:31.327279  7291 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.336167  7258 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.360159  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45556 (request call id 647) took 1350 ms. Trace:
I20260709 13:03:31.389569  1570 rpcz_store.cc:276] 0709 13:03:30.009109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.017663 (+  8554us) service_pool.cc:224] Handling call
0709 13:03:31.359408 (+1341745us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54000cba54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:31.359793 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319100288}
I20260709 13:03:31.327034  6986 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:31.385447  1575 pending_rounds.cc:107] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:03:31.398950  1572 raft_consensus.cc:2410] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:31.387955  7291 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.405782  7081 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.407475  1573 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:31.408696  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2840) took 1381 ms. Trace:
I20260709 13:03:31.403625  1575 raft_consensus.cc:2889] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:31.418428  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2831) took 1393 ms. Trace:
I20260709 13:03:31.418815  7291 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.416465  1572 rpcz_store.cc:276] 0709 13:03:30.027385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.030471 (+  3086us) service_pool.cc:224] Handling call
0709 13:03:30.935623 (+905152us) spinlock_profiling.cc:243] Waited 8.54 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:31.363264 (+427641us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49e9c 00007fc830c49a5b 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:31.408598 (+ 45334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28087296}
I20260709 13:03:31.417335  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.420739  1571 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.421172  1573 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:31.419596  1575 rpcz_store.cc:276] 0709 13:03:30.025237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.370680 (+1345443us) service_pool.cc:224] Handling call
0709 13:03:31.371291 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.418075 (+ 46784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.418334 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.424548  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:31.425786  1575 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.425639  7258 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.421239  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:31.418385  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:31.365685  7294 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.433503  1572 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
W20260709 13:03:31.439352  1688 leader_election.cc:336] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.771s (SENT)
W20260709 13:03:31.444566  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2424) took 1498 ms (client timeout 1589 ms). Trace:
W20260709 13:03:31.445147  1567 rpcz_store.cc:269] 0709 13:03:29.946275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.949143 (+  2868us) service_pool.cc:224] Handling call
0709 13:03:31.434661 (+1485518us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54003106d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:31.444318 (+  9657us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739917184}
I20260709 13:03:31.454068  7109 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:31.455822  1569 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:31.460467  7274 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.460884  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59939.1us
W20260709 13:03:31.462129  7332 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:03:31.461728  7274 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.450170  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2825) took 3576 ms. Trace:
I20260709 13:03:31.462585  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:31.464931  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.462997  1576 rpcz_store.cc:276] 0709 13:03:27.726794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.726973 (+   179us) service_pool.cc:224] Handling call
0709 13:03:27.727364 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.746629 (+ 19265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.746681 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.747447 (+   766us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.705310 (+957863us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:31.065476 (+2360166us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b18003f43c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf
0709 13:03:31.065843 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.208745 (+142902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.229482 (+ 20737us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.243362 (+ 13880us) raft_consensus.cc:1685] finished
0709 13:03:31.270007 (+ 26645us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.303194 (+ 33187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18690,"prepare.queue_time_us":511445,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":31879,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":57113,"spinlock_wait_cycles":5134720,"thread_start_us":946136,"threads_started":3,"wal-append.queue_time_us":1178640,"wal-append.run_cpu_time_us":2919,"wal-append.run_wall_time_us":1056209}
W20260709 13:03:31.468400  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50482.1us
I20260709 13:03:31.467676  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2838) took 1428 ms. Trace:
I20260709 13:03:31.475579  7298 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.479075  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.481570  7081 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:31.481348  7298 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.482321  7081 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.482805  7081 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.415927  6986 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.484792  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2832) took 1452 ms. Trace:
I20260709 13:03:31.474403  1573 rpcz_store.cc:276] 0709 13:03:30.026890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.425248 (+1398358us) service_pool.cc:224] Handling call
0709 13:03:31.427474 (+  2226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.449899 (+ 22425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.449963 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.455298 (+  5335us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:31.455352 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.455449 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.455635 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12097,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":9419}
I20260709 13:03:31.485433  1568 rpcz_store.cc:276] 0709 13:03:30.025468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.376021 (+1350553us) service_pool.cc:224] Handling call
0709 13:03:31.402258 (+ 26237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.470532 (+ 68274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.470572 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:31.477716 (+  7144us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:31.477790 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.477867 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.478035 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:31.487818  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 279136us
I20260709 13:03:31.489329  6986 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.484551  7261 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:31.490554  7261 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.491031  7261 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.491423  7187 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.494398  7185 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:31.498420  7185 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.497534  7277 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.495694  1570 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:03:31.494659  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 188529us
I20260709 13:03:31.501122  1574 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:31.500814  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.509199  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2837) took 1482 ms. Trace:
I20260709 13:03:31.509850  1570 rpcz_store.cc:276] 0709 13:03:30.026637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.395676 (+1369039us) service_pool.cc:224] Handling call
0709 13:03:31.442753 (+ 47077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.506898 (+ 64145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.509121 (+  2223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.511960  1647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "740ac262ab184918992607d66446b8f3"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:03:31.512730  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 108989us
I20260709 13:03:31.513363  1647 raft_consensus.cc:606] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:31.513929  1647 raft_consensus.cc:614] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:31.508181  1568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1acf73e529394004842219637cff7cfc"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:03:31.505612  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2834) took 1479 ms. Trace:
I20260709 13:03:31.509235  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:31.517766  1574 rpcz_store.cc:276] 0709 13:03:30.025955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.384626 (+1358671us) service_pool.cc:224] Handling call
0709 13:03:31.449463 (+ 64837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.503398 (+ 53935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.505547 (+  2149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.519373  7083 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:31.525121  7083 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.515503  7109 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:31.517040  1688 leader_election.cc:336] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.235s (SENT)
I20260709 13:03:31.536136  7333 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:31.537815  1644 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:31.533672  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:31.543023  1576 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:03:31.555071  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 81333.9us
I20260709 13:03:31.540681  1568 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:31.556499  1568 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.557857  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.517010  7258 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.561321  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:31.529410  7274 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.569561  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:31.546612  7298 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.576268  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.585268  1650 raft_consensus.cc:2368] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
I20260709 13:03:31.577003  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.577275  1570 pending_rounds.cc:85] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 10
I20260709 13:03:31.572886  6879 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.600406  1570 pending_rounds.cc:107] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:31.603667  1570 raft_consensus.cc:2889] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:31.607721  7097 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:31.607935  6447 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.608601  7097 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:31.609166  7097 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.609439  7207 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:31.617784  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:31.606346  7277 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:31.623023  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.624969  7335 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.168s	user 0.000s	sys 0.002s
I20260709 13:03:31.624439  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:31.627048  7337 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.158s	user 0.001s	sys 0.004s
W20260709 13:03:31.624265  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 116345us
I20260709 13:03:31.634673  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2116) took 2884 ms. Trace:
I20260709 13:03:31.637563  1571 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:03:31.629110  7336 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:03:31.624161  7294 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.654059  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.642995  7207 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:31.656648  7258 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:31.656908  7207 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:31.657670  7207 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.657552  7258 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.658839  7258 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.641134  7083 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.663055  1647 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:31.664705  1717 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:03:31.667728  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2791) took 1196 ms. Trace:
I20260709 13:03:31.668310  1720 rpcz_store.cc:276] 0709 13:03:30.470990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:30.471183 (+   193us) service_pool.cc:224] Handling call
0709 13:03:30.471568 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.508207 (+ 36639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.508251 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.530483 (+ 22232us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.178006 (+647523us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b18000512e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:31.344781 (+166775us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:31.344900 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.345367 (+   467us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.663349 (+317982us) raft_consensus.cc:1685] finished
0709 13:03:31.663647 (+   298us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.667467 (+  3820us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61481,"raft.queue_time_us":36834,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":142,"spinlock_wait_cycles":9718016,"thread_start_us":129343,"threads_started":1,"wal-append.queue_time_us":991383}
I20260709 13:03:31.671481  7109 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:31.681103  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 132347us
I20260709 13:03:31.635167  1646 rpcz_store.cc:276] 0709 13:03:28.749767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.749899 (+   132us) service_pool.cc:224] Handling call
0709 13:03:28.871222 (+121323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.022280 (+2151058us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18002e1800. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81557 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:31.226867 (+204587us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b18002e1800. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d66bfb 00007fc833d7164b
0709 13:03:31.227149 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.227193 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.228088 (+   895us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.313375 (+ 85287us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:31.337591 (+ 24216us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.435272 (+ 97681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.443225 (+  7953us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.539376 (+ 96151us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b18002e1800. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:31.623441 (+ 84065us) raft_consensus.cc:1685] finished
0709 13:03:31.623471 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.629646 (+  6175us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54000ba750. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:31.634370 (+  4724us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173043,"prepare.queue_time_us":23855,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":6011,"raft.queue_time_us":236230,"raft.run_cpu_time_us":484,"raft.run_wall_time_us":26976,"spinlock_wait_cycles":392248960,"thread_start_us":302847,"threads_started":2,"wal-append.queue_time_us":217586}
I20260709 13:03:31.683809  6906 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.684827  7185 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.684907  6906 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.686654  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:31.687492  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.688290  1718 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:31.687532  1722 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:31.689770  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:31.690667  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46304.9us
W20260709 13:03:31.683610  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.515s (SENT)
W20260709 13:03:31.707288  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.693459  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:31.727224  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.693629  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.694406  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:31.698334  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.611838  7216 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:31.714330  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 221922us
I20260709 13:03:31.729985  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:31.728597  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 89370.6us
W20260709 13:03:31.712772  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 196955us
I20260709 13:03:31.739184  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.751909  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.746925  1571 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:03:31.760989  1571 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:03:31.761466  1571 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:31.762501  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.765050  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:31.765843  1724 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.731076  7334 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:31.769785  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:31.775507  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89456.5us
I20260709 13:03:31.785804  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:31.776793  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:31.789467  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2839) took 1749 ms. Trace:
I20260709 13:03:31.771302  7227 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:31.769197  7277 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:31.805078  1688 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.380s (SENT)
I20260709 13:03:31.805207  1717 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:31.809062  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68460.6us
I20260709 13:03:31.804510  7227 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.812443  1722 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:31.816399  7227 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.813235  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:31.817739  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:31.819453  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 136091us
I20260709 13:03:31.821240  6923 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:31.821821  6923 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:31.822468  6923 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:31.824218  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87029.6us
I20260709 13:03:31.825380  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.827116  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:31.828162  7187 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:31.828729  7187 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.828826  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.829217  7187 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.829993  1719 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:31.838376  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:31.790104  1572 rpcz_store.cc:276] 0709 13:03:30.027173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.446201 (+1419028us) service_pool.cc:224] Handling call
0709 13:03:31.474388 (+ 28187us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:31.494622 (+ 20234us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:31.500950 (+  6328us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.566188 (+ 65238us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:31.603398 (+ 37210us) log.cc:844] Serialized 52 byte log entry
0709 13:03:31.603613 (+   215us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:31.624972 (+ 21359us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b180041c9e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:31.647585 (+ 22613us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.764469 (+116884us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.765148 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.771478 (+  6330us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:31.771572 (+    94us) raft_consensus.cc:1685] finished
0709 13:03:31.771591 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:31.776610 (+  5019us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172126,"prepare.queue_time_us":53561,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":98,"raft.queue_time_us":2778,"raft.run_cpu_time_us":655,"raft.run_wall_time_us":17265,"spinlock_wait_cycles":7950592,"thread_start_us":53961,"threads_started":1,"wal-append.queue_time_us":139013}
I20260709 13:03:31.840261  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:31.848802  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:31.849388  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:31.847397  1723 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:31.845644  1568 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:31.857641  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:31.872642  6447 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:31.873014  7109 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:31.877360  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.878742  1649 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:31.880453  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.880887  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.881599  1721 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:31.881928  1651 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 18 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:03:31.873545  6447 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.885845  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:31.887324  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:31.872695  7333 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.888177  1719 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.891896  1647 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
W20260709 13:03:31.897207  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46040.6us
I20260709 13:03:31.886940  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.896171  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.887324  6447 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.906278  7178 raft_consensus.cc:2764] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:03:31.912890  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:31.897123  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.830s (SENT) [suppressed 24 similar messages]
I20260709 13:03:31.916093  1568 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:31.920275  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.920622  1717 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:31.887586  7291 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.927304  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.928946  1651 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:31.930346  1724 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:31.932595  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.945039  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44760 (request call id 2445) took 1004 ms. Trace:
I20260709 13:03:31.962826  1643 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:31.952472  1568 rpcz_store.cc:276] 0709 13:03:30.915620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.486708 (+571088us) service_pool.cc:224] Handling call
0709 13:03:31.919329 (+432621us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:31.966540  1644 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:31.963246  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.913772  7277 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:31.971380  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.965324  1645 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:31.988909  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.970816  6923 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:31.977275  1568 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:03:31.980687  7097 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.010366  1716 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:32.010521  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.982172  1648 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:31.985132  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:31.934545  6906 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:31.988909  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.989369  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.989919  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.990439  7183 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:31.991607  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 92917.7us
I20260709 13:03:32.020040  1651 raft_consensus.cc:2410] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:32.015782  1719 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:32.037046  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.025558  7258 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.040990  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.042415  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.042711  7258 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:32.043429  1725 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:32.044755  1720 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:03:32.044822  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.048678  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.049468  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.039881  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.051349  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.045997  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.047590  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:32.060703  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.045884  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:32.064610  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.065311  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:31.997318  7277 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:32.020380  7261 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.009s
I20260709 13:03:32.064703  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.072314  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.073768  1649 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:32.048883  7183 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:32.072016  7277 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:32.080719  1575 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.004s
I20260709 13:03:32.081488  1575 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:32.075809  2373 raft_consensus.cc:2410] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:03:32.075618  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 48193.5us
I20260709 13:03:32.070808  1721 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:32.046103  6906 raft_consensus.cc:2764] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:03:32.043779  7081 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.560s	user 0.016s	sys 0.000s
I20260709 13:03:32.084862  1725 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:32.094480  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.096642  7083 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.097244  7083 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.097318  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.097738  7083 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.098165  1651 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:31.961877  7291 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.096323  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.080963  7183 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.099053  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.100983  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:32.109560  1720 raft_consensus.cc:606] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:32.110004  1720 raft_consensus.cc:614] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:32.107764  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.112690  1648 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:32.117060  7258 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:32.117655  1723 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:32.118436  7258 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:32.118788  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:32.118884  7258 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.119823  1642 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:32.077733  7261 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.099607  7081 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.112974  7207 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.124794  6906 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:32.161259  7294 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:32.169878  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.170594  7294 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:32.171129  1720 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:32.177848  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.150864  7216 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:32.180405  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 330927us
I20260709 13:03:32.199414  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:32.130463  7258 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.201018  6906 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:32.183214  7097 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:32.200222  6923 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:32.229552  7341 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:03:32.230839  6923 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.159133  6986 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:03:32.231820  6923 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.233558  6986 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:32.234331  6986 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.235495  6923 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:32.236943  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.016s
W20260709 13:03:32.237486  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.005s	sys 0.016s
W20260709 13:03:32.239144  1570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.003s	sys 0.015s
I20260709 13:03:32.237828  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:32.253320  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "58cdb682fbf54792a702a7a455f265a2"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:32.254212  1642 raft_consensus.cc:606] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Received request to transfer leadership to TS 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:32.254865  1642 raft_consensus.cc:614] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:32.257282   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:32.258376  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:32.259771  7178 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.260336  7178 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.260612  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:32.261780   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:32.262660  6923 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:32.261458  7109 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:32.260964  7178 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.236048  6986 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.265931  6986 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.269490  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:32.272459  7227 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.273084  6906 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.274106  7109 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:32.276103  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.278707  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.280786  6923 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:32.281455  6923 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.281608  7291 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:32.282449  7291 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:32.282891  7291 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:32.282788  6923 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.287531  7261 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:32.288085  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.288690  7261 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:32.289356  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.249303  7097 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:32.293390  7097 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:32.294060  7178 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.294690  7097 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.297523  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2844) took 2230 ms. Trace:
I20260709 13:03:32.298087  1567 rpcz_store.cc:276] 0709 13:03:30.067238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.453574 (+1386336us) service_pool.cc:224] Handling call
0709 13:03:31.454363 (+   789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.459763 (+  5400us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:31.460229 (+   466us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.489339 (+ 29110us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b4800002dd8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:03:31.610391 (+121052us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b1800387a00. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:03:31.762686 (+152295us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.815653 (+ 52967us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.839017 (+ 23364us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:31.859168 (+ 20151us) log.cc:844] Serialized 14 byte log entry
0709 13:03:31.863415 (+  4247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.863508 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.297031 (+433523us) raft_consensus.cc:1685] finished
0709 13:03:32.297074 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.297405 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17960,"prepare.queue_time_us":45394,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":116,"raft.queue_time_us":10187,"raft.run_cpu_time_us":2615,"raft.run_wall_time_us":35579,"spinlock_wait_cycles":23520000,"thread_start_us":120582,"threads_started":1,"wal-append.queue_time_us":642440}
I20260709 13:03:32.287673  7258 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.301520  7227 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:32.303076  7227 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:32.304507  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.304872  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2450) took 1268 ms. Trace:
I20260709 13:03:32.305253  7081 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:32.301329  7291 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.306156  7081 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:32.306849  7081 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.307405  7207 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:32.313829  7178 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:32.281069  7216 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:32.329046  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.305575  1574 rpcz_store.cc:276] 0709 13:03:31.036915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.530308 (+493393us) service_pool.cc:224] Handling call
0709 13:03:31.546532 (+ 16224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.580633 (+ 34101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.580679 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.638971 (+ 58292us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.144595 (+505624us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:32.144649 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.144745 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.304502 (+159757us) raft_consensus.cc:1685] finished
0709 13:03:32.304537 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.304774 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8193,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":2249,"thread_start_us":475287,"threads_started":1,"wal-append.queue_time_us":615650}
I20260709 13:03:32.320708  6906 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:32.296128  7187 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.329536  6447 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.331022  1722 raft_consensus.cc:2368] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13.
I20260709 13:03:32.331441  1720 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:32.331544  7187 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:32.329052  7216 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:32.327571  7083 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.335835  7083 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:32.307994  7081 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.303841  7227 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:32.360185  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.018s	sys 0.071s
W20260709 13:03:32.362803  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.018s	sys 0.071s
I20260709 13:03:32.365244  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.375448  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.375859  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2460) took 1057 ms. Trace:
I20260709 13:03:32.376398  1723 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:32.378887  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.393816  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:32.392629  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 504142us
I20260709 13:03:32.401580  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.414413  1723 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
W20260709 13:03:32.420245  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.009s	sys 0.012s
W20260709 13:03:32.424823  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.010s	sys 0.013s
W20260709 13:03:32.425263  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 566524us
W20260709 13:03:32.407932  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.016s
W20260709 13:03:32.428611  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.004s	sys 0.016s
W20260709 13:03:32.420503  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.019s
W20260709 13:03:32.437335  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.004s	sys 0.019s
W20260709 13:03:32.438058  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 572160us
I20260709 13:03:32.454504  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:32.408571  7185 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.009s	sys 0.023s
I20260709 13:03:32.456660  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.459263  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:32.462862  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.465651  1723 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:32.389870  7183 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.473383  1716 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:32.473723  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.467020  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:32.469696  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.477180  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:32.478055  1722 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.376504  1569 rpcz_store.cc:276] 0709 13:03:31.322643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.675669 (+353026us) service_pool.cc:224] Handling call
0709 13:03:31.676680 (+  1011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:31.677102 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:31.760410 (+ 83308us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.375397 (+614987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:32.375475 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:32.375535 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.375638 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.375786 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31215,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":11019,"raft.queue_time_us":9379,"thread_start_us":563867,"threads_started":1}
I20260709 13:03:32.478834  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:32.472532  1721 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:32.462060  7185 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.010s	sys 0.024s
W20260709 13:03:32.486688  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 694517us
I20260709 13:03:32.487323  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:32.494095  1718 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:32.480525  1720 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:32.491251  1646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.023s	sys 0.036s
W20260709 13:03:32.496443  1646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.024s	sys 0.036s
W20260709 13:03:32.497390  1688 connection.cc:256] RPC call timeout handler was delayed by -2.4610421869999755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 13:03:32.497918  1688 connection.cc:256] RPC call timeout handler was delayed by -2.3152421869999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260709 13:03:32.497447  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:32.498544  1688 connection.cc:256] RPC call timeout handler was delayed by -2.2865891689999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
W20260709 13:03:32.499029  1688 connection.cc:256] RPC call timeout handler was delayed by -1.7559120849999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:03:32.499452  1688 connection.cc:256] RPC call timeout handler was delayed by -1.7352120850000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:03:32.499869  1688 connection.cc:256] RPC call timeout handler was delayed by -1.7352120850000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:03:32.500270  1688 connection.cc:256] RPC call timeout handler was delayed by -1.7200098109999544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 13:03:32.501717  1688 connection.cc:256] RPC call timeout handler was delayed by -1.6253984139999602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30610000000000004s for a response
W20260709 13:03:32.502269  1688 connection.cc:256] RPC call timeout handler was delayed by -1.5787098109999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 13:03:32.502643  1688 connection.cc:256] RPC call timeout handler was delayed by -1.5571098109999753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
W20260709 13:03:32.502961  1688 connection.cc:256] RPC call timeout handler was delayed by -1.509409810999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 13:03:32.503386  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1979120849999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
W20260709 13:03:32.503794  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1511612659999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 13:03:32.504176  1688 connection.cc:256] RPC call timeout handler was delayed by -1.1511612659999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
I20260709 13:03:32.506385  1721 raft_consensus.cc:2368] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:32.493322  6447 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:32.509877  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:32.516072  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 360213us
W20260709 13:03:32.515717  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 291648us
W20260709 13:03:32.520475  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:32.508315  7298 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.012s	sys 0.025s
I20260709 13:03:32.450807  7227 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:32.531594  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 242296us
I20260709 13:03:32.488992  7178 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:03:32.510712  7109 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.524037  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:32.525137  7298 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.013s	sys 0.025s
I20260709 13:03:32.531468  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:32.489197  7277 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.549256  7350 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:32.550166  7350 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:32.553359  7346 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:03:32.554327  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.555593  6986 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:32.556255  7083 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:03:32.556619  7185 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:32.562067  7185 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.562476  7185 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.563495  7277 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:32.565371  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.578025  1722 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:32.551461  7350 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.542856  6447 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:32.591257  7334 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.595809  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 120485us
I20260709 13:03:32.568946  7261 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:32.592160  7334 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.605180  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.580619  6986 consensus_queue.cc:1243] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:03:32.594319  7344 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.209s	user 0.000s	sys 0.002s
I20260709 13:03:32.546039  7187 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:03:32.613236  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:32.587320  7083 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:32.611725  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 80328.9us
I20260709 13:03:32.561550  7294 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:03:32.622308  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2446) took 1704 ms. Trace:
W20260709 13:03:32.625072  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59761.8us
I20260709 13:03:32.613152  7207 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:03:32.636065  7353 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:32.636703  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 82791.2us
I20260709 13:03:32.628362  1569 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:32.626036  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.625275  1570 rpcz_store.cc:276] 0709 13:03:30.918964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.521238 (+602274us) service_pool.cc:224] Handling call
0709 13:03:31.550263 (+ 29025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.687600 (+137337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.687643 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.688225 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.321314 (+633089us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:32.321367 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.321463 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.621918 (+300455us) raft_consensus.cc:1685] finished
0709 13:03:32.621955 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.622187 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":504035,"threads_started":1,"wal-append.queue_time_us":683434}
I20260709 13:03:32.634990  7261 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.657495  7354 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:32.658331  7354 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:32.658835  7354 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.659171  7334 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:32.659955  7178 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.661051  7178 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:32.661607  7178 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.663367  6923 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:32.664453  7187 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:32.664834  7187 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.665388  7358 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:03:32.665907  7187 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.621918  7083 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:32.668221  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55514.6us
I20260709 13:03:32.641855  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.673326  7187 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:32.673650  7347 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:03:32.674659  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2444) took 1760 ms. Trace:
I20260709 13:03:32.675362  1573 rpcz_store.cc:276] 0709 13:03:30.913866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.486679 (+572813us) service_pool.cc:224] Handling call
0709 13:03:31.518241 (+ 31562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.549601 (+ 31360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.549644 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.579725 (+ 30081us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.237955 (+658230us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:32.238002 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.238105 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.674252 (+436147us) raft_consensus.cc:1685] finished
0709 13:03:32.674286 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.674535 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14503,"raft.queue_time_us":14696,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"thread_start_us":590841,"threads_started":1,"wal-append.queue_time_us":747831}
I20260709 13:03:32.678503  7277 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:32.681398  7357 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:32.683754  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2451) took 1645 ms. Trace:
I20260709 13:03:32.684284  1576 rpcz_store.cc:276] 0709 13:03:31.038425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.563663 (+525238us) service_pool.cc:224] Handling call
0709 13:03:31.578516 (+ 14853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.596994 (+ 18478us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:31.895103 (+298109us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.417043 (+521940us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:32.453621 (+ 36578us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.453863 (+   242us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:32.453904 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.453990 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.683448 (+229458us) raft_consensus.cc:1685] finished
0709 13:03:32.683478 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.683658 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14313,"prepare.queue_time_us":53256,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":90,"raft.queue_time_us":26809,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":17563,"thread_start_us":696345,"threads_started":2,"wal-append.queue_time_us":664382}
I20260709 13:03:32.644255  7258 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:32.690891  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63659.5us
I20260709 13:03:32.698081  6986 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:32.699910  6923 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:32.699103  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:32.701063  7356 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:32.717914  7258 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.726579  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2453) took 1657 ms. Trace:
I20260709 13:03:32.726876  7258 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.727164  1571 rpcz_store.cc:276] 0709 13:03:31.068993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.649029 (+580036us) service_pool.cc:224] Handling call
0709 13:03:31.746555 (+ 97526us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:31.810930 (+ 64375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.810971 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:31.847753 (+ 36782us) log.cc:844] Serialized 52 byte log entry
0709 13:03:32.408302 (+560549us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:32.432017 (+ 23715us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.432290 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.432421 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.726236 (+293815us) raft_consensus.cc:1685] finished
0709 13:03:32.726270 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.726490 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23366,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":71,"raft.queue_time_us":21071,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":24883,"thread_start_us":472584,"threads_started":1,"wal-append.queue_time_us":634938}
I20260709 13:03:32.730407  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.731158  1614 leader_election.cc:400] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:32.731689  1614 leader_election.cc:403] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:03:32.702234  7348 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:03:32.706252  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52586.9us
I20260709 13:03:32.709174  7333 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:32.709761  7109 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:32.695796  7291 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:32.712721  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.751492  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:32.713557  7083 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:32.715407  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59859.2us
W20260709 13:03:32.750566  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.685258  7183 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:32.757483  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.757241  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:32.772948  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.777499  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.779265  1570 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:32.782037  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.723610  6986 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.787016  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:32.787874  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.756273  7109 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.747736  7334 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:32.747414  7333 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.806550  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:32.798444  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:32.805614  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:32.808547  1718 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.806039  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.806195  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.809803  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:32.810801  1717 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:32.810913  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:32.781220  6923 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.812067  2372 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:32.786339  7358 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.816314  1570 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:32.805764  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:32.819367  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.819766  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:32.822523  1716 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.825016  1569 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:32.836990  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:32.837864  1574 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.841775  1720 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:32.799842  7334 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:32.850587  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:32.794452  7353 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:32.838250  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 107910us
I20260709 13:03:32.820909  7358 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:32.839587  7294 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.870395  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:32.852927  7334 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.894063  1569 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:32.897396  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.899299  1575 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:32.858916  7083 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.904291  1718 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:32.905597  7333 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.867590  7358 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.896366  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:32.910435  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 122679us
I20260709 13:03:32.867545  7294 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:32.893085  7216 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.560s	user 0.016s	sys 0.000s
I20260709 13:03:32.910148  7363 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:32.926786  7363 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.928231  7363 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:32.916517  7367 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:03:32.918322  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:32.932019  1722 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:32.919658  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.941344  7352 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:32.940980  7354 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:32.941941  7352 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:32.942301  7378 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:32.943064  7352 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.945883  7352 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.946748  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:32.947605  1571 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.950788  1572 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:32.945037  7368 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:32.958242  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:32.968297  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 89475.1us
I20260709 13:03:32.910193  6986 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:32.954629  7294 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.000324  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.001675  1567 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.993103  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:32.994669  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.005250  1569 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.006644  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:33.007787  1575 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:33.010300  2342 connection.cc:256] RPC call timeout handler was delayed by -1.3124084940000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
I20260709 13:03:32.982247  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.015393  1570 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:33.011713  7378 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:32.975703  7261 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:32.964685  7368 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:33.026103  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 131520us
I20260709 13:03:33.031844  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.028260  1576 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:33.037247  1574 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:03:33.041548  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.041919  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68510.7us
I20260709 13:03:32.905599  7353 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.043685  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:33.043857  7258 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.016388  7373 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:32.964690  7216 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.056326  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:33.059244  2342 connection.cc:256] RPC call timeout handler was delayed by -1.185166272999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 13:03:33.087046  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.067148  1716 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:33.064199  7373 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.028728  7354 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.080054  7375 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:33.085194  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.054709  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:33.096257  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.098623  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.119007  1571 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:33.104635  7371 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:03:33.109244  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:33.109623  7185 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.003s
W20260709 13:03:33.111224  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.062520  7227 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.119796  1642 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:33.132629  1650 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:33.139425  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:33.134328  1576 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:33.119493  1646 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:33.055606  7298 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.136397  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:33.149194  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 102643us
I20260709 13:03:33.151453  1571 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:33.161665  1569 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
W20260709 13:03:33.165405  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 121791us
I20260709 13:03:33.110746  7373 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:33.168282  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 127093us
I20260709 13:03:33.171530  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:33.175154  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:33.177467  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:33.184142  1575 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:33.031697  7378 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.104696  7183 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:03:33.195199  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.191365  1722 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:33.191846  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.190086  1576 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 13:03:33.200721  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49018.7us
I20260709 13:03:33.205487  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.198057  1651 raft_consensus.cc:2376] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17. Re-sending same reply.
I20260709 13:03:33.211258  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:33.138411  7185 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.218315  7385 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:33.219354  7385 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.219908  7385 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.220510  7384 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:33.223423  7380 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:03:33.179153  7298 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.225322  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:33.226569  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:33.236266  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:33.240319  1649 raft_consensus.cc:2368] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16.
I20260709 13:03:33.243815  1645 raft_consensus.cc:2376] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15. Re-sending same reply.
I20260709 13:03:33.255071  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:33.248176  1576 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
W20260709 13:03:33.221689  7376 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:03:33.215590  7187 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.542s	user 0.015s	sys 0.000s
I20260709 13:03:33.163949  7227 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:33.280519  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 69399.8us
I20260709 13:03:33.220526  7372 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:33.279210  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:33.152559  7375 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.293895  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:33.297591  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 130914us
I20260709 13:03:33.223472  7109 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.303849  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.292155  1571 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:03:33.286746  7178 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.599s	user 0.016s	sys 0.000s
I20260709 13:03:33.309954  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.286121  6906 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:33.244943  7354 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:33.295099  7384 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:33.318922  1575 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:33.317499  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:33.321772  1650 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:33.315217  1651 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.326905  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:33.330761  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.335891  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.297101  7258 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.297307  7183 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:33.350680  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.325718  7178 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.365074  7109 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.365984  7109 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:33.372107  7109 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.367012  7178 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:33.369733  7187 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.373399  7216 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.337745  7384 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:33.379042  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.418947  1567 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:33.378850  7375 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:33.322863  6906 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:33.380036  7366 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:33.366802  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 141505us
I20260709 13:03:33.381533  7298 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.383160  7390 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:33.364001  7372 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:33.452993  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 159111us
I20260709 13:03:33.435880  1575 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:33.454425  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:33.454811  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:33.438647  1569 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:33.456171  1567 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:33.386234  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68786.1us
I20260709 13:03:33.439543  1570 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:33.457442  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:33.447067  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 120165us
I20260709 13:03:33.459601  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:33.422216  7187 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.433513  7384 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.468237  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:33.466187  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:33.479993  7405 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:33.485280  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:33.485507  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:33.488759  1569 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:33.447801  7216 raft_consensus.cc:2764] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:03:33.492125  7366 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.496136  7227 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.496726  7333 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:33.497453  7388 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:03:33.497910  7378 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:33.500583  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.503437  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.509017  7402 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:33.512898  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.516242  7395 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:03:33.520512  7400 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:33.453270  7294 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.483614  7261 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:33.526010  7403 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:33.526840  7403 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:33.527900  7403 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.465850  7390 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.531843  7390 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.532636  7351 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:33.533407  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:33.534097  1722 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:33.542392  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65109.9us
I20260709 13:03:33.548193  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:33.551957  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:33.541337  7261 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:33.552676  7393 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.287s
I20260709 13:03:33.571007  7372 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:33.574361  7109 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:33.554812  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 86747.9us
I20260709 13:03:33.577926  1690 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:33.578455  7334 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.004s
I20260709 13:03:33.579016  7334 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.580410  7405 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:33.581176  7405 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:33.585662  7291 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.009s
W20260709 13:03:33.586906  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 126072us
I20260709 13:03:33.588369  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:33.589457  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:33.538178  6923 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.718s	user 0.013s	sys 0.003s
I20260709 13:03:33.590649  6923 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16.
I20260709 13:03:33.590945  1648 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:03:33.497524  6906 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.591524  1642 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:33.561460  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:33.592943  6906 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.563004  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:33.593804  1716 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.565145  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:33.313870   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:33.545655  7400 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:33.566515  7261 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.598809  7400 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:33.591575  1648 pending_rounds.cc:85] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 10
I20260709 13:03:33.600863  1648 pending_rounds.cc:107] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:03:33.601090  1648 raft_consensus.cc:2889] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.601984  7294 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:33.561108  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.562602  7083 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.003s
W20260709 13:03:33.605050  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.605342  7227 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.592943  1718 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.605967  7227 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.552699  7298 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:33.607748  7298 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:33.609468  7097 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:33.609134  7298 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.610097  7097 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:33.561605  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:33.613426  1719 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:33.605577  1573 raft_consensus.cc:2410] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:33.505026  7368 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:33.604643  1569 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:33.625929  1651 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:03:33.644842  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67010.5us
I20260709 13:03:33.646720  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:33.830796  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.694626  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:33.785981  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:33.899751  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.735993  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.724635  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78020.9us
I20260709 13:03:33.655133  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:33.928524  1716 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
W20260709 13:03:33.893990  7352 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.920s	user 0.014s	sys 0.000s
I20260709 13:03:33.632601  1722 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:33.675751  1718 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:33.693511  1567 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:33.634752  7375 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:33.538502  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.705691  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 400480us
W20260709 13:03:33.935580  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.916105  1719 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:33.663965  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:33.778076  7358 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.823s	user 0.012s	sys 0.002s
I20260709 13:03:33.766716  1569 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:33.938508  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:33.766335  7183 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:33.947772  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:33.818017  7368 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:33.674098  7083 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:33.932045  7227 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:33.775920  7391 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:33.932444  7353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.004s	sys 0.030s
W20260709 13:03:33.969405  7353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.004s	sys 0.031s
I20260709 13:03:33.970320  1721 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:33.971220  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:33.974900  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2861) took 1024 ms. Trace:
I20260709 13:03:33.723863  7109 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.945113  7393 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:33.977083  7109 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:33.951879  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.840919  7187 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.954169  1724 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:33.978865  1568 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:03:33.980448  2342 leader_election.cc:400] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
I20260709 13:03:33.980877  2342 leader_election.cc:403] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:33.981597  7382 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:33.982313  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:33.986418  7354 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:33.987130  7354 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:33.988559  7291 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.923029  7366 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:33.955751  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2859) took 1006 ms. Trace:
I20260709 13:03:33.949147  7375 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:33.956643  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.931486  7402 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.998674  7402 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.965013  7083 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:34.000918  7083 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.003463  7360 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:33.994704  6447 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.004421  7375 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.005438  6447 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:34.008977  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.009600  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.009946  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:33.958433  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 172530us
I20260709 13:03:33.931300  7333 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.962517  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 319704us
I20260709 13:03:33.651580  7294 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:33.947605  7378 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:33.951742  7396 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.938426  7407 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:33.958264  7352 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.782049  7408 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.633024  7399 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.325s
I20260709 13:03:33.990355  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:33.980012  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2862) took 1029 ms. Trace:
I20260709 13:03:34.004012  7360 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:34.029639  1568 rpcz_store.cc:276] 0709 13:03:32.950369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.977906 (+1027537us) service_pool.cc:224] Handling call
0709 13:03:33.978580 (+   674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.979705 (+  1125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.979948 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.952004  2379 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:33.992142  1569 rpcz_store.cc:276] 0709 13:03:32.949407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.709155 (+759748us) service_pool.cc:224] Handling call
0709 13:03:33.766209 (+ 57054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.955357 (+189148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.955663 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.975674  1567 rpcz_store.cc:276] 0709 13:03:32.950059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.951304 (+1001245us) service_pool.cc:224] Handling call
0709 13:03:33.964179 (+ 12875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.974114 (+  9935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.974147 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:33.974218 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:33.974558 (+   340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.974655 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.974818 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:33.991732  7291 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:33.996421  7385 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.739s	user 0.017s	sys 0.000s
I20260709 13:03:34.031105  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:03:34.030395  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 371125us
I20260709 13:03:34.019181  7407 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:34.018576  7391 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:33.695055  7097 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.028731  7396 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:34.041914  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.022784  7333 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.019007  7378 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:33.988305  7354 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.023716  7294 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:33.959703  7358 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.049965  7407 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.060448  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.065254  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:34.067312  1568 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:34.034436  7334 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:34.069015  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43497.9us
I20260709 13:03:34.071035  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.076152  7396 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.081451  1568 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:34.082736  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:34.088025  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.993275  7409 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:03:34.061558  7378 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.041558  7352 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.049702  7109 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:34.066298  7373 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.827s	user 0.016s	sys 0.000s
I20260709 13:03:34.067160  7294 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:34.098124  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68739.4us
I20260709 13:03:34.083617  7410 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.084167  6447 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:34.124667  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53709.4us
I20260709 13:03:34.093307  7397 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.044875  7385 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.088716  7411 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:03:34.135939  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 53315.1us
I20260709 13:03:34.119087  7396 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.145283  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:34.041630  7408 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.128018  7393 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.195228  7393 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.199373  7393 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:34.199846  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.200670  7358 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.124650  7333 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.201341  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:34.202363  7333 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:34.202963  7333 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:34.151476  7417 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:34.204016  7333 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.158793  7385 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:34.143911  7401 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.208281  7397 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:34.208922  7408 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.160949  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:34.174422  1573 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 2847) took 1469 ms (client timeout 1569 ms). Trace:
I20260709 13:03:34.210808  7387 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:34.210985  1573 rpcz_store.cc:269] 0709 13:03:32.705074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.749339 (+ 44265us) service_pool.cc:224] Handling call
0709 13:03:34.174171 (+1424832us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54000c2454. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:34.174340 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889822080}
I20260709 13:03:34.213806  1573 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:03:34.214331  1573 pending_rounds.cc:85] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:03:34.214625  1573 pending_rounds.cc:107] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:03:34.214951  1573 raft_consensus.cc:2889] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:34.175314  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2136) took 1063 ms. Trace:
I20260709 13:03:34.175158  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:34.061615  7360 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.216878  1717 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.178637  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2139) took 1010 ms. Trace:
I20260709 13:03:34.217480  1642 rpcz_store.cc:276] 0709 13:03:33.168265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.168462 (+   197us) service_pool.cc:224] Handling call
0709 13:03:34.178213 (+1009751us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54001ba8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:34.178557 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908165504}
I20260709 13:03:34.179389  1720 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:03:34.218096  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57268us
I20260709 13:03:34.181356  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:34.219810  1718 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:34.185755  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:34.187214  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:34.173182  6986 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.220630  2372 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:34.188119  1643 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:34.184237  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:34.183914  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:34.189759  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2832) took 1006 ms. Trace:
I20260709 13:03:34.147178  7391 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.187229  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:34.192881  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.216084  7185 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.207963  7401 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.122350  7109 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.216576  1645 rpcz_store.cc:276] 0709 13:03:33.111919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.112148 (+   229us) service_pool.cc:224] Handling call
0709 13:03:34.175073 (+1062925us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54004114d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:34.175235 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237145344}
I20260709 13:03:34.224867  1719 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:34.223706  1721 rpcz_store.cc:276] 0709 13:03:33.183590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.183868 (+   278us) service_pool.cc:224] Handling call
0709 13:03:33.201891 (+ 18023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.970806 (+768915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.189416 (+218610us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b5400171bd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:34.189675 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765827968}
I20260709 13:03:34.232288  7401 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.232777  2373 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:34.222704  2379 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:34.219587  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:34.243652  7405 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.662s	user 0.007s	sys 0.007s
I20260709 13:03:34.244863  7398 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:34.244666  7405 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.247973  7365 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:34.248472  7365 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:34.249692  7382 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
W20260709 13:03:34.249495  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 48213.1us
I20260709 13:03:34.253289  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:34.254987  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:34.264394  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.267325  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2480) took 1269 ms. Trace:
W20260709 13:03:34.268625  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139709us
I20260709 13:03:34.273013  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:34.243685  1717 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:34.180259  7390 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.648s	user 0.008s	sys 0.002s
I20260709 13:03:34.282174  7390 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.282946  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:34.284446  7363 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:34.286270  6906 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.000s
I20260709 13:03:34.286687  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:34.286996  6906 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.287281  1716 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:34.287621  6906 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.258888  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:34.310379  1573 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:34.247344  7083 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:34.318423  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59667.6us
I20260709 13:03:34.326395  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2486) took 1002 ms. Trace:
I20260709 13:03:34.327072  1573 rpcz_store.cc:276] 0709 13:03:33.323687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.216316 (+892629us) service_pool.cc:224] Handling call
0709 13:03:34.310081 (+ 93765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.325950 (+ 15869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.326292 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:34.339015  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2483) took 1027 ms. Trace:
I20260709 13:03:34.249823  7365 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.269671  6447 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.289628  7398 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:34.347731  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79379.7us
W20260709 13:03:34.348508  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75594us
I20260709 13:03:34.353796  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:34.339792  1574 rpcz_store.cc:276] 0709 13:03:33.312531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.013972 (+701441us) service_pool.cc:224] Handling call
0709 13:03:34.014528 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.014883 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:34.034066 (+ 19183us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.337417 (+303351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.337490 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:34.337556 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.338750 (+  1194us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.338932 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19194,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":63,"raft.queue_time_us":5164,"thread_start_us":292640,"threads_started":1,"wal-append.queue_time_us":298784}
W20260709 13:03:34.329005  7384 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.757s	user 0.015s	sys 0.000s
I20260709 13:03:34.131754  7373 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.323344  7363 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:34.373415  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 228114us
I20260709 13:03:34.362442  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2474) took 1487 ms. Trace:
I20260709 13:03:34.374794  1576 rpcz_store.cc:276] 0709 13:03:32.874957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.305481 (+430524us) service_pool.cc:224] Handling call
0709 13:03:33.306165 (+   684us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:33.443803 (+137638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.443842 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:33.452299 (+  8457us) log.cc:844] Serialized 52 byte log entry
0709 13:03:33.536188 (+ 83889us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18004190e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:33.926896 (+390708us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:34.334158 (+407262us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.347732 (+ 13574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.349056 (+  1324us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.349109 (+    53us) raft_consensus.cc:1685] finished
0709 13:03:34.361969 (+ 12860us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.362305 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4288,"prepare.queue_time_us":50312,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":147478,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":19722,"spinlock_wait_cycles":26508544,"thread_start_us":896132,"threads_started":3,"wal-append.queue_time_us":587610}
I20260709 13:03:34.376612  1573 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:34.364295  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:34.317052  7350 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:34.323321  7389 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.288620  7109 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.268049  1570 rpcz_store.cc:276] 0709 13:03:32.876827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.459017 (+582190us) service_pool.cc:224] Handling call
0709 13:03:33.487826 (+ 28809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:33.488094 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:33.931910 (+443816us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.145647 (+213737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.145731 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:34.145829 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.145928 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.146121 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49407,"prepare.queue_time_us":488408,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":115,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":12692,"thread_start_us":620265,"threads_started":2,"wal-append.queue_time_us":207566}
I20260709 13:03:34.222628  7407 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.323808  7353 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:03:34.388475  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:34.362298  7385 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.403213  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:34.394734  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2485) took 1081 ms. Trace:
W20260709 13:03:34.395344  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 404693us
I20260709 13:03:34.372728  7334 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:34.396996  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:34.400880  1572 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:34.342298  7387 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:34.366788  7333 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:34.404074  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50432.5us
I20260709 13:03:34.425741  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.411221  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2873) took 1090 ms. Trace:
I20260709 13:03:34.428762  1567 rpcz_store.cc:276] 0709 13:03:33.320598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.200067 (+879469us) service_pool.cc:224] Handling call
0709 13:03:34.213774 (+ 13707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.214291 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:34.214323 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:34.214945 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.330323 (+115378us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:34.331159 (+   836us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.331817 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.331932 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.410899 (+ 78967us) raft_consensus.cc:1685] finished
0709 13:03:34.410925 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.411129 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":223,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":3142,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":252,"spinlock_wait_cycles":101632,"thread_start_us":114839,"threads_started":1,"wal-append.queue_time_us":115367}
I20260709 13:03:34.372213  7398 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:34.404016  1568 rpcz_store.cc:276] 0709 13:03:33.313444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.124916 (+811472us) service_pool.cc:224] Handling call
0709 13:03:34.135392 (+ 10476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:34.135812 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:34.136229 (+   417us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.394291 (+258062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:34.394360 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:34.394421 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.394497 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.394656 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15979,"prepare.queue_time_us":5297,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":23305,"thread_start_us":233572,"threads_started":1}
I20260709 13:03:34.382040  7081 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:34.419502  1574 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:34.415643  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:34.364387  6447 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.351740  7402 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.439910  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:34.443814  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.373008  6923 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:34.413599  7390 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.413432  7291 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.413364  7350 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:34.436167  1570 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:34.366657  7393 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:03:34.371309  7384 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.456100  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:34.447475  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74091.9us
I20260709 13:03:34.437112  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.412868  7334 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.494733  7334 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:34.495046  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58125.9us
I20260709 13:03:34.496575  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.473289  2377 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:34.446238  7187 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.505646  7187 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.506541  7187 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.388844  7183 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:34.443810  7333 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:34.443522  7258 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:34.484797  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.830s (SENT)
W20260709 13:03:34.485940  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.456511  7227 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.529031  7365 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.529196  7350 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.529052  7227 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.531000  7227 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.509795  7358 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:34.537189  7360 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.541414  7258 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.542325  7258 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:34.491175  7416 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.419s	user 0.000s	sys 0.002s
W20260709 13:03:34.493050  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 204814us
I20260709 13:03:34.502336  1723 raft_consensus.cc:1217] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Deduplicated request from leader. Original: 13.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 13:03:34.503494  7384 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:34.510147  2372 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:34.405570  7400 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:03:34.456100  7406 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:34.406267  7375 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:34.526965  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.527999  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:34.554351  1723 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:34.557222  7398 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.558245  7363 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:34.559966  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.565726  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:34.567943  7407 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.569020  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:34.573945  6447 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:34.575528  7373 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.457792  7185 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:03:34.543296  7333 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:34.580299  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.581282  1648 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:34.554231  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 163087us
W20260709 13:03:34.606153  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.589594  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2478) took 1713 ms. Trace:
I20260709 13:03:34.621090  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:34.622386  1643 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:03:34.626938  7277 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:34.626422  7227 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.628434  7382 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:34.629045  7409 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:34.629221  7406 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:34.516906  7352 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:34.603760  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.576788  7083 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.543813  7207 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:34.673353  1642 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:03:34.673301  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.698387  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.662204  1571 rpcz_store.cc:276] 0709 13:03:32.876106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.319967 (+443861us) service_pool.cc:224] Handling call
0709 13:03:33.357275 (+ 37308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:33.705696 (+348421us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:33.947644 (+241948us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.380813 (+433169us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:34.381594 (+   781us) log.cc:844] Serialized 52 byte log entry
0709 13:03:34.383107 (+  1513us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:34.538493 (+155386us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b18004731a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:03:34.547011 (+  8518us) log.cc:844] Serialized 14 byte log entry
0709 13:03:34.547380 (+   369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.547517 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.589189 (+ 41672us) raft_consensus.cc:1685] finished
0709 13:03:34.589231 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.589496 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20404,"prepare.queue_time_us":395792,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":86,"raft.queue_time_us":96175,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":48396,"spinlock_wait_cycles":73780224,"thread_start_us":794558,"threads_started":3,"wal-append.queue_time_us":565682}
W20260709 13:03:34.657994  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 102425us
W20260709 13:03:34.673877  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 177414us
I20260709 13:03:34.701050  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.702224  1646 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:34.704883  7400 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:34.705430  7400 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:34.706369  7400 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.707706  7400 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.700824  7207 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:34.708649  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 223816us
W20260709 13:03:34.710215  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.654s (SENT)
I20260709 13:03:34.710743  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:34.629470  7183 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.721082  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2834) took 1535 ms. Trace:
W20260709 13:03:34.691816  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.721642  1723 rpcz_store.cc:276] 0709 13:03:33.185971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.186177 (+   206us) service_pool.cc:224] Handling call
0709 13:03:33.186747 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.678286 (+1491539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.720718 (+ 42432us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003ab954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:34.720979 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3023629056}
I20260709 13:03:34.723644  7366 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.677366  7393 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:34.709733  7207 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.689703  7425 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.703238  7409 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.696883  7382 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.718093  7421 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.169s
I20260709 13:03:34.730932  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.732113  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:34.687700  6906 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:34.736752  1646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2138) took 1588 ms (client timeout 1928 ms). Trace:
W20260709 13:03:34.744930  1646 rpcz_store.cc:269] 0709 13:03:33.148007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.163064 (+ 15057us) service_pool.cc:224] Handling call
0709 13:03:33.990154 (+827090us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:34.736321 (+746167us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:34.736552 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1551055488}
I20260709 13:03:34.686779  7298 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.689555  7097 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:34.703025  7412 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.686570  7405 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:34.747138  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 193493us
W20260709 13:03:34.753809  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53506.3us
W20260709 13:03:34.748046  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 126957us
I20260709 13:03:34.769795  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:34.743907  7387 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.758810  7438 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.773913  7438 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:34.776010  7375 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.777900  7438 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.760665  7352 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:34.779004  7352 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:34.780053  7352 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.782495  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2867) took 1792 ms. Trace:
I20260709 13:03:34.783032  1575 rpcz_store.cc:276] 0709 13:03:32.990549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.985153 (+994604us) service_pool.cc:224] Handling call
0709 13:03:33.985587 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:33.986102 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:33.986133 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:33.986703 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:03:34.353905 (+367202us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:34.353956 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:34.354088 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:34.782225 (+428137us) raft_consensus.cc:1685] finished
0709 13:03:34.782250 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:34.782422 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7177,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":31013,"thread_start_us":366683,"threads_started":1,"wal-append.queue_time_us":710724}
I20260709 13:03:34.752727  7083 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.759397  7207 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:34.773027  7368 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.808s	user 0.010s	sys 0.004s
W20260709 13:03:34.798977  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45395.2us
W20260709 13:03:34.800617  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.801394  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 113630us
I20260709 13:03:34.804298  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:34.818650  7441 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:03:34.820722  7334 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:34.825125  7429 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:03:34.825601  7227 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.826813  7227 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:34.746215  7393 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.783540  7387 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.828711  7387 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.830315  7393 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:34.831068  7393 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:34.831475  7393 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.831847  7393 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.757073  7298 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.832548  1688 leader_election.cc:400] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16.
I20260709 13:03:34.772682  7366 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.834002  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:34.771023  7109 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.858726  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.854215  7409 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.855074  7406 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.855710  1571 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:03:34.857721  7445 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:34.857621  7375 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:34.799300  7081 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:34.884047  7294 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.000s
W20260709 13:03:34.886529  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54085.3us
W20260709 13:03:34.920993  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151201us
I20260709 13:03:34.891360  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.889690  1571 pending_rounds.cc:85] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:03:34.922485  1571 pending_rounds.cc:107] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:03:34.922832  1571 raft_consensus.cc:2889] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:34.902356  1719 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:34.901024  1723 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:34.917340  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:34.921243  1721 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:34.890761  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 180485us
I20260709 13:03:34.904592  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.860466  7402 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:34.757493  7412 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:34.783888  7437 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
I20260709 13:03:34.932391  1716 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:34.934918  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:34.936419  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.941843  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:34.949330  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.954401  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:34.955333  1718 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:34.916548  7445 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:34.903087  6906 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:03:34.924933  7334 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:34.975818  1573 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:34.976269  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:34.989976  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.982002  1571 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:34.995119  1571 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:34.997323  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:34.991345  1576 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:35.001211  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.002173  1569 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:35.002630  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:35.003448  1575 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.003764  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.004520  1572 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:03:35.006785  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.935427  6447 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:03:35.011097  1576 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:35.012627  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:35.002674  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77048.5us
I20260709 13:03:35.016992  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.980690  7445 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.019279  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.029134  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.029359  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:34.962132  7402 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.943503  7366 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
W20260709 13:03:35.061880  7378 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.940s	user 0.015s	sys 0.001s
I20260709 13:03:35.062458  7378 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.064622  1646 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:34.980993  7396 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.935734  7354 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:03:35.036495  2372 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.084666  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.085590  1576 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:35.009094  7421 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.038995  7081 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:35.040269  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 123171us
I20260709 13:03:35.041162  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:34.930751  7294 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:35.050835  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.052470  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:35.035797  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.689s (SENT)
I20260709 13:03:35.110241  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.101910  7366 consensus_queue.cc:1243] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:35.092187  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:35.072414  1571 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.098510  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:35.108539  1576 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:35.112761  1725 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.070580  1567 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:35.100049  1573 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
W20260709 13:03:35.117280  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.120365  1575 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:35.123201  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.129626  1719 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:35.125986  1569 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:35.139066  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.142239  1571 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:35.146328  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110530us
I20260709 13:03:35.149406  1574 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:35.137353  7412 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:35.151566  7455 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:03:35.146072  7365 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.611s	user 0.015s	sys 0.000s
W20260709 13:03:35.163056  7449 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:03:35.118283  7396 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.164697  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.166474  7396 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:35.167553  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.107542  7421 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:35.138116  7258 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.001s
I20260709 13:03:35.047657  7333 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.211404  2372 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:35.190255  7408 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.214534  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:35.215802  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:35.144392  7354 consensus_queue.cc:1243] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:35.226545  2379 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:35.229439  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 61956.7us
I20260709 13:03:35.190356  7294 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:35.195998  7368 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.185580  7378 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:35.182076  7382 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:35.210006  7407 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:35.252970  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.176s (SENT)
I20260709 13:03:35.257243  1650 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:35.254881  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:35.259752  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:35.176218  7393 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.256978  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.193085  7424 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:35.222919  7366 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:35.281453  1725 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:35.301560  7387 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:35.303802  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44042.1us
I20260709 13:03:35.290191  7358 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.309573  1571 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:03:35.309803  1721 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:35.310906  1571 pending_rounds.cc:85] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
W20260709 13:03:35.311344  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58431us
I20260709 13:03:35.311611  1571 pending_rounds.cc:107] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:03:35.228492  7258 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.315428  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.317109  1569 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:35.317130  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.320626  1567 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:35.232972  7408 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.297587  7382 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:35.312947  1571 raft_consensus.cc:2889] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:35.331744  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.830s (SENT)
I20260709 13:03:35.332726  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.339326  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:35.341449  1567 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:03:35.350346  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 363958us
W20260709 13:03:35.351522  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 370093us
I20260709 13:03:35.358150  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.359037  1572 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:35.348541  7425 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.373782  7393 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:35.365213  1539 leader_election.cc:400] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:35.393556  1539 leader_election.cc:403] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:35.378829  7258 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:35.381330  7408 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:35.363330  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:35.360320  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.389055  1649 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:35.403069  2379 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:35.375257  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:35.429563  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97831.4us
W20260709 13:03:35.433660  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.850s (SENT)
I20260709 13:03:35.413084  1722 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:35.436666  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.437680  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:35.434892  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:35.442363  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 77313.5us
W20260709 13:03:35.439754  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52523.5us
W20260709 13:03:35.444417  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 69351.6us
I20260709 13:03:35.445461  1720 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:35.444985  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.444571  1718 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: [11.9-13.10]
W20260709 13:03:35.452433  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.364960  7452 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.296s	user 0.000s	sys 0.004s
W20260709 13:03:35.463321  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.362149  7421 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:35.464923  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 102235us
W20260709 13:03:35.467909  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 258618us
I20260709 13:03:35.479791  1567 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:35.481304  1645 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:35.434289  7407 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.430559  7294 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15.
I20260709 13:03:35.363018  7333 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:35.484606  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.432307  7387 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:35.489089  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:35.490813  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:35.496599  7365 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:35.497013  1723 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:35.498374  7365 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:35.498837  7365 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:35.473387  7382 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.500074  7465 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:03:35.503561  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:35.508142  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74489.1us
I20260709 13:03:35.509537  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.481971  7393 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:35.486862  7400 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.779s	user 0.020s	sys 0.000s
W20260709 13:03:35.497109  7423 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:03:35.516667  1616 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.466511  7463 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:35.524245  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.506440  7441 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.524756  7354 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:03:35.533965  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.535846  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.550751  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.552836  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:35.515679  7387 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:35.499938  7365 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.526070  7185 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:35.525363  7463 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.569341  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:35.537212  7081 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.509721  7407 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.577559  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:35.550473  7375 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:35.584656  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 48899.5us
I20260709 13:03:35.550776  7294 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:35.522635  7382 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:03:35.563428  7412 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.589661  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:35.585804  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:35.586189  6923 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.586953  7352 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:03:35.599293  7407 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:35.608487  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:35.567297  7378 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.529022  7333 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:03:35.588567  7408 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:35.578332  7406 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:35.608186  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 86445.5us
I20260709 13:03:35.578794  7463 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:35.670796  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60428.1us
I20260709 13:03:35.675618  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:35.605644  7365 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:35.681113  1643 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:35.681113  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.695233  1644 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:03:35.681435  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:35.682315  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.683982  7298 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:35.692674  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:35.695097  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 105479us
I20260709 13:03:35.697381  7425 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:35.698383  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:35.699765  7399 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:35.611110  7408 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.768452  7408 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.768903  2374 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:03:35.769387  2374 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 10
I20260709 13:03:35.769639  2374 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:03:35.769898  2374 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:35.702979  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.703917  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.585466  7465 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:03:35.773563  7465 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:35.774056  7465 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:35.775357  7384 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:35.775194  7465 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.578718  7400 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:03:35.776026  7384 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.778004  7400 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:35.778941  7400 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:35.669723  7406 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:35.779827  7406 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.780809  7469 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:35.781445  7469 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.782352  1648 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:35.610814  7398 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:35.783619  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.783931  7398 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.784196  7469 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:35.784476  7398 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.780117  7384 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:35.671921  7407 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:35.708508  7461 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.454s	user 0.009s	sys 0.000s
W20260709 13:03:35.710170  7445 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.627s	user 0.013s	sys 0.000s
I20260709 13:03:35.718782  2372 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:35.721716  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:35.730470  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 856) took 1165 ms. Trace:
I20260709 13:03:35.731021  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:35.791812  2372 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:35.736583  7358 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:35.733788  7378 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:35.742175  7391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.503s	user 0.010s	sys 0.015s
W20260709 13:03:35.747820  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.750540  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 860) took 1179 ms. Trace:
I20260709 13:03:35.695456  7387 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:35.752233  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:35.754472  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.755649  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:35.801842  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46264.2us
I20260709 13:03:35.802491  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:35.756767  6923 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.670199  7354 consensus_queue.cc:1243] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 15.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:03:35.799893  1650 rpcz_store.cc:276] 0709 13:03:34.571394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.578065 (+  6671us) service_pool.cc:224] Handling call
0709 13:03:34.578435 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.259222 (+680787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.750214 (+490992us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54000915d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:35.750455 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559810432}
I20260709 13:03:35.765887  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.766800  7399 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:35.733338  2373 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:35.784505  1648 raft_consensus.cc:2368] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:03:35.792614  2379 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:35.792285  1645 rpcz_store.cc:276] 0709 13:03:34.564667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.566068 (+  1401us) service_pool.cc:224] Handling call
0709 13:03:34.566416 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.481804 (+915388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.730065 (+248261us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54000b9d54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:35.730365 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102293888}
I20260709 13:03:35.794273  2372 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.796439  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:35.796456  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:35.750990  7185 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:35.806648  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54805us
I20260709 13:03:35.847144  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:35.809731  1569 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:35.815115  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:35.938043  1570 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:35.813941  1650 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:35.790028  1644 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:35.814036  1574 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:35.814699  1567 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:35.813998  1576 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:35.941509  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.847317  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.814096  1572 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:35.810815  2373 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:35.815779  6923 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:35.848543  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46115.3us
W20260709 13:03:35.798956  7391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.560s	user 0.010s	sys 0.015s
W20260709 13:03:35.809082  7478 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 13:03:35.880862  1647 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:35.788094  7408 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.900812  7400 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.901634  7406 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.905112  7399 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.907174  2379 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:35.860042  7387 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:35.861621  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 94280.5us
I20260709 13:03:35.912182  7353 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:35.956056  2373 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:35.916487  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 51659.8us
I20260709 13:03:35.922821  7358 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.809693  7445 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.877358  6447 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:35.820317  7298 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:35.928562  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.931847  2372 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.829174  7469 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:35.961503  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.913718  7424 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:35.984625  7469 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:35.985219  7469 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.971349  1651 raft_consensus.cc:2368] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:03:35.972087  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 863) took 1392 ms. Trace:
I20260709 13:03:35.986438  1649 rpcz_store.cc:276] 0709 13:03:34.579696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.581228 (+  1532us) service_pool.cc:224] Handling call
0709 13:03:34.581833 (+   605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.401057 (+819224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.971830 (+570773us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54001ba3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:35.972026 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765239552}
I20260709 13:03:35.987815  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:35.973356  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:35.974653  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:35.978304  7406 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:35.989877  1650 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.989233  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:36.005921  1650 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:35.980057  2378 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:35.982412  1648 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:36.010000  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:36.010540  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.012080  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.006295  1474 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 6 to 9, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:36.026245  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 51649.8us
I20260709 13:03:36.012817  1571 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:36.011713  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:36.015318  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.010743  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.031337  1645 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:36.017303  7365 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.018970  7408 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:35.957923  7261 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:03:36.021193  7358 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:35.950795  7384 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:03:36.023890  1567 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:36.024806  7399 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.969935  7407 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:03:35.967262  7353 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:36.065987  7353 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:35.979394  7445 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:36.037909  2373 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.049227  7382 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:03:36.078482  7382 consensus_queue.cc:1243] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:36.079378  7425 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:36.046862  7109 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:03:36.080456  7360 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.081080  7360 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.082208  2379 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:36.082665  2379 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:03:36.083556  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 117566us
I20260709 13:03:36.042614  7298 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:36.085975  7298 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.087198  7465 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:36.089303  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.090917  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.055338  6923 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:36.057843  6447 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:36.057482  7365 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.058861  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:36.067423  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.050051  7363 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:35.946738  7476 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:03:36.110862  7363 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:36.071352  1570 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:36.111356  2372 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:03:36.072171  7353 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:36.092840  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.092785  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:36.104516  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.083746  2379 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.067332  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.083848  7387 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.048584  7354 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:03:36.120718  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.122371  1725 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:36.122815  2379 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.107200  7463 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.083556  7360 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:36.112044  2372 pending_rounds.cc:85] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 11
I20260709 13:03:36.091194  7081 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:36.140561  2372 pending_rounds.cc:107] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:03:36.141136  2372 raft_consensus.cc:2889] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:36.125077  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:36.103403  7402 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:36.068545  7352 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:03:36.149155  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:36.133072  1720 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:36.150003  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:36.150449  1718 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:36.133626  1722 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:36.150686  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:36.111191  7476 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:36.111517  7412 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.143894  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:36.149441  1719 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:36.128288  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.151661  1648 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:36.134155  1716 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:03:36.125308  7109 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:36.148643  7277 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:03:36.112012  7363 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.127979  7407 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.163000  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.165284  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.187639  6447 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.193424  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:36.195680  1718 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:03:36.187719  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.171456  1643 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:36.188009  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:36.155812  7476 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:36.200896  1721 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.199959  1725 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:03:36.210906  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.212808  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 49939.5us
I20260709 13:03:36.213764  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:36.156198  2379 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:03:36.161841  1716 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:36.218338  2379 pending_rounds.cc:85] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 9
I20260709 13:03:36.218618  2379 pending_rounds.cc:107] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:03:36.218873  2379 raft_consensus.cc:2889] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:36.223752  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:36.218106  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.226531  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:36.226536  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.197263  7365 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:36.220495  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.216017  7384 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.215744  7334 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:03:36.244992  7334 consensus_queue.cc:1243] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:36.244669  7384 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.246393  7334 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:36.247488  7384 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:36.247601  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:36.248548  1647 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:36.197427  7109 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:36.249460  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:36.250212  1647 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:36.208758  6923 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.210950  7402 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:36.260145  1649 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:03:36.232072  7382 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:03:36.256073  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:36.233698  7438 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:36.229713  7407 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:36.284158  7360 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:36.285573  2376 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.286818  7469 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.220829  7441 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:03:36.203498  7408 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:03:36.264057  7384 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.297396  2376 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:36.313817  2376 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.15, Last appended by leader: 15, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.324241  2376 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.291664  7438 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:36.284225  7358 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:36.306149  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.297351  7360 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:36.323967  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 112497us
I20260709 13:03:36.328625  1647 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:36.306840  1720 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:36.259899  2373 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:36.322230  7407 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:36.321326  7469 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:36.316601  7466 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:03:36.288825  6923 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.341614  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 805) took 1307 ms. Trace:
W20260709 13:03:36.345625  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.302225  7183 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:03:36.346941  7450 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:03:36.332721  7463 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:36.353978  2373 rpcz_store.cc:276] 0709 13:03:35.033905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.034106 (+   201us) service_pool.cc:224] Handling call
0709 13:03:35.058414 (+ 24308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.733196 (+674782us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400457f20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d701c0 00007fc833d71231 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:03:36.341199 (+608003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.341521 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396346240}
I20260709 13:03:36.337648  7360 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.309799  7385 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:03:36.363915  7398 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.579s	user 0.013s	sys 0.000s
I20260709 13:03:36.336613  7393 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:03:36.321331  7384 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:36.410772  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.377166  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:36.353140  7109 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:03:36.317320  7476 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.359957  7258 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:03:36.415503  7393 consensus_queue.cc:1243] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:36.420369  7372 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:36.455267  2376 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 15 index: 15. (term mismatch)
I20260709 13:03:36.463019  2376 pending_rounds.cc:85] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 14
I20260709 13:03:36.463369  2376 pending_rounds.cc:107] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 14.15
I20260709 13:03:36.463636  2376 raft_consensus.cc:2889] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:36.467213  1575 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:36.467427  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.467790  1575 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:36.468294  1572 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:36.476583  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 256130us
I20260709 13:03:36.477089  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.477671  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.479249  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.479018  1575 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.480123  1570 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:36.416899  7408 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:03:36.480306  1575 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.422040  7441 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:03:36.481015  7408 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:36.481801  7408 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:36.482977  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.482817  7408 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.483618  1567 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:36.484191  7408 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.413606  7398 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.488715  1539 leader_election.cc:400] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13.
I20260709 13:03:36.488831  1576 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:03:36.489362  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.489758  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.489233  1539 leader_election.cc:403] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:36.498682  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 865) took 1520 ms. Trace:
I20260709 13:03:36.490669  7407 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:36.474629  7479 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:03:36.494036  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.519981  1570 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:36.424515  7183 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:03:36.470371  7372 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:36.526420  1575 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:03:36.527662  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.499177  1646 rpcz_store.cc:276] 0709 13:03:34.978864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.979129 (+   265us) service_pool.cc:224] Handling call
0709 13:03:35.006304 (+ 27175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.065138 (+ 58834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.498387 (+1433249us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b5400411254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:36.498605 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133528832}
I20260709 13:03:36.501024  1642 raft_consensus.cc:2368] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:36.501747  1569 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:36.503539  7384 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:36.504145  6923 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:36.504884  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.505079  7499 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:03:36.505633  1647 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:36.529552  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:36.508646  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.530926  1569 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:36.509131  7398 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:36.510541  1717 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:03:36.513072  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.476620  7476 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:36.493029  7487 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:03:36.514683  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:36.515700  7492 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:36.517309  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.545456  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:36.521973  1570 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:36.546299  2373 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:36.524325  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.499050  1571 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.542883  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:36.530550  7500 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.077s	user 0.000s	sys 0.002s
I20260709 13:03:36.548993  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.527277  7258 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.551255  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:36.533504  7363 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:03:36.572208  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.589731  1567 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:36.590612  1575 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:03:36.590806  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64998.6us
I20260709 13:03:36.530857  7372 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:36.518671  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 142310us
I20260709 13:03:36.595312  1571 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:36.523376  7407 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13.
I20260709 13:03:36.575224  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.598767  1717 pending_rounds.cc:85] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 8
I20260709 13:03:36.603363  1717 pending_rounds.cc:107] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:03:36.603986  1717 raft_consensus.cc:2889] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:36.599807  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.600586  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:36.612902  2378 raft_consensus.cc:2410] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:03:36.600950  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:36.540486  7385 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.574476  2380 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:36.629343  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.638715  1569 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:36.652805  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.661696  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:36.665132  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 123025us
I20260709 13:03:36.666184  1688 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.575093  7387 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:36.676970  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.686604  1688 leader_election.cc:400] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:03:36.553540  7384 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 13:03:36.627745  7385 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:36.694511  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.702786  1643 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:03:36.624142  7425 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:03:36.704388  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:36.686058  7353 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 13:03:36.623000  7258 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:36.721959  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.724287  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.681480  7393 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:36.649042  7363 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.723174  7365 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:36.669389  7109 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:03:36.742086  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.742332  1646 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:36.701408  7277 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:03:36.766292  1572 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:36.776145  1651 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:36.776574  1570 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:03:36.770632  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102432us
W20260709 13:03:36.755946  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.756342  1648 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:36.780068  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 869) took 1083 ms. Trace:
I20260709 13:03:36.780777  1648 rpcz_store.cc:276] 0709 13:03:35.696327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.200281 (+503954us) service_pool.cc:224] Handling call
0709 13:03:36.200771 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.756050 (+555279us) spinlock_profiling.cc:243] Waited 61.8 ms on lock 0x7b54001859a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d701c0 00007fc833d71231 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:03:36.779634 (+ 23584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.779992 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142043008}
W20260709 13:03:36.783740  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:36.784619  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.790189  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.690025  7387 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.814540  7298 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.815140  7298 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:36.743453  7365 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.771680  1568 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:36.770525  7400 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:36.800766  1572 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 17. Current term is 18.
W20260709 13:03:36.809845  2405 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:32927 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:32927 timed out after 15.000s (SENT)
I20260709 13:03:36.810457  7109 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:36.811983  7258 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:36.819927  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.821784  1568 raft_consensus.cc:2368] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14.
I20260709 13:03:36.791648  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.827193  1574 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
W20260709 13:03:36.823807  7507 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:03:36.824331  7503 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:03:36.824713  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.675s (SENT)
I20260709 13:03:36.832310  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:36.807263  7510 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:36.815361  7387 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:36.754599  7398 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:03:36.838601  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.798163  7425 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:36.822557  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.839952  1572 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:36.844417  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.822010  7365 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:36.849481  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:36.850154  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 72132.5us
I20260709 13:03:36.851087  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.847081  7499 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:36.851590  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:36.845727  1574 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:36.820936  7109 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:36.863396  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:36.852989  1572 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:36.853370  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 188896us
I20260709 13:03:36.856231  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.879495  1688 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:36.882212  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.843641  7489 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:36.892398  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:36.894546  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.894869  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.844184  7425 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:36.904603  1568 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:36.908118  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.910171  1572 raft_consensus.cc:2368] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15.
I20260709 13:03:36.919198  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:36.920404  1718 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:36.921967  1719 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:36.924998  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:36.926012  1723 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:36.938997  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.939891  1574 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 17. Current term is 18.
I20260709 13:03:36.885356  7424 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.944638  7393 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.946080  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2154) took 1268 ms. Trace:
I20260709 13:03:36.970013  1642 rpcz_store.cc:276] 0709 13:03:35.678043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.765435 (+ 87392us) service_pool.cc:224] Handling call
0709 13:03:36.945833 (+1180398us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:36.946015 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595971968}
I20260709 13:03:36.905963  7463 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:36.891096  7489 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:36.884092  7499 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:36.940850  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48544.1us
I20260709 13:03:36.979979  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:36.977630  7489 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:36.980866  7463 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:36.953804  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:36.983445  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:36.984194  1572 raft_consensus.cc:2368] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
I20260709 13:03:36.984723  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.985419  7463 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:36.986703  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:36.988186  1572 raft_consensus.cc:2376] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 16. Re-sending same reply.
I20260709 13:03:36.980321  7499 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:36.991649  1572 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:36.896663  7490 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:36.992645  7490 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.994755  7109 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.996114  7109 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:36.996801  7438 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:36.997807  7438 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:37.006120  7490 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.010210  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:37.010366  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.012777  7385 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.015352  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:36.977600  1642 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:37.040431  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2174) took 1105 ms. Trace:
I20260709 13:03:37.040908  1642 rpcz_store.cc:276] 0709 13:03:35.934709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.976627 (+1041918us) service_pool.cc:224] Handling call
0709 13:03:36.977131 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.040069 (+ 62938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.040329 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:37.020486  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.022084  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2159) took 1111 ms. Trace:
I20260709 13:03:37.023128  1574 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:03:37.023519  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.883s (SENT)
I20260709 13:03:37.024271  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2163) took 1110 ms. Trace:
I20260709 13:03:37.044402  1647 rpcz_store.cc:276] 0709 13:03:35.913757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.354012 (+440255us) service_pool.cc:224] Handling call
0709 13:03:36.505430 (+151418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.529582 (+ 24152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.021291 (+491709us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400410d50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:37.024194 (+  2903us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323775232}
I20260709 13:03:37.047773  1568 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:37.028057  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:36.945660  6923 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:37.049602  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.003s (SENT) [suppressed 10 similar messages]
I20260709 13:03:37.042948  1645 rpcz_store.cc:276] 0709 13:03:35.910596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.263937 (+353341us) service_pool.cc:224] Handling call
0709 13:03:36.293165 (+ 29228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.312270 (+ 19105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.312310 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.321189 (+  8879us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.453228 (+132039us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:36.453771 (+   543us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.019918 (+566147us) spinlock_profiling.cc:243] Waited 888 us on lock 0x7b18003f99a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287
0709 13:03:37.020097 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.020193 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.020267 (+    74us) raft_consensus.cc:1685] finished
0709 13:03:37.021870 (+  1603us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.022026 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80312,"prepare.queue_time_us":251,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":99080,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":45434,"spinlock_wait_cycles":2041600,"thread_start_us":196310,"threads_started":2,"wal-append.queue_time_us":116869}
I20260709 13:03:37.024758  7277 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:37.061275  1688 leader_election.cc:400] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:03:37.024549  7479 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.066375  1642 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:37.062398  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2177) took 1126 ms. Trace:
W20260709 13:03:37.068020  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.069489  1645 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:36.861338  7298 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.036779  7454 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:37.082839  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2180) took 1002 ms. Trace:
I20260709 13:03:37.089329  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.143184  1574 raft_consensus.cc:2368] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14.
W20260709 13:03:37.131784  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.142807  1645 rpcz_store.cc:276] 0709 13:03:36.080088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.068301 (+988213us) service_pool.cc:224] Handling call
0709 13:03:37.069033 (+   732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.082588 (+ 13555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.082779 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.081902  1647 rpcz_store.cc:276] 0709 13:03:35.936306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.045605 (+1109299us) service_pool.cc:224] Handling call
0709 13:03:37.061674 (+ 16069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.061922 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.061957 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.062030 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:37.062110 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.062182 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.062325 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.153915  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.146529  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2179) took 1083 ms. Trace:
W20260709 13:03:37.064878  7360 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.655s	user 0.015s	sys 0.000s
I20260709 13:03:37.064007  7393 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:37.155131  1645 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:37.154727  1642 rpcz_store.cc:276] 0709 13:03:36.063027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.046149 (+983122us) service_pool.cc:224] Handling call
0709 13:03:37.065900 (+ 19751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.080944 (+ 15044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.146465 (+ 65521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.070715  7385 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.160079  1572 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.064224  7363 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:37.060577  7465 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:37.203336  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.206053  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2182) took 1045 ms. Trace:
W20260709 13:03:37.216434  2376 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.037s
W20260709 13:03:37.220024  2376 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.010s	sys 0.038s
I20260709 13:03:37.219832  1645 rpcz_store.cc:276] 0709 13:03:36.160390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.154570 (+994180us) service_pool.cc:224] Handling call
0709 13:03:37.154937 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.205765 (+ 50828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.205994 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:37.220968  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2181) took 1101 ms. Trace:
I20260709 13:03:37.060384  6923 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.224754  1647 rpcz_store.cc:276] 0709 13:03:36.120084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.154470 (+1034386us) service_pool.cc:224] Handling call
0709 13:03:37.220382 (+ 65912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.220581 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.220606 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.220651 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:37.220696 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.220757 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.220886 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:37.225557  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 175885us
I20260709 13:03:37.231448  1724 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:37.070662  7424 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.148s
I20260709 13:03:37.204185  7298 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.245097  1572 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
W20260709 13:03:37.250876  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.238528  1642 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:03:37.258757  1642 pending_rounds.cc:85] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 9
I20260709 13:03:37.268282  1642 pending_rounds.cc:107] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:37.262547  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:37.268607  1642 raft_consensus.cc:2889] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.224449  7465 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.277274  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2184) took 1014 ms. Trace:
I20260709 13:03:37.277792  1642 rpcz_store.cc:276] 0709 13:03:36.263004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.206265 (+943261us) service_pool.cc:224] Handling call
0709 13:03:37.238040 (+ 31775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.276999 (+ 38959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.277217 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:37.280385  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.098029  7479 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.290421  7479 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:37.272604  7372 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.000s
I20260709 13:03:37.300798  7372 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.303097  7372 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:37.303951  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2170) took 1387 ms. Trace:
I20260709 13:03:37.304332  1648 rpcz_store.cc:276] 0709 13:03:35.916190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.811169 (+894979us) service_pool.cc:224] Handling call
0709 13:03:36.811653 (+   484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:36.812098 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.812132 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:36.812926 (+   794us) log.cc:844] Serialized 52 byte log entry
0709 13:03:37.089740 (+276814us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:37.148229 (+ 58489us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.203350 (+ 55121us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.203540 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.203622 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.303715 (+100093us) raft_consensus.cc:1685] finished
0709 13:03:37.303737 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.303891 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72199,"prepare.queue_time_us":58380,"prepare.run_wall_time_us":4652,"raft.queue_time_us":86905,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":38490,"spinlock_wait_cycles":24708480,"thread_start_us":272633,"threads_started":1,"wal-append.queue_time_us":412246}
W20260709 13:03:37.305702  7408 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.821s	user 0.011s	sys 0.001s
I20260709 13:03:37.306859  1648 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:37.311461  6923 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:37.315069  7519 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:03:37.277374  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.317878  1572 raft_consensus.cc:2368] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15.
I20260709 13:03:37.318714  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.319442  1572 raft_consensus.cc:2368] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 9. Current term is 10.
W20260709 13:03:37.321250  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.065s (SENT)
I20260709 13:03:37.206512  7360 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.322620  1648 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:37.283497  7454 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:03:37.242594  7518 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:37.331913  1642 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:37.274423  7465 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.337656  1572 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:37.348477  1648 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:37.333372  7438 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:37.354418  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:37.336855  7372 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:03:37.363194  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:37.355991  7512 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
I20260709 13:03:37.358871  7438 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:37.370030  7515 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:03:37.372691  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.374264  7372 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:37.374691  7372 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:37.375447  7372 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.335311  7518 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.376600  7372 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.377023  1643 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:37.378494  7518 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:37.378728  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2166) took 1464 ms. Trace:
I20260709 13:03:37.379101  1643 rpcz_store.cc:276] 0709 13:03:35.914782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.726668 (+811886us) service_pool.cc:224] Handling call
0709 13:03:37.376791 (+650123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.377440 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.378548 (+  1108us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400133b50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:37.378674 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492685184}
I20260709 13:03:37.382038  7499 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:37.382577  7499 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.386019  7360 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:37.386334  7499 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:37.389138  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.365732  7365 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 13:03:37.390923  7365 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.391606  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.364353  7400 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:37.392320  1568 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.394266  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:37.398804  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.402242  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.405432  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:37.393520  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.413978  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:37.414168  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.406626  1716 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:37.417761  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.418341  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
W20260709 13:03:37.418742  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.867s (SENT)
I20260709 13:03:37.419088  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.419183  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.434319  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.437486  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.429116  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.438354  1716 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:37.445259  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.446681  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.448067  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:37.451063  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.452183  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:37.473802  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.492585  1721 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:37.492897  1720 raft_consensus.cc:2368] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:37.493340  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.432921  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.463948  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.494796  1568 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.496284  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:37.496490  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71539us
I20260709 13:03:37.498171  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.499017  1648 raft_consensus.cc:2410] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:03:37.500068  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71050.4us
I20260709 13:03:37.500794  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.503100  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:37.503295  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.498296  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:37.503829  2372 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:37.505505  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.506417  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.507550  1646 raft_consensus.cc:1217] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Deduplicated request from leader. Original: 11.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 13:03:37.508076  1646 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:03:37.508704  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.509215  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.401345  7400 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:37.510872  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:37.511076  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:37.393095  7365 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:37.512485  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.511081  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:37.518074  2376 raft_consensus.cc:2410] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:03:37.502547  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 139358us
I20260709 13:03:37.514372  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.519527  1643 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:37.514951  2380 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:37.436635  7518 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:37.515516  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.524613  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.526713  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.576197  1648 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:37.576953  2374 raft_consensus.cc:2368] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:37.587445  1722 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
W20260709 13:03:37.512349  7537 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:03:37.502962  7535 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:37.606880  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 101408us
I20260709 13:03:37.512137  7227 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:37.595862  1649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 873) took 1607 ms (client timeout 1746 ms). Trace:
W20260709 13:03:37.690618  1649 rpcz_store.cc:269] 0709 13:03:35.988090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.988964 (+   874us) service_pool.cc:224] Handling call
0709 13:03:37.595726 (+1606762us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54000f5ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:37.595813 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338737920}
I20260709 13:03:37.691648  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.595554  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.693069  1648 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:37.635987  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.694458  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:37.695713  1644 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:37.431674  7465 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:37.688797  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 132599us
I20260709 13:03:37.598423  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.606057  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:37.520628  7400 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:37.703209  1642 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
W20260709 13:03:37.702765  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.701234  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.689924  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:37.700676  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 92799.6us
I20260709 13:03:37.522172  7499 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.401891  7438 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.714190  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.711813  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.716960  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:37.717722  2374 raft_consensus.cc:2393] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:37.719700  1721 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:37.499989  7525 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.204s
I20260709 13:03:37.689862  7535 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.689414  7518 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.728950  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.729772  1568 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.407557  7425 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.732300  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.738507  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:37.738893  7517 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:03:37.740371  7517 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:37.740559  7408 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:37.741129  7408 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:37.742161  7408 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.594820  7540 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.168s
I20260709 13:03:37.721192  7499 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:37.748240  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.749485  7499 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:37.750339  7548 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.750891  7548 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.752992  7548 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:37.753625  7548 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.754062  7548 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.754729  1650 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:37.606972  7536 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.162s
W20260709 13:03:37.756901  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2157) took 1910 ms (client timeout 2184 ms). Trace:
W20260709 13:03:37.757321  1650 rpcz_store.cc:269] 0709 13:03:35.846208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.052227 (+206019us) service_pool.cc:224] Handling call
0709 13:03:37.015097 (+962870us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d340a0 00007fc838d33984 00007fc8322ae6d4 00007fc8322ae662
0709 13:03:37.756202 (+741105us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54001bbcd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:37.756307 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651013248}
I20260709 13:03:37.758858  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.761142  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:37.762712  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:37.765259  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.705144  7385 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:37.766502  1571 raft_consensus.cc:2410] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 11 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:03:37.767714  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.768421  1571 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:37.769531  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.770233  1571 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:37.724133  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.725019  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:37.700198  7490 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.637s	user 0.007s	sys 0.003s
I20260709 13:03:37.726933  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:37.777073  2380 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:37.727895  1716 raft_consensus.cc:2368] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:37.699864  7334 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:37.781277  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:37.782483  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:37.787482  1539 leader_election.cc:400] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:37.791172  1571 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:37.792240  1539 leader_election.cc:403] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:37.776742  7352 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:37.709555  7081 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:03:37.728379  7535 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.718976  7333 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:03:37.794942  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.795092  1718 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:03:37.793282  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.819605  1568 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.820676  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.811368  1539 leader_election.cc:400] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:37.796833  1688 leader_election.cc:400] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 12 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13.
I20260709 13:03:37.774246  7517 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:37.821408  1539 leader_election.cc:403] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:37.822091  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:37.782507  7385 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.828009  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:37.774739  7499 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:37.790006  7334 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.852015  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:37.766073  7445 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:03:37.840222  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.840394  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:37.855014  1721 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:37.852124  1571 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:37.789801  7298 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:37.852828  1717 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:37.764509  7518 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.793965  7352 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:37.818874  7081 consensus_queue.cc:1243] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 16.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:37.868976  1725 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:37.817442  7384 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:37.889070  1643 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:37.897408  1642 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 9. Current term is 11. Ops: [9.11-9.11]
I20260709 13:03:37.889530  1568 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:37.890483  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.837054  7425 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.789958  7490 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:37.892213  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.907377  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.892151  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.892166  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:37.889744  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49391.9us
W20260709 13:03:37.910811  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.911713  1571 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.913918  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.902349  1718 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:37.861647  7334 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:37.860491  7536 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:37.896943  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.903303  7399 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:37.933339  1720 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:37.888767  7183 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:03:37.876931  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2530) took 1089 ms. Trace:
I20260709 13:03:37.860018  7469 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:37.938882  1574 rpcz_store.cc:276] 0709 13:03:36.787241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.144110 (+356869us) service_pool.cc:224] Handling call
0709 13:03:37.205421 (+ 61311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:37.205825 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.205851 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:37.206414 (+   563us) log.cc:844] Serialized 52 byte log entry
0709 13:03:37.356318 (+149904us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:37.532916 (+176598us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.876577 (+343661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.876659 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.876701 (+    42us) raft_consensus.cc:1685] finished
0709 13:03:37.876713 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.876867 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":366010,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":220343,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":10207,"thread_start_us":473318,"threads_started":3,"wal-append.queue_time_us":135828}
I20260709 13:03:37.918902  7548 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:37.920744  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.945070  1568 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:37.922189  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.945710  1717 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 13:03:37.946245  1571 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 10.
I20260709 13:03:37.947111  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.947806  1571 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.864678  7466 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:03:37.927079  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:37.912640  7542 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:03:37.950642  7399 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:37.951640  7399 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:37.952548  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:37.953521  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.955265  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:37.955816  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:37.945981  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:37.931105  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.894304  7372 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:03:37.917498  1718 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:37.894358  7360 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:03:37.932732  7365 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:37.957221  1568 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.926263  7384 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:03:37.927901  7536 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:37.945940  7407 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14.
I20260709 13:03:37.961434  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:37.924994  7352 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:37.964857  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.860651  7499 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:37.972177  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.964975  7540 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.982381  1568 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:37.974491  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:37.962391  7548 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:37.983934  1573 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:37.984673  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:37.989007  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:37.996742  1568 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:37.999137  1646 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:37.970999  7365 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:37.972232  7227 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:38.040585  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 144157us
I20260709 13:03:37.977092  7536 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:38.042747  1568 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:37.974282  7465 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.048534  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2539) took 1199 ms. Trace:
I20260709 13:03:38.051347  1568 rpcz_store.cc:276] 0709 13:03:36.849149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.047133 (+1197984us) service_pool.cc:224] Handling call
0709 13:03:38.047865 (+   732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.048074 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.048101 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.048150 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:38.048226 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.048299 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.048465 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.054170  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:38.029199  7407 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.041677  7365 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.064144  1571 pending_rounds.cc:85] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:03:38.067138  1571 pending_rounds.cc:107] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:03:38.046725  7551 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:38.066996  1568 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:38.067808  1571 raft_consensus.cc:2889] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:38.065054  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93752.8us
I20260709 13:03:38.047538  7454 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:38.046654  7549 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:38.030593  7540 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:38.076136  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 114749us
I20260709 13:03:38.074067  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2540) took 1021 ms. Trace:
I20260709 13:03:38.077178  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:38.076969  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:38.080789  1539 leader_election.cc:400] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:38.081235  1539 leader_election.cc:403] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:03:38.059031  7550 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:38.081588  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 126320us
I20260709 13:03:38.082417  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:38.077629  1568 rpcz_store.cc:276] 0709 13:03:37.052497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.056044 (+1003547us) service_pool.cc:224] Handling call
0709 13:03:38.066797 (+ 10753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.073733 (+  6936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.074001 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.087410  1574 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:38.088160  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:38.089259  1568 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:38.090773  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2541) took 1038 ms. Trace:
I20260709 13:03:38.091148  1568 rpcz_store.cc:276] 0709 13:03:37.052769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.089899 (+1037130us) service_pool.cc:224] Handling call
0709 13:03:38.090240 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.090439 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.090464 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.090508 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:38.090554 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.090607 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.090713 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.078819  7469 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:03:38.073372  7454 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:38.072424  7385 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:38.102349  7499 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.086181  7551 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:38.076960  7556 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:38.098959  7466 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:38.096776  7518 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.125934  7540 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:38.070598  7407 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.100546  7549 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:03:38.141352  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 58641.4us
W20260709 13:03:38.098619  7545 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:03:38.143692  7561 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:38.144796  7558 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.145293  7557 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:38.155206  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:38.158313  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.130115  7490 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.162721  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:38.142745  7556 consensus_queue.cc:1243] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:38.147408  7499 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:38.177500  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:38.185747  7365 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.197348  7454 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.198968  7454 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:38.201733  7365 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:38.202527  7523 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.940s	user 0.028s	sys 0.001s
I20260709 13:03:38.204893  7562 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:38.205586  7562 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:38.206684  7562 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.161269  7557 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:38.210227  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:38.210769  7557 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:38.211136  1574 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:38.212729  7393 consensus_queue.cc:1175] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3b57321031416cba9db75450ef302a" responder_term: 12 status { last_received { term: 12 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 12 index: 11 } }
I20260709 13:03:38.213680  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2175) took 2278 ms. Trace:
I20260709 13:03:38.214221  1651 rpcz_store.cc:276] 0709 13:03:35.935292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.004222 (+1068930us) service_pool.cc:224] Handling call
0709 13:03:37.004654 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.266469 (+261815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.266505 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.266546 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:37.266599 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.213396 (+946797us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400389fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:38.213451 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.213614 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601224064}
I20260709 13:03:38.216542  2374 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:38.216598  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 818) took 1618 ms. Trace:
I20260709 13:03:38.217348  7407 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.169956  7558 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.171087  7550 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:38.162703  7466 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:38.222721  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2544) took 1120 ms. Trace:
I20260709 13:03:38.217196  2373 rpcz_store.cc:276] 0709 13:03:36.598052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.599706 (+  1654us) service_pool.cc:224] Handling call
0709 13:03:36.631012 (+ 31306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.912541 (+281529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.912594 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.913340 (+   746us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.206860 (+293520us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:37.206906 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.206985 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.314313 (+107328us) raft_consensus.cc:1685] finished
0709 13:03:37.314335 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.216324 (+901989us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b54002d14d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:38.216543 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5251,"raft.queue_time_us":10781,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":11143,"spinlock_wait_cycles":71038848,"thread_start_us":544651,"threads_started":2,"wal-append.queue_time_us":399020}
I20260709 13:03:38.242009  1574 rpcz_store.cc:276] 0709 13:03:37.102318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.221035 (+1118717us) service_pool.cc:224] Handling call
0709 13:03:38.221314 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.221489 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.221515 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.221559 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:38.221590 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.221645 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.222664 (+  1019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.224887  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2915) took 1700 ms. Trace:
W20260709 13:03:38.225832  7400 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 13:03:38.203888  7334 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.227123  7558 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.219188  7535 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.229113  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2537) took 1410 ms. Trace:
I20260709 13:03:38.262622  7535 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:38.264657  1573 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:38.261932  1572 rpcz_store.cc:276] 0709 13:03:36.828768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.338368 (+509600us) service_pool.cc:224] Handling call
0709 13:03:37.351234 (+ 12866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.355180 (+  3946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.355211 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.362568 (+  7357us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.707341 (+344773us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:37.707380 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.707446 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.228898 (+521452us) raft_consensus.cc:1685] finished
0709 13:03:38.228921 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.229056 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":3325,"thread_start_us":303304,"threads_started":1,"wal-append.queue_time_us":440799}
I20260709 13:03:38.232194  7466 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.232640  7425 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.232921  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.234541  6447 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:38.281678  6447 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:38.234670  7518 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.236505  7527 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.238060  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60573.2us
I20260709 13:03:38.243880  1574 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:38.243423  1576 rpcz_store.cc:276] 0709 13:03:36.525102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.552429 (+ 27327us) service_pool.cc:224] Handling call
0709 13:03:36.588597 (+ 36168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.589059 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.589091 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.589697 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.048889 (+459192us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:37.048934 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.049051 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.201755 (+1152704us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:38.223205 (+ 21450us) raft_consensus.cc:1685] finished
0709 13:03:38.223227 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.224824 (+  1597us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":17793,"spinlock_wait_cycles":13346688,"thread_start_us":360275,"threads_started":1,"wal-append.queue_time_us":563875}
I20260709 13:03:38.254683  7400 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.248888  7450 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:38.256669  7334 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.264479  7536 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.277627  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 652) took 1724 ms. Trace:
I20260709 13:03:38.304540  1575 rpcz_store.cc:276] 0709 13:03:36.552937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.554639 (+  1702us) service_pool.cc:224] Handling call
0709 13:03:36.590376 (+ 35737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.591172 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.277239 (+1686067us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b540030a050. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:38.277482 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81291648}
I20260709 13:03:38.223704  7499 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:38.286618  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:38.293121  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 669) took 1090 ms. Trace:
I20260709 13:03:38.294435  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2543) took 1222 ms. Trace:
I20260709 13:03:38.246651  7561 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:38.299834  1573 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:38.308255  1569 rpcz_store.cc:276] 0709 13:03:37.071651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.218544 (+1146893us) service_pool.cc:224] Handling call
0709 13:03:38.218850 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.219063 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:38.219381 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.294052 (+ 74671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.294112 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:38.294166 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.294237 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.294366 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2874,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":7377,"thread_start_us":74247,"threads_started":1}
I20260709 13:03:38.253350  7550 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:38.293867  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:38.314474  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:38.315553  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.315450  1574 rpcz_store.cc:276] 0709 13:03:37.202441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.243340 (+1040899us) service_pool.cc:224] Handling call
0709 13:03:38.243697 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.292755 (+ 49058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.293034 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:38.282820  6447 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.319619  1574 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:03:38.320525  6447 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:38.323468  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:38.325052  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:38.325315  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:38.326364  1717 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:03:38.310865  1569 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:38.327423  1721 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:38.327615  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:38.328662  1725 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:38.331673  7365 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.305466  7450 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:38.340742  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:38.341073  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:38.315147  7548 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:38.359019  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2542) took 1303 ms. Trace:
I20260709 13:03:38.360544  1569 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:03:38.362895  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:38.361354  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:38.363715  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.325436  7445 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:38.367897  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:38.326071  7561 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:38.302809  7518 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.295462  7425 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.342993  7558 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.378003  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2903) took 1450 ms. Trace:
I20260709 13:03:38.353358  7354 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:38.321102  7525 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:38.359935  1568 rpcz_store.cc:276] 0709 13:03:37.055896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.092129 (+1036233us) service_pool.cc:224] Handling call
0709 13:03:38.092823 (+   694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:38.093194 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:38.131707 (+ 38513us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.317567 (+185860us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.358390 (+ 40823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:38.358639 (+   249us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:38.358712 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.358809 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.358950 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56654,"prepare.queue_time_us":42259,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":85,"raft.queue_time_us":73059,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":154,"thread_start_us":151489,"threads_started":2,"wal-append.queue_time_us":183490}
I20260709 13:03:38.384166  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 653) took 1761 ms. Trace:
I20260709 13:03:38.386001  1570 rpcz_store.cc:276] 0709 13:03:36.623332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.624619 (+  1287us) service_pool.cc:224] Handling call
0709 13:03:36.776186 (+151567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.794800 (+ 18614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.383894 (+1589094us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540027d650. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:38.384079 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347344256}
I20260709 13:03:38.378580  1724 rpcz_store.cc:276] 0709 13:03:36.927691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.060301 (+132610us) service_pool.cc:224] Handling call
0709 13:03:37.060667 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.241235 (+180568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.377643 (+1136408us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:38.377920 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392026368}
I20260709 13:03:38.384979  7354 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.391296  7354 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:38.332269  7365 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.371891  7548 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:38.315012  7527 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.407665  2379 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:38.411334  7360 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:38.411633  7354 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.412135  7360 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:38.413475  7360 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.414893  7360 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:38.416913  7561 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.426097  7354 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.441937  7536 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.442898  7536 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.457648  7425 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.458617  7425 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:38.459158  7425 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.459676  7425 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:38.443511  7536 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.467483  1539 leader_election.cc:400] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:38.467994  1539 leader_election.cc:403] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:38.471464  7551 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:03:38.504966  7400 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:38.500959  7564 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:38.506924  7527 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.404865  7499 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:38.508620  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.384040  7550 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.509713  7548 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.512096  7365 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.403416  7525 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.500797  7549 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.423393  7557 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:38.618084  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.523321  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:38.619328  1649 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:38.401288  7558 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:38.522913  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.624480  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.190791   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:38.634549  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:38.644935  1651 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:38.636483  1648 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 13:03:38.661185  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2464) took 1054 ms. Trace:
I20260709 13:03:38.661962  1648 rpcz_store.cc:276] 0709 13:03:37.606523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.706604 (+100081us) service_pool.cc:224] Handling call
0709 13:03:37.706940 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.636122 (+929182us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540018a4a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d701c0 00007fc833d71231 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:03:38.660656 (+ 24534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.661087 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477156096}
W20260709 13:03:38.665124  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 335001us
W20260709 13:03:38.666370  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.669021  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2185) took 2364 ms. Trace:
I20260709 13:03:38.668941  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:38.670909  1568 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:38.672178  1574 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:38.677641  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:38.576848  7261 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.446611  7450 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.678932  1648 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:38.682658  2373 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:38.606837  7527 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.696951  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:38.638612  7400 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.719020  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 44915.9us
I20260709 13:03:38.608235  7365 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:38.730885  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:38.715827  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 354467us
W20260709 13:03:38.738478  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.691674  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:38.664436  7549 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.681810  7398 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.744539  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:38.745859  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:38.745792  1651 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:38.757886  1648 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:38.758518  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:38.760224  1723 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:38.768040  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:38.779817  1474 catalog_manager.cc:5697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 2 to 9, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:38.784297  1474 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 5 to 8, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:38.669879  1645 rpcz_store.cc:276] 0709 13:03:36.304535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.222038 (+917503us) service_pool.cc:224] Handling call
0709 13:03:37.222434 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.222844 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.222886 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.230835 (+  7949us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.509245 (+278410us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:37.857488 (+348243us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.605467 (+747979us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18004525c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:38.605739 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.605841 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:38.605902 (+    61us) raft_consensus.cc:1685] finished
0709 13:03:38.668250 (+ 62348us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:38.668957 (+   707us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17212,"prepare.queue_time_us":423135,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":753495,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":88744,"spinlock_wait_cycles":23016192,"thread_start_us":394958,"threads_started":2,"wal-append.queue_time_us":603513}
W20260709 13:03:38.797293  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 615094us
I20260709 13:03:38.730439  7445 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:38.794865  7398 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:38.801934  7445 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.803342  7398 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:38.805601  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:38.806710  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:38.889088  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.798137  7556 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:38.730955  7261 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.737890  7527 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.823410  7536 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.741364  7400 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:38.922835  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 231173us
I20260709 13:03:38.744433  7365 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:38.737020  7354 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 13:03:38.753410  7384 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:38.938081  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:38.688289  7407 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:03:38.797274  7469 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.949124  6923 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:38.941884  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 186937us
W20260709 13:03:38.933215  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 124805us
I20260709 13:03:38.950167  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:38.953540  1651 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:38.966890  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:38.971577  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:38.973731  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:38.976954  1646 raft_consensus.cc:1240] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:03:39.027911  1648 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.027962  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.028522  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.028856  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.029659  1646 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:38.977988  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.030328  1644 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:39.030704  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:39.031133  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:38.918988  7298 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:38.945611  7387 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:38.941951  6447 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.948513  7421 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:38.946151  7556 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:38.967803  7466 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.010s
I20260709 13:03:38.945189  7562 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:39.023886  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.048470  1650 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:38.982026  7109 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.027488  1645 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.029526  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:38.960220  7563 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.214s	user 0.000s	sys 0.005s
I20260709 13:03:38.958127  7378 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:39.032795  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.062408  1642 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.024950  1723 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:39.035948  6447 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.039220  1648 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:39.065341  1648 pending_rounds.cc:85] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 5
I20260709 13:03:39.065680  1648 pending_rounds.cc:107] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:03:39.065987  1648 raft_consensus.cc:2889] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:39.066828  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2538) took 2220 ms. Trace:
I20260709 13:03:39.017275  7441 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.018055  7183 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:39.067711  7183 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.026234  6923 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.067203  1571 rpcz_store.cc:276] 0709 13:03:36.846199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.045824 (+1199625us) service_pool.cc:224] Handling call
0709 13:03:38.063876 (+ 18052us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:38.653473 (+589597us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18003b1ac0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d66bfb 00007fc833d7164b
0709 13:03:38.653730 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.653782 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:38.676484 (+ 22702us) log.cc:844] Serialized 52 byte log entry
0709 13:03:38.915995 (+239511us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:38.916053 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.916163 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.066535 (+150372us) raft_consensus.cc:1685] finished
0709 13:03:39.066557 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.066767 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52691,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":12371,"spinlock_wait_cycles":7435648,"thread_start_us":247432,"threads_started":2,"wal-append.queue_time_us":305425}
I20260709 13:03:39.017385  7518 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.054425  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.065474  6447 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:38.945325  7352 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:03:39.083447  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2550) took 1175 ms. Trace:
W20260709 13:03:39.043167  7466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.000s	sys 0.011s
I20260709 13:03:39.083873  1573 rpcz_store.cc:276] 0709 13:03:37.908393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.314886 (+406493us) service_pool.cc:224] Handling call
0709 13:03:38.331086 (+ 16200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.338747 (+  7661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.338796 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.366321 (+ 27525us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.722085 (+355764us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:38.722150 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.722245 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.083200 (+360955us) raft_consensus.cc:1685] finished
0709 13:03:39.083222 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.083384 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10817,"raft.queue_time_us":10779,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":16061,"thread_start_us":300098,"threads_started":1,"wal-append.queue_time_us":671060}
I20260709 13:03:39.086031  6447 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:39.087502  7489 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:39.042141  7556 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.092702  1651 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:39.021051  7384 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.067478  7441 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.060014  7555 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:39.033413  7469 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.080271  6923 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.067209  7365 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.105808  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.102830  7549 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.117847  7558 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:39.048427  7421 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.103699  7576 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:39.113075  7298 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.101487  7518 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.142341  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.069715  7109 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.078557  1650 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:39.143556  7580 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:39.062558  7562 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.146435  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2188) took 2771 ms. Trace:
I20260709 13:03:39.127274  7525 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.163015  1642 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:39.165027  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.165939  1642 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
W20260709 13:03:39.167488  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.172000  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:39.133459  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.104756  7573 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.156898  1647 rpcz_store.cc:276] 0709 13:03:36.375478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.230692 (+855214us) service_pool.cc:224] Handling call
0709 13:03:37.261129 (+ 30437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.264252 (+  3123us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:37.948078 (+683826us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b1800274a20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:03:38.059617 (+111539us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.718550 (+658933us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.720242 (+  1692us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.720544 (+   302us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:38.951966 (+231422us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.952404 (+   438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.952487 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.146130 (+193643us) raft_consensus.cc:1685] finished
0709 13:03:39.146169 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.146362 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":649793,"prepare.queue_time_us":630074,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":599954,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":24740,"spinlock_wait_cycles":98644736,"thread_start_us":390411,"threads_started":3,"wal-append.queue_time_us":123773}
I20260709 13:03:39.188613  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:39.185232  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:39.191704  2379 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.193483  1650 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:39.147259  7576 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.143240  7365 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.201454  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.211242  1719 raft_consensus.cc:2410] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:39.208873  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:39.218259  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 187555us
I20260709 13:03:39.219623  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:39.222150  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.223007  1721 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
W20260709 13:03:39.217099  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 189840us
W20260709 13:03:39.196733  7438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.003s	sys 0.018s
I20260709 13:03:39.240916  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.127636  7555 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.183511  7479 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:39.179589  7580 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:39.208523  7372 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.120s	user 0.018s	sys 0.045s
W20260709 13:03:39.244263  7438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.005s	sys 0.018s
W20260709 13:03:39.284590  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 249794us
I20260709 13:03:39.157634  7518 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.280434  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.280407  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.210196  7576 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.199405  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:39.261739  7563 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:03:39.289273  1718 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:39.293726  2379 raft_consensus.cc:2410] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:39.294288  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:39.276764  7372 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.026s	sys 0.045s
I20260709 13:03:39.302469  1725 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:39.263131  7463 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:39.314581  1725 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:39.297189  1717 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:39.299495  1724 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:39.197881  7573 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:39.300757  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 325623us
I20260709 13:03:39.307204  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:39.295351  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 96007.1us
I20260709 13:03:39.268173  7261 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:39.319228  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 352365us
I20260709 13:03:39.289311  7527 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.315291  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2929) took 1510 ms. Trace:
I20260709 13:03:39.284641  7535 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.327508  2373 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.327669  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:39.237970  7365 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.290320  7400 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:39.344445  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52905.4us
I20260709 13:03:39.313776  7183 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.325709  7438 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.302500  7596 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.348630  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:39.289604  7586 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:39.336374  1572 rpcz_store.cc:276] 0709 13:03:37.805005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.313365 (+508360us) service_pool.cc:224] Handling call
0709 13:03:38.329146 (+ 15781us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:38.358679 (+ 29533us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:38.358730 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:38.359581 (+   851us) log.cc:844] Serialized 52 byte log entry
0709 13:03:38.681119 (+321538us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:38.681176 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.681320 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.315114 (+633794us) raft_consensus.cc:1685] finished
0709 13:03:39.315131 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.315242 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10342,"raft.queue_time_us":12567,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":16597,"thread_start_us":281263,"threads_started":1,"wal-append.queue_time_us":666868}
I20260709 13:03:39.315867  7479 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:39.373749  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:39.374362  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.374862  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.410775  1574 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:39.416029  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:39.411613  1576 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:39.415894  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.411700  1650 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:39.362490  2373 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:39.345938  7555 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:39.402511  1645 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:39.381721  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.377990  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:39.446851  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.358677  7561 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.503613  2372 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:39.513468  2378 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:39.371167  7527 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:39.503760  1567 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 11.
I20260709 13:03:39.410593  7479 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.418994  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.506919  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.325862  7463 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.520421  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.525079  7445 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.419504  7438 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:39.444265  7579 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 13:03:39.372447  7525 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.532537  1573 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:39.590415  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.418509  7509 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:39.364019  7535 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:39.530747  7425 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.063s	user 0.013s	sys 0.000s
I20260709 13:03:39.616875  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:39.529893  7454 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.330s	user 0.018s	sys 0.000s
I20260709 13:03:39.572150  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:39.621721  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49670.2us
I20260709 13:03:39.622740  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:39.521250  7589 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.526777  7527 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:39.624433  7527 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.573450  7584 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:39.531065  7469 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:39.562086  7466 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.544840  2380 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:39.592367  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.587867  1650 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:39.631691  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71328.8us
I20260709 13:03:39.611713  1574 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:03:39.562044  7384 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:39.620556  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 400788us
I20260709 13:03:39.628926  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.575032  7421 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.630895  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.548202  7479 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.371593  7580 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.637145  1724 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:03:39.530032  7441 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.633441  1720 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:03:39.641536  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.643347  1642 raft_consensus.cc:2376] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 13. Re-sending same reply.
I20260709 13:03:39.654379  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.655821  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:39.575950  7558 consensus_queue.cc:1059] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
W20260709 13:03:39.658751  7575 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:03:39.659138  7558 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:39.659494  7558 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.659862  7558 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.661548  7573 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.662325  7573 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.662786  7573 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:39.665057  7606 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:03:39.666288  7593 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:03:39.543795  7561 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.587596  7109 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.670501  7561 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:39.633474  7499 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:39.672777  7450 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:39.679915  7581 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.402s	user 0.000s	sys 0.001s
I20260709 13:03:39.648795  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:39.578207  7600 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:03:39.658375  7463 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.443668  7596 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.683303  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2927) took 1901 ms. Trace:
I20260709 13:03:39.616549  7445 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.630638  7509 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:39.703752  7509 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.687325  1575 rpcz_store.cc:276] 0709 13:03:37.782133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.305456 (+523323us) service_pool.cc:224] Handling call
0709 13:03:38.305876 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.314346 (+  8470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:38.314382 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:38.319311 (+  4929us) log.cc:844] Serialized 26 byte log entry
0709 13:03:38.679552 (+360241us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:38.796720 (+117168us) log.cc:844] Serialized 14 byte log entry
0709 13:03:38.796961 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:38.797051 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:39.660193 (+863142us) raft_consensus.cc:1685] finished
0709 13:03:39.660213 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.683208 (+ 22995us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112921,"prepare.run_cpu_time_us":4095,"prepare.run_wall_time_us":111939,"raft.queue_time_us":5178,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":7433,"thread_start_us":346899,"threads_started":1,"wal-append.queue_time_us":737387}
I20260709 13:03:39.681582  7562 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:39.732149  7549 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.640547  7557 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.615096  7525 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.702586  7445 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:39.723341  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.731998  7596 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:39.702481  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:39.735850  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53575.1us
I20260709 13:03:39.743117  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:39.738149  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.631359  7425 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.744586  1574 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 6
I20260709 13:03:39.745851  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.751346  1574 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 16.7
I20260709 13:03:39.751750  1574 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:39.752000  1719 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:39.747313  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:39.747596  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.747746  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:39.752905  1717 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:39.757952  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.758365  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:39.761821  7365 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:39.762502  7607 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:03:39.641870  7584 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.742611  7561 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:39.765908  7558 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.689625  7387 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:39.771974  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.773813  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:39.773969  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151289us
I20260709 13:03:39.708985  7518 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.637764  7589 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.793128  1570 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:39.778491  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:39.778771  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:39.781203  7558 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.785166  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.812469  1573 raft_consensus.cc:2393] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:39.644053  7466 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.788278  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:39.732806  7549 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.801590  2379 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:39.804565  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:39.749019  7400 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.799093  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:39.828326  1572 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:39.765939  7584 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.816156  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:39.797667  7606 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:03:39.823355  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.810094  7365 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:39.791275  7598 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:39.798344  7499 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:39.766477  7561 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.836416  7400 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.837631  7582 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:39.840451  7589 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.832861  7558 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:39.843032  6923 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.003s
I20260709 13:03:39.814935  7580 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:39.825999  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.819584  7573 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.822633  7421 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:39.790246  7600 consensus_queue.cc:1059] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:03:39.878623  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:39.882499  7558 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:39.879558  1647 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:39.825816  7551 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:39.880973  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:39.884502  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51455.4us
I20260709 13:03:39.896489  7454 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.847205  7365 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:39.872654  7556 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.000s
I20260709 13:03:39.852677  7393 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:39.824628  7445 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:39.851408  7598 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:39.893306  6923 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:39.914669  7558 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.932488  7445 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:39.965467  7613 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:03:39.968497  7611 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:03:39.969435  7463 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:39.969902  7463 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:39.970081  7400 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.971153  7425 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:39.970858  7463 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.965492  7445 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:39.974078  7624 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:39.929708  7489 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.842s	user 0.011s	sys 0.003s
I20260709 13:03:39.977427  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:39.980736  2376 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:39.980664  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.981853  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.980252  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:39.902433  7408 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:03:39.933037  7365 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:39.932643  7598 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:40.021809  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 133758us
I20260709 13:03:39.972174  7463 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:39.924907  7438 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:39.910560  7333 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:03:40.029978  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:39.937381  7556 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.048985  7549 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.049995  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.051635  2372 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:40.052435  7625 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:03:40.052629  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.053022  2375 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:40.054332  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.054744  6923 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.055038  2371 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:40.055855  7548 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.056408  7548 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.057379  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:40.057547  7618 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:40.057992  7438 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.058470  7438 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.058845  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.059150  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2206) took 1080 ms. Trace:
I20260709 13:03:40.060520  7438 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.061086  7438 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:40.061888  7438 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:40.077590  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.004040  7551 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:39.933517  7454 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:39.930387  7421 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:03:40.066239  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:40.066746  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 185821us
I20260709 13:03:40.088923  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.091302  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2209) took 1068 ms. Trace:
W20260709 13:03:40.092638  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.091683  1651 rpcz_store.cc:276] 0709 13:03:39.022613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.153163 (+130550us) service_pool.cc:224] Handling call
0709 13:03:39.153475 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.325789 (+172314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.325819 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:39.349475 (+ 23656us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.546668 (+197193us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:39.546724 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.547035 (+   311us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.091103 (+544068us) raft_consensus.cc:1685] finished
0709 13:03:40.091125 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.091245 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231132,"raft.queue_time_us":226735,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":50591,"thread_start_us":100621,"threads_started":1,"wal-append.queue_time_us":516249}
I20260709 13:03:40.094473  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.095079  1651 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:40.097036  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.098065  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.098611  1648 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:40.103360  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.070554  1722 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:40.078459  1647 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:40.107090  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.107482  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.107735  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:40.108379  1688 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.108640  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.065361  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2201) took 1089 ms. Trace:
W20260709 13:03:40.061020  7592 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 13:03:40.083773  7629 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:03:40.084918  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.065423  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.103549  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.059571  1644 rpcz_store.cc:276] 0709 13:03:38.978647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.039303 (+ 60656us) service_pool.cc:224] Handling call
0709 13:03:39.039655 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.244968 (+205313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.267474 (+ 22506us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.292627 (+ 25153us) log.cc:844] Serialized 52 byte log entry
0709 13:03:39.633289 (+340662us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:39.648444 (+ 15155us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.648787 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.648967 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.649035 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.058925 (+409890us) raft_consensus.cc:1685] finished
0709 13:03:40.058948 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.059091 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":387037,"prepare.queue_time_us":53321,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":260648,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":17046,"thread_start_us":232720,"threads_started":1,"wal-append.queue_time_us":411424}
I20260709 13:03:40.104815  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.125458  1650 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:40.126854  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.125625  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:39.974704  7624 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:40.123894  7576 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.000s
I20260709 13:03:40.086434  7400 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.004320  7583 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.350s
I20260709 13:03:40.109400  1643 rpcz_store.cc:276] 0709 13:03:38.976475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.976609 (+   134us) service_pool.cc:224] Handling call
0709 13:03:38.976901 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:38.980881 (+  3980us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:39.209970 (+229089us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.678281 (+468311us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b1800467560. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a
0709 13:03:39.678468 (+   187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:39.678936 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.679088 (+   152us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:39.679306 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.679455 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.679518 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.050742 (+371224us) raft_consensus.cc:1685] finished
0709 13:03:40.050782 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.065301 (+ 14519us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":286461,"prepare.queue_time_us":306424,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":99,"raft.queue_time_us":9526,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":11251,"spinlock_wait_cycles":34786176,"thread_start_us":230750,"threads_started":2,"wal-append.queue_time_us":152443}
I20260709 13:03:40.148046  7551 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.145891  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.089686  7425 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.142565  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.131788  7360 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:40.130069  7553 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:40.129230  7561 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.120959  7466 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.109659  7525 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.109805  7549 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.117015  7464 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.116784  7556 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.179723  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.178758  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.144085  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.121251  7610 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
W20260709 13:03:40.199613  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53823.3us
I20260709 13:03:40.239328  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:40.240114  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60453.4us
I20260709 13:03:40.241118  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:40.253396  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 109477us
I20260709 13:03:40.254132  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.175575  7576 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.256848  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.215126  7378 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:03:40.258747  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:40.259622  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 117099us
I20260709 13:03:40.260672  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.261970  2376 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 11.
W20260709 13:03:40.216758  7527 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.002s
I20260709 13:03:40.210248  7584 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.204859  7360 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.241025  7525 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.290848  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.291206  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:40.210489  7550 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.187116  7400 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.220194  7561 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.218676  7623 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.207013  7562 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:40.298708  7609 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:40.287563  7412 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.270955  7372 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:40.303728  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:40.310726  1720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2578) took 1987 ms (client timeout 1795 ms). Trace:
I20260709 13:03:40.289377  7536 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:40.331036  1720 rpcz_store.cc:269] 0709 13:03:38.323037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:38.323220 (+   183us) service_pool.cc:224] Handling call
0709 13:03:40.310459 (+1987239us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400232154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:40.310598 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3009487104}
I20260709 13:03:40.319608  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.210355  7425 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.340867  7509 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:03:40.343219  7558 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:40.344561  7632 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
I20260709 13:03:40.315297  7584 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.295285  7445 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.347002  7584 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.347426  7445 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:40.303089  7628 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:03:40.365260  6447 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.289103  7466 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:03:40.347704  7490 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:40.309644  7400 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.351349  1647 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:40.352823  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.349004  2375 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:03:40.351701  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.365989  6447 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.353292  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.352558  1650 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:40.207671  7438 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.300846  7360 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.298128  7549 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:40.309593  7624 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.309732  7506 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.332449  7550 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.332860  7623 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.303898  7553 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.308199  7407 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.392009  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.393541  1644 raft_consensus.cc:2368] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:40.393527  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.412058  1650 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.369601  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:40.303273  7621 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:40.350449  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.367275  7227 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:03:40.335963  7536 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.400483  7564 raft_consensus.cc:3055] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:40.395294  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.462458  1647 raft_consensus.cc:2376] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16. Re-sending same reply.
I20260709 13:03:40.377009  7589 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:03:40.354291  7609 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.289951  7464 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.302121  7450 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:40.374322  7639 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:40.488394  7639 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:40.489984  7550 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.489552  7639 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.300503  7527 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.491539  7550 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:40.491734  7580 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.000s
I20260709 13:03:40.492069  7550 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.492595  7527 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.347627  7584 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.493523  7584 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.494197  7550 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.495451  7584 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.496235  2373 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 18: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.372545  7454 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.497201  7450 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:03:40.401667  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 82107.3us
I20260709 13:03:40.500061  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:40.501130  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.403720  6447 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:40.497212  7479 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.807s	user 0.010s	sys 0.000s
I20260709 13:03:40.502810  7479 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.504233  2371 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.407891  7558 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.332613  7561 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.505296  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2907) took 1218 ms. Trace:
I20260709 13:03:40.505594  7479 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.506866  7561 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.414697  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.505748  1716 rpcz_store.cc:276] 0709 13:03:39.287277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.288036 (+   759us) service_pool.cc:224] Handling call
0709 13:03:39.288516 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.359122 (+ 70606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.359179 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:39.359824 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.789465 (+429641us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:39.835067 (+ 45602us) log.cc:844] Serialized 14 byte log entry
0709 13:03:39.884365 (+ 49298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.884475 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.505101 (+620626us) raft_consensus.cc:1685] finished
0709 13:03:40.505124 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.505240 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61153,"prepare.queue_time_us":50815,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":156385,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":49350,"thread_start_us":315084,"threads_started":2,"wal-append.queue_time_us":488171}
I20260709 13:03:40.412982  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.507840  1644 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:40.508898  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.510190  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.511018  1644 raft_consensus.cc:2410] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:40.511888  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.454062  1643 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:40.513088  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.514271  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.515221  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.335543  7372 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.515571  7412 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.515900  7372 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.519655  7548 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.523514  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2914) took 1074 ms. Trace:
W20260709 13:03:40.400962  7642 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:03:40.524021  1718 rpcz_store.cc:276] 0709 13:03:39.448713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.448896 (+   183us) service_pool.cc:224] Handling call
0709 13:03:39.681990 (+233094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:39.682439 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.682480 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:39.688547 (+  6067us) log.cc:844] Serialized 26 byte log entry
0709 13:03:39.908445 (+219898us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:39.908488 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.908552 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.130044 (+221492us) raft_consensus.cc:1685] finished
0709 13:03:40.130077 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.523196 (+393119us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54003516d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:40.523439 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6241,"raft.queue_time_us":113,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":17879,"spinlock_wait_cycles":398056704,"thread_start_us":98561,"threads_started":1,"wal-append.queue_time_us":371747}
I20260709 13:03:40.394732  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.460729  7536 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.533326  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.534535  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.535089  1648 raft_consensus.cc:2376] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15. Re-sending same reply.
I20260709 13:03:40.535964  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.461468  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.537089  1651 raft_consensus.cc:2368] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:40.537961  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.539032  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.539767  1651 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:40.463425  7646 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:40.463510  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.541517  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.542713  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.543320  1647 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:40.395576  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.468266  7643 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.459614  7549 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.544798  1642 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.545138  7549 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.546917  1650 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:40.547560  7549 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.470767  7623 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:40.471854  7620 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:03:40.549649  7633 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.581s	user 0.005s	sys 0.000s
I20260709 13:03:40.550673  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2204) took 1573 ms. Trace:
I20260709 13:03:40.551149  1649 rpcz_store.cc:276] 0709 13:03:38.977830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.029521 (+ 51691us) service_pool.cc:224] Handling call
0709 13:03:39.029861 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.217264 (+187403us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:39.217319 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.246715 (+ 29396us) log.cc:844] Serialized 52 byte log entry
0709 13:03:39.647321 (+400606us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:39.647369 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.647440 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.550402 (+902962us) raft_consensus.cc:1685] finished
0709 13:03:40.550445 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.550609 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288904,"raft.queue_time_us":211914,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":20208,"thread_start_us":353061,"threads_started":1,"wal-append.queue_time_us":1026889}
I20260709 13:03:40.544117  7536 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.402181  7365 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.555117  7384 raft_consensus.cc:3055] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:40.555574  7384 raft_consensus.cc:740] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:40.555899  7536 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.556748  7365 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.556694  7372 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.558620  2374 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.558686  7372 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.559904  1720 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:40.559963  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.560832  1721 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:40.561650  1723 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:40.455475  7407 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.564595  7407 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.463558  7650 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.565375  7650 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:40.566264  7635 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:03:40.566340  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2207) took 1587 ms. Trace:
I20260709 13:03:40.567863  7650 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.566926  1646 rpcz_store.cc:276] 0709 13:03:38.979190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.054418 (+ 75228us) service_pool.cc:224] Handling call
0709 13:03:39.054856 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:39.270311 (+215455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.270349 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:39.271054 (+   705us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.316761 (+1045707us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b180033e8c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:40.564265 (+247504us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:40.564860 (+   595us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.565962 (+  1102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.566050 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.566088 (+    38us) raft_consensus.cc:1685] finished
0709 13:03:40.566168 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.566290 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":342156,"prepare.queue_time_us":146,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":264535,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":34773,"spinlock_wait_cycles":13495680,"thread_start_us":247310,"threads_started":1,"wal-append.queue_time_us":1211797}
I20260709 13:03:40.482861  7576 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.484717  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:40.569633  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 84969.8us
I20260709 13:03:40.571033  1474 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.462244  7564 raft_consensus.cc:740] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:40.573064  7564 consensus_queue.cc:260] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.574486  7564 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.487926  7609 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.577101  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.578130  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.498483  7450 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.580585  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:40.584061  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.590855  6923 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:40.591341  6923 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.591722  6923 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.594017  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.596917  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.597595  1570 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:40.598161  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:40.598328  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.598754  1568 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.598855  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.348966  7585 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:03:40.599701  1571 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:40.599776  7585 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:40.600261  7585 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:40.600467  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.601378  1716 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:40.529559  7653 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.601135  7535 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:40.602623  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:40.532016  1722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2563) took 2937 ms (client timeout 1778 ms). Trace:
W20260709 13:03:40.410805  7634 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:03:40.597555  1572 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:40.599020  7517 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:40.603768  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:40.604192  7517 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:40.604506  1721 raft_consensus.cc:2376] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13. Re-sending same reply.
I20260709 13:03:40.601158  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.605189  7517 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.606057  1573 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:40.606420  7517 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.607136  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 822) took 1217 ms. Trace:
W20260709 13:03:40.603310  1722 rpcz_store.cc:269] 0709 13:03:37.594902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.598101 (+  3199us) service_pool.cc:224] Handling call
0709 13:03:40.531760 (+2933659us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b5400087854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:40.531905 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5289740672}
I20260709 13:03:40.602262  7653 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.529973  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.610162  1645 raft_consensus.cc:2376] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16. Re-sending same reply.
I20260709 13:03:40.611455  7653 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.611707  7421 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:40.607504  2379 rpcz_store.cc:276] 0709 13:03:39.389803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.390128 (+   325us) service_pool.cc:224] Handling call
0709 13:03:39.390572 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.820733 (+430161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.606902 (+786169us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54000a87d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:40.607080 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944325888}
I20260709 13:03:40.603343  1575 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:40.601109  7585 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.597442  7464 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.616510  7639 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.621551  7509 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:40.634518  7585 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.634822  7509 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:40.635802  7509 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.637189  7509 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.618024  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.620709  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.639919  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.621233  7609 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.621866  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.527405  7598 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.525704  7551 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:40.644598  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52292.7us
I20260709 13:03:40.645370  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:40.632628  7421 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:40.646129  1571 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.520174  7425 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.646806  7421 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:40.639820  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.640712  1567 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:40.621529  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.649471  1644 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:40.650128  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.641844  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.651208  1576 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:40.652175  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:40.643188  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.655639  1567 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:40.658970  1567 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:40.633386  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.659893  1575 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:40.632740  6923 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.645442  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.611150  7384 consensus_queue.cc:260] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.650698  2379 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.663282  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.666142  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.675004  1571 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:03:40.679222  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.679838  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.678133  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.680446  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.680477  2373 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:40.681437  1571 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:40.682502  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.679840  2378 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:40.684121  1649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:40.684753  1649 raft_consensus.cc:606] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:40.685070  1649 raft_consensus.cc:614] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:40.687194  1568 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:40.688974  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.689677  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.689930  1651 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:40.690667  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.691779  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.692378  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.696924  6923 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:40.702340  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.703254  2378 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.703956  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:40.704941  1571 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:40.492267  7580 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:40.678184  7425 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:40.687644  7525 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:40.714538  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.716084  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.716473  1649 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:03:40.711608  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.695348  7562 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:40.692198  7450 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:40.725762  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:40.725998  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.721942  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.728654  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.724606  7556 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:40.734861  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.762393  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.763090  1646 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.735703  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.736730  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.764057  1644 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:03:40.737375  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.764508  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.765082  1650 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.766451  1651 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:40.737565  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.738072  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.767789  1642 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:40.738142  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.705842  7536 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:40.740588  7527 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:40.772895  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:40.774633  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:40.740871  7609 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.742630  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:40.743820  1465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 207) took 28687 ms (client timeout 15000 ms). Trace:
I20260709 13:03:40.720809  7525 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:40.744923  7631 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.779s	user 0.010s	sys 0.003s
I20260709 13:03:40.746318  7441 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.790449  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.703274  7360 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.791757  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.706290  7576 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:40.747144  7562 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.749364  7450 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:40.750567  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.753327  7598 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:40.714854  7438 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.755538  7479 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.712865  7365 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:40.755849  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.726836  7421 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.770042  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.829237  1569 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:40.830602  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.831498  1569 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:40.726933  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:40.776994  1649 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:40.777395  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74208.8us
I20260709 13:03:40.832836  1568 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.833925  7551 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:40.835430  1539 leader_election.cc:400] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
W20260709 13:03:40.777678  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53772.9us
I20260709 13:03:40.725636  7654 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:03:40.775038  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.780381  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.780894  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:40.780962  1650 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:03:40.842433  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 67439.7us
I20260709 13:03:40.782022  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38bef56447d743db91bd1f59274db995"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:40.797694  1646 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:40.718398  7535 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.725345  7399 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.799057  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:40.804654  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:40.849786  1651 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.805549  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.806020  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.801849  7450 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.798488  7576 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:40.711313  7580 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:40.808868  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58352.1us
I20260709 13:03:40.779213  7536 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.811128  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:40.837699  1643 raft_consensus.cc:2368] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 19.
I20260709 13:03:40.776516  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:40.858166  1570 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.837586  7589 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:40.837049  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.864177  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.840178  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.865801  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.866637  7653 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:40.866936  7653 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:40.867035  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.867267  7653 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.867893  7650 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:40.842469  1647 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:40.868567  1642 raft_consensus.cc:606] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:40.851121  1571 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:40.868938  1642 raft_consensus.cc:614] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:40.784106  7525 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.850967  7553 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.875783  1568 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:40.881675  2374 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:40.843497  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.801838  7583 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.852799  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:40.771989  7527 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.888881  1571 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:40.810297  7438 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.856417  2379 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:40.839704  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:40.893587  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53939.3us
I20260709 13:03:40.895392  1570 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:40.869928  1539 leader_election.cc:400] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:40.845774  7365 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:40.802274  7441 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.886078  2375 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:40.853392  7450 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.854975  7576 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.887801  1574 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:03:40.855983  2373 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 16. Ops: []
I20260709 13:03:40.890545  1571 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:40.861323  1575 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:40.846864  7564 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:03:40.852849  7535 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.899641  2378 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:40.941406  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.959685  1571 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:40.909148  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65682.1us
I20260709 13:03:40.897892  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.902401  1651 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:40.900192  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:40.949651  6923 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:03:40.897640  1570 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:40.966967  1724 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:40.875057  7400 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:40.862267  7589 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.965224  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.971529  1575 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:03:40.971491  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 73617us
I20260709 13:03:40.978233  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:40.969355  1570 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:03:40.856305  7399 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.981098  1721 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:40.891479  7527 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.887372  7553 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.983573  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:40.950691  7372 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:40.984470  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:40.908542  7502 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:40.984866  7650 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:40.994596  7502 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:40.995556  7502 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.890200  7583 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.998140  7583 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:40.993274  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 93116.1us
W20260709 13:03:40.999858  7664 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:03:41.000574  7656 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:41.001327  7656 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:41.001919  7399 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.002548  7399 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:41.002466  7656 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.989060  7333 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:41.006127  7333 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:41.007066  7333 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:40.982688  7408 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:03:41.009516  7378 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:40.973016  7407 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:40.972184  7551 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:41.014396  6923 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:41.021709  1724 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:40.979306  7589 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.980640  7582 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:41.020663  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.027360  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.031018  6923 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:40.776693  1465 rpcz_store.cc:269] 0709 13:03:12.056074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.101261 (+ 45187us) service_pool.cc:224] Handling call
0709 13:03:40.743558 (+28642297us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:28.703830 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:28.704667 (+   837us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:28.704735 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:03:28.704757 (+    22us) tablet.cc:662] Decoding operations
0709 13:03:28.886574 (+181817us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:28.939951 (+ 53377us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:28.944063 (+  4112us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:03:29.251278 (+307215us) tablet.cc:701] Row locks acquired
0709 13:03:29.406753 (+155475us) write_op.cc:260] PREPARE: finished
0709 13:03:29.562290 (+155537us) write_op.cc:270] Start()
0709 13:03:29.605543 (+ 43253us) write_op.cc:276] Timestamp: P: 1783602209549262 usec, L: 0
0709 13:03:29.629146 (+ 23603us) op_driver.cc:348] REPLICATION: starting
0709 13:03:29.804948 (+175802us) log.cc:844] Serialized 9164 byte log entry
0709 13:03:30.760082 (+955134us) op_driver.cc:464] REPLICATION: finished
0709 13:03:30.808658 (+ 48576us) write_op.cc:301] APPLY: starting
0709 13:03:30.808769 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:31.076018 (+267249us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:31.168781 (+ 92763us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:31.979222 (+810441us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b180036ebc0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc833aad0ec 00007fc833af9c0a 00007fc833af94f7 00007fc833af9490 00007fc833ab0f48 00007fc8377c51bb 00007fc83780c5d4 00007fc8376b88da 00007fc8376bb3cb
0709 13:03:32.123927 (+144705us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:32.136991 (+ 13064us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 13:03:32.137151 (+   160us) write_op.cc:312] APPLY: finished
0709 13:03:32.211265 (+ 74114us) log.cc:844] Serialized 204 byte log entry
0709 13:03:32.535357 (+324092us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:32.624857 (+ 89500us) write_op.cc:454] Released schema lock
0709 13:03:32.651583 (+ 26726us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:32.681253 (+ 29670us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:32.681286 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":224255,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":48128,"apply.run_cpu_time_us":82053,"apply.run_wall_time_us":1872529,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100405,"num_ops":23,"prepare.queue_time_us":1081662,"prepare.run_cpu_time_us":32515,"prepare.run_wall_time_us":1298110,"raft.queue_time_us":190534,"raft.run_cpu_time_us":27651,"raft.run_wall_time_us":42651,"replication_time_us":1120206,"spinlock_wait_cycles":63134080,"thread_start_us":725702,"threads_started":5,"wal-append.queue_time_us":606350,"wal-append.run_cpu_time_us":54570,"wal-append.run_wall_time_us":2854880}]]}
I20260709 13:03:41.031114  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.022451  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:41.039208  6923 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:41.043905  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.048573  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.049065  1718 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.036330  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.052968  1717 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:41.049191  1725 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:40.995963  7412 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:03:40.992698  7553 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.042099  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.055718  1723 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:03:41.056232  1723 pending_rounds.cc:85] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 7
I20260709 13:03:41.056001  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.056598  1723 pending_rounds.cc:107] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:41.057001  1723 raft_consensus.cc:2889] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:41.012919  7372 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.060272  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:40.974633  7638 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:03:41.072144  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.073321  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.040102  7463 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:41.075807  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.070859  1651 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:41.069495  7650 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.086700  7650 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.087162  7650 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:41.089157  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.089905  7623 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.090389  7623 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:41.090787  7623 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.985769  7510 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:03:41.103477  6923 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.105314  6923 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.109031  7510 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:41.109555  7510 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:41.110599  7510 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.072142  7372 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.038823  7582 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:41.114558  7502 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.127969  7502 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.130447  7589 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.132398  7639 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:03:41.132771  7489 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.133800  7489 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:41.112277  7510 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.064277  7407 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.143505  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 165303us
I20260709 13:03:41.070210  7360 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.101790  7564 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.152426  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.169092  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.174906  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.181422  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.182740  1720 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:41.192131  1718 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:41.168208  7564 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:41.102475  7365 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.157203  7407 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.180312  7536 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.215430  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.221235  1720 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.229106  1723 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:41.213129  7438 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:41.234772  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 248427us
I20260709 13:03:41.208525  7564 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.235687  6923 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:41.208382  7580 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.206893  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.225122  7535 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.244971  1718 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:41.228879  7441 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.191090  7582 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:41.253736  7585 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.001s
I20260709 13:03:41.261332  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.250327  7580 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:41.263477  7580 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:41.263888  7580 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:41.236972  7412 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:03:41.257627  7506 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.801s	user 0.006s	sys 0.005s
I20260709 13:03:41.269506  7407 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.269933  7506 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:41.273343  7673 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:41.273972  7621 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:03:41.274312  7674 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:41.274771  7621 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:41.275813  7621 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:41.276176  7621 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:41.277539  7621 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.279012  7621 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:41.282487  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 212223us
I20260709 13:03:41.285951  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:41.287556  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.255478  7684 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:41.286185  1722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.014s	sys 0.011s
I20260709 13:03:41.289599  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:41.267916  7525 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:41.291321  1722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.014s	sys 0.011s
I20260709 13:03:41.261883  7675 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:41.281016  7610 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:41.295153  7684 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:41.288401  7557 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:41.291152  7582 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.309252  2379 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:41.306173  7557 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:41.312130  7557 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:41.296942  7421 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.290683  7653 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.303663  7675 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:41.363562  7675 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.366237  7589 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.368314  7653 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.370126  7576 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.371304  7684 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:41.450016  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 376339us
I20260709 13:03:41.451881  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.458340  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.462908  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:41.464018  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.464407  1571 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:41.473476  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.017s	sys 0.004s
W20260709 13:03:41.474049  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.017s	sys 0.005s
I20260709 13:03:41.474159  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.461942  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.293574  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.293941  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:41.498680  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 210943us
I20260709 13:03:41.501492  7407 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:41.513705  1571 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
W20260709 13:03:41.515651  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 346268us
I20260709 13:03:41.518141  7576 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.331265  7421 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.355666  7425 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:41.538254  1718 raft_consensus.cc:2368] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:41.540211  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.579314  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.580479  7562 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:41.581308  7562 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:41.594097  7421 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.596612  7653 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:41.597282  7653 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:41.597844  7653 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:41.598819  7667 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.801s	user 0.005s	sys 0.000s
I20260709 13:03:41.600652  7576 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:41.600503  1716 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:41.601464  7576 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:03:41.603463  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67368us
I20260709 13:03:41.604127  1720 raft_consensus.cc:2410] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:41.605230  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:41.581923  7562 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.379537  7527 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.588685  7407 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:41.627589  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.629710  7525 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.612823  7518 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.376232  7580 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:41.642378  7650 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.008s
I20260709 13:03:41.644999  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2564) took 1068 ms. Trace:
W20260709 13:03:41.647981  7681 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:41.610114  7688 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.652274  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73174.4us
I20260709 13:03:41.603267  7576 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.663816  1616 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:41.666947  7673 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:41.667065  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2944) took 1023 ms. Trace:
I20260709 13:03:41.670701  7527 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.672777  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:41.697743  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139191us
I20260709 13:03:41.672744  7688 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.673379  7506 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.674284  2371 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: [12.12-14.13]
I20260709 13:03:41.657716  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:41.637627  7479 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.800s	user 0.017s	sys 0.000s
W20260709 13:03:41.707237  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 245509us
I20260709 13:03:41.695066  1569 rpcz_store.cc:276] 0709 13:03:40.643520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.840821 (+197301us) service_pool.cc:224] Handling call
0709 13:03:41.664736 (+823915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.665181 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:41.665217 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.665267 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:41.665354 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.665457 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.666662 (+  1205us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b5400364cd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:41.666836 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857030016}
I20260709 13:03:41.653970  1574 rpcz_store.cc:276] 0709 13:03:40.576729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.576856 (+   127us) service_pool.cc:224] Handling call
0709 13:03:41.644539 (+1067683us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400364cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:41.644792 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712701696}
I20260709 13:03:41.716610  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:41.726228  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62632.3us
I20260709 13:03:41.721050  7479 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:41.731899  7372 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.003s
I20260709 13:03:41.732797  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.733729  2377 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:41.734323  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.737561  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.739051  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.740235  1721 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:41.740491  7506 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.742262  7557 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.743089  7506 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:41.744469  7689 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.744757  7676 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:41.745083  7689 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.745584  7676 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:41.629769  7678 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260709 13:03:41.747005  7676 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.748754  7407 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.752553  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:41.753862  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:41.745661  7689 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.732563  7372 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.760632  1716 raft_consensus.cc:1240] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:41.768126  7557 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:41.775724  7623 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.009s
I20260709 13:03:41.781637  7372 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.783054  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:41.776296  7623 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.786689  7623 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.787508  7623 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:41.787252  7421 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.787685  7384 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.788318  7623 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.702719  7650 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.807073  7706 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:41.807919  1570 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:41.805343  1725 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:41.754966  7518 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.822300  7518 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.821946  7562 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.826081  7421 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.740273  7564 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.858378  1574 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:03:41.866595  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54476us
I20260709 13:03:41.867740  1569 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:41.867904  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43812.2us
W20260709 13:03:41.871512  7510 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.735s	user 0.006s	sys 0.011s
I20260709 13:03:41.857559  7562 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.876158  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:41.863814  7518 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:41.888249  7688 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:41.895987  7450 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.920s	user 0.007s	sys 0.011s
W20260709 13:03:41.967588  7621 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.008s
I20260709 13:03:41.968474  7450 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.969409  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.970077  2374 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:41.970142  1721 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:41.971271  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:41.972535  7450 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:41.897771  7580 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.880381  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.882494  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2573) took 1143 ms. Trace:
I20260709 13:03:41.920089  7650 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:41.934721  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.978231  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.979209  1571 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:41.980604  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.923995  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2502) took 1525 ms. Trace:
I20260709 13:03:41.981277  1643 rpcz_store.cc:276] 0709 13:03:40.399184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.515015 (+115831us) service_pool.cc:224] Handling call
0709 13:03:41.923614 (+1408599us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400350a54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:41.923820 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739797376}
W20260709 13:03:41.926633  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 62836cb7c4a548b5a6c44c89782a958c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:41.922345  7564 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.920728  7623 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:41.915372  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.940372  7643 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:41.985241  7643 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:41.985966  7081 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:41.925457  7553 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.007s
I20260709 13:03:41.932279  7525 consensus_queue.cc:1059] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:03:41.908910  7463 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:41.897579  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2917) took 1178 ms. Trace:
I20260709 13:03:41.903308  7686 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.959985  7639 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:42.000435  7686 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.000968  7686 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.000752  7639 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.002218  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:42.003492  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.003736  7639 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:41.996441  1722 rpcz_store.cc:276] 0709 13:03:40.718726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.719010 (+   284us) service_pool.cc:224] Handling call
0709 13:03:40.719428 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.719646 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:41.295291 (+575645us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.697615 (+402324us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.889997 (+192382us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b18004965a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc837c4258f 00007fc830c81571 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd
0709 13:03:41.896940 (+  6943us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.897020 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:41.897120 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.897226 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.897399 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210354,"prepare.queue_time_us":179741,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":4079,"raft.queue_time_us":217929,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":30804,"spinlock_wait_cycles":5116672,"thread_start_us":770694,"threads_started":2,"wal-append.queue_time_us":327811}
W20260709 13:03:41.960126  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78316.9us
I20260709 13:03:41.961685  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.962195  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2245) took 1186 ms. Trace:
I20260709 13:03:42.006711  7525 raft_consensus.cc:3055] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:42.007280  7525 raft_consensus.cc:740] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:42.006855  1645 rpcz_store.cc:276] 0709 13:03:40.775607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.801739 (+ 26132us) service_pool.cc:224] Handling call
0709 13:03:40.802179 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.984219 (+182040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.984253 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.984295 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:40.984361 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.961684 (+977323us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000ba754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:41.961793 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.962002 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417938816}
I20260709 13:03:41.949198  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2581) took 1328 ms. Trace:
I20260709 13:03:41.947198  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:42.008945  7525 consensus_queue.cc:260] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.010834  7525 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.951879  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:41.974511  1576 rpcz_store.cc:276] 0709 13:03:40.618237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.653289 (+ 35052us) service_pool.cc:224] Handling call
0709 13:03:40.654076 (+   787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:40.654579 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.654620 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:40.676772 (+ 22152us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.797325 (+120553us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:40.797370 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.797457 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:41.597373 (+799916us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:41.705261 (+107888us) raft_consensus.cc:1685] finished
0709 13:03:41.705284 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.761265 (+ 55981us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":15654,"spinlock_wait_cycles":2311936,"thread_start_us":88010,"threads_started":1,"wal-append.queue_time_us":106027}
I20260709 13:03:41.900695  1718 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:41.954911  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 836) took 1049 ms. Trace:
W20260709 13:03:41.900638  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 171856us
I20260709 13:03:41.958487  7653 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.964581  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:41.967597  1720 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:03:42.012619  2371 rpcz_store.cc:276] 0709 13:03:40.905350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.905525 (+   175us) service_pool.cc:224] Handling call
0709 13:03:40.918850 (+ 13325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:41.709487 (+790637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.954448 (+244961us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540064f354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:41.954814 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699384192}
I20260709 13:03:41.958597  7689 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.008512  1572 rpcz_store.cc:276] 0709 13:03:40.660870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.663226 (+  2356us) service_pool.cc:224] Handling call
0709 13:03:41.948852 (+1285626us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54001aa9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:41.949021 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793238272}
I20260709 13:03:41.936121  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:41.929049  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:41.929654  7518 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:41.981295  1719 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:41.986539  7643 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:41.988929  7668 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 1.123s	user 0.003s	sys 0.003s
I20260709 13:03:41.988595  7553 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.989323  7081 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:41.989826  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 834) took 1105 ms. Trace:
I20260709 13:03:41.990588  1569 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: [14.9-17.10]
I20260709 13:03:42.029742  1568 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.992326  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:41.994449  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:41.986797  1458 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60385.6us
I20260709 13:03:42.004194  1724 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:42.032619  2374 rpcz_store.cc:276] 0709 13:03:40.884776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.885211 (+   435us) service_pool.cc:224] Handling call
0709 13:03:40.900229 (+ 15018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:41.970718 (+1070489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.989549 (+ 18831us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400547254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:41.989754 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458294016}
I20260709 13:03:42.039345  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:42.043615  1576 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:42.040608  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 316360us
I20260709 13:03:42.047466  7553 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:42.053038  1571 raft_consensus.cc:2410] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:42.059530  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:42.060421  1642 raft_consensus.cc:606] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:42.060992  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68788.4us
I20260709 13:03:42.061699  7639 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.062840  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:42.065748  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:42.067508  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:42.056202  1571 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:42.068609  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:42.064164  1568 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:42.084736  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:42.070390  1569 raft_consensus.cc:2376] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19. Re-sending same reply.
I20260709 13:03:42.069298  7643 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:42.087864  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 158868us
I20260709 13:03:42.068545  7702 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.091678  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:42.069909  7081 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.090875  7639 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:42.104818  7639 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.105252  7639 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.094180  7653 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:42.099511  7518 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.104453  7553 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:42.106093  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.095214  1571 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: [12.10-13.11]
I20260709 13:03:42.111215  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:42.142025  1643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:42.142802  1643 raft_consensus.cc:606] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:42.149885  1643 raft_consensus.cc:614] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:42.148995  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.145095  1721 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.157478  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:42.157235  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:42.164186  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:42.170620  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 77870us
W20260709 13:03:42.175503  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 412600us
I20260709 13:03:42.153918  7081 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.185206  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.161036  7653 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:42.189682  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:42.194402  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 124189us
W20260709 13:03:42.183502  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.994s (SENT) [suppressed 25 similar messages]
I20260709 13:03:42.200181  1571 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:42.190277  7623 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.210368  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:42.212563  7463 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:42.213007  7535 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:03:42.213670  7412 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:42.214129  6923 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:42.214509  7517 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:42.214728  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:42.225167  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.225455  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2943) took 1582 ms. Trace:
W20260709 13:03:42.226585  7715 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:03:42.231725  2373 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:42.239025  7518 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:42.256635  7518 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.257105  7518 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.241698  6923 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.260401  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:42.261546  7653 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.261996  6923 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.263832  7653 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.265971  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:42.236456  7081 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.249244  7412 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:42.250825  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 53478.5us
I20260709 13:03:42.243347  7517 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.251641  7722 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.244252  1567 rpcz_store.cc:276] 0709 13:03:40.643289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.667638 (+ 24349us) service_pool.cc:224] Handling call
0709 13:03:40.668040 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.668316 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:40.863627 (+195311us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.561838 (+698211us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.631161 (+ 69323us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.925020 (+293859us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b180020c4a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:41.929399 (+  4379us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b180020c4a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:42.192601 (+263202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.198745 (+  6144us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:42.206681 (+  7936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.224799 (+ 18118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.225247 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230464,"prepare.queue_time_us":421086,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":4413,"raft.queue_time_us":446525,"raft.run_cpu_time_us":523,"raft.run_wall_time_us":57513,"spinlock_wait_cycles":42406016,"thread_start_us":875972,"threads_started":5,"wal-append.queue_time_us":317963}
I20260709 13:03:42.238950  7463 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.252048  7690 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.255448  1571 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:42.276283  1721 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:42.285192  6923 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.289026  7535 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.269408  7712 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:03:42.273813  7517 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:42.273891  7689 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:42.302636  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87976.8us
I20260709 13:03:42.306804  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:42.310312  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:42.311209  1571 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:42.244071  7686 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.316231  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:42.303717  7702 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.303994  7081 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.323694  1571 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:42.324302  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.298056  7535 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.283963  7722 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.313020  7626 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:03:42.328548  7702 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.333657  7690 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:03:42.343570  7384 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.364951  7333 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.371433  7463 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:42.371953  7333 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.374647  7333 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:42.371963  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:42.375927  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 67918.6us
I20260709 13:03:42.376543  1651 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.376684  7676 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:42.376940  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.378412  7412 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.379534  7535 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:42.390061  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.390980  1721 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:42.383196  7690 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:42.392582  7690 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:42.387735  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:42.394497  1651 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:42.394625  7690 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.394497  1643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
W20260709 13:03:42.401173  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 95154.7us
W20260709 13:03:42.408633  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 81416.1us
I20260709 13:03:42.412679  6923 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:42.414450  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50893.1us
I20260709 13:03:42.415480  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:42.416843  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:42.420974  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:42.421577  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:42.382073  7702 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.425896  7464 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:42.426331  7464 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:42.427058  7384 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:42.420156  7690 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.427536  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:42.428177  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 162196us
I20260709 13:03:42.427346  7464 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.371374  7722 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.435334  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:42.438609  7738 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.435200  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.451599  6447 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.452123  7735 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.454350  7535 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:42.447227  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:42.427675  7384 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:42.452042  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 75202us
I20260709 13:03:42.446991  7464 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.428221  7450 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:42.436195  7621 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:42.445482  7525 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:42.448828  7653 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:42.454445  7623 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.482573  6923 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.493978  6923 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.434962  7598 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:03:42.513345  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.435381  7463 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:03:42.519181  6923 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:42.512184  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.476032  7621 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.521121  7598 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:42.488899  7412 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.524827  7489 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.528780  7384 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.489553  7686 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:42.489753  7650 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.489344  7653 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:42.465065  7738 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.499235  7450 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:42.542269  7623 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:42.559626  7732 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:03:42.555419  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.560693  7517 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:42.561362  7730 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:03:42.561785  1724 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:42.563246  1642 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:42.551625  7621 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:42.567013  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 143815us
I20260709 13:03:42.593698  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.550694  7598 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:42.557410  7653 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.591296  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.592278  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.554133  7650 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:42.617484  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:42.620962  1721 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:42.646190  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 128035us
I20260709 13:03:42.623885  7650 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.659653  1645 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:42.662272  1724 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:42.646132  7576 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:03:42.647336  7623 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.648412  7704 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.591744  7412 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.672880  1718 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:42.673482  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.608898  7676 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:03:42.700362  7527 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.700629  7466 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.701018  7527 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:42.705180  7736 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 13:03:42.706236  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:42.707388  1644 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:42.705178  7527 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:42.708706  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46920.8us
I20260709 13:03:42.709728  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:42.716012  7466 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:42.700102  7684 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:42.647543  7489 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.682140  7517 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:42.726544  7598 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:42.763921  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 101689us
W20260709 13:03:42.766212  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 56568.6us
I20260709 13:03:42.766420  1716 raft_consensus.cc:2410] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:42.753058  7527 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:42.757979  7639 consensus_meta.cc:287] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.007s
W20260709 13:03:42.771965  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.768452  7378 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.784986  7743 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:03:42.785670  1648 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:42.718132  7450 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.800093  7489 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:42.727615  7704 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.789266  7378 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:42.805218  7378 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.805910  7412 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.808650  7746 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.809404  7746 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.773494  7517 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.763926  7626 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:03:42.817231  7746 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:42.821821  6447 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:42.793823  7598 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.824050  1647 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:42.835105  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:42.851323  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.789606  7639 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.854383  7739 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:03:42.823410  7517 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.856743  7738 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:42.859573  7733 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.442s	user 0.004s	sys 0.009s
I20260709 13:03:42.792665  7527 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:42.873183  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 72269.6us
I20260709 13:03:42.860586  7689 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:42.882892  7739 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:42.883517  7739 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:42.884620  7739 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.886555  7739 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.888756  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:42.890029  1647 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:42.893163  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:42.901463  1643 raft_consensus.cc:606] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:42.901935  1643 raft_consensus.cc:614] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:42.908624  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:42.910951  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:42.921300  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 303767us
I20260709 13:03:42.850530  7743 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:03:42.924641  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:42.927093  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 77305.3us
I20260709 13:03:42.880446  7689 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:42.932197  2371 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:42.934751  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:42.935652  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:42.934192  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:42.931062  7518 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.003s
I20260709 13:03:42.948616  7518 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.949230  2375 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:42.952050  7742 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:42.952580  7742 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:42.952621  7518 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:42.944538  7737 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.525s	user 0.004s	sys 0.007s
W20260709 13:03:42.953634  7723 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.533s	user 0.005s	sys 0.006s
I20260709 13:03:42.955943  7517 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.945837  7639 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.959025  7527 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.959549  7081 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:42.959951  7081 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:42.960134  7704 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:42.929864  7687 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.961046  7081 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.961524  7687 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:42.962483  7081 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.963697  7687 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:42.968756  7378 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.969426  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:42.970047  1721 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.972043  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.969014  7689 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:42.969062  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.979555  1646 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
W20260709 13:03:42.981093  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.981166  7743 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.982486  2373 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 12: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:42.969959  1648 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.985854  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:42.986028  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:42.990988  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:42.982245  7464 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 13:03:42.994308  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.995381  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.997602  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:42.999153  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.001737  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.001480  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.010617  1719 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:03:43.011222  1725 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 14.
I20260709 13:03:43.013120  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.013427  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.017328  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:43.018916  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.024379  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.025054  1568 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:42.993813  7704 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:42.984781  7527 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:03:43.029850  1719 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.026945  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.030995  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:43.027226  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:43.027961  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:42.997620  7687 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.033205  1717 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:43.030762  1723 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:43.035261  1569 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:43.009244  7689 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.049458  1721 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:43.051239  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.051388  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.051976  1571 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:43.053902  1725 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:42.990437  7743 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.054988  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.056384  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:43.057402  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.026127  7518 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:43.084194  1568 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:03:43.066103  1647 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:42.984830  7742 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.087569  7742 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:43.090585  1720 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:43.067456  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.061887  1570 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:43.037576  7360 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:43.091883  1574 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:43.089416  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.052495  7693 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:43.104669  1718 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:43.061887  7639 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:43.088096  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.102229  1646 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:43.108870  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 149540us
I20260709 13:03:43.034253  7704 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.097635  7510 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:43.044493  7687 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.114277  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.052562  7621 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:43.192994  7728 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.772s	user 0.000s	sys 0.013s
I20260709 13:03:43.194969  7621 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.119693  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.120064  1647 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.196247  1575 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 12: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:43.196946  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.197865  1647 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:43.198406  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.102412  7693 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.198822  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.199337  7693 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.199788  7704 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.062333  7653 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.200400  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.201732  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.203100  1647 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:43.204456  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.205327  1647 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:43.097889  7360 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:43.207494  7360 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.116837  7650 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.208667  7360 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.126999  7747 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:43.127949  7509 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.130930  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.132548  7464 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.212724  7509 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.213142  7464 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.215240  7509 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.215518  7464 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:43.132393  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:43.216728  2371 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:43.216905  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.217734  1572 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:43.218788  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.219441  1572 raft_consensus.cc:2376] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 14. Re-sending same reply.
I20260709 13:03:43.133716  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:43.134305  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:43.136292  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:43.224117  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.222951  2379 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
W20260709 13:03:43.212726  7727 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.849s	user 0.007s	sys 0.005s
I20260709 13:03:43.227376  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.228714  1645 raft_consensus.cc:2393] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:43.229130  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.230593  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.231325  1570 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:43.233367  7650 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.234825  1568 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:43.236099  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.237401  2377 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:43.237353  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.143237  1569 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:43.144595  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.146322  1571 raft_consensus.cc:1240] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:43.151748  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.241178  1720 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:43.152802  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.153254  7746 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:43.154235  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.156191  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:43.120563  7639 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.156682  1646 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:03:43.158294  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.245860  1645 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:43.246255  1650 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.160566  7518 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:43.244740  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 88634.5us
I20260709 13:03:43.162322  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.163375  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 2972) took 1199 ms. Trace:
I20260709 13:03:43.163467  7081 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:43.164104  7510 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.164391  7735 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.167223  7744 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:43.180951  7378 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.192453  7687 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.119131  6923 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.206260   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:43.219537  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.219048  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.221800  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.133476  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.278290  1576 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.278853  1649 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:43.203099  7653 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.277813  1574 raft_consensus.cc:2376] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 17. Re-sending same reply.
W20260709 13:03:43.250625  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.265422  1725 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:03:43.273757  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 183045us
I20260709 13:03:43.280390  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.297360  1718 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.243193  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.164403  7743 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.289533  1646 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:43.299049  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.299769  1725 raft_consensus.cc:3060] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.303359  6923 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.304708  1647 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.244004  7746 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:43.261773  7527 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:43.318163  1572 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:43.280464  7704 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.319469  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.321267  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.334481  1568 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:43.272431  7744 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:43.327016  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.327987  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.328737  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.337170  6923 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.253463  1573 rpcz_store.cc:276] 0709 13:03:41.963474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.196892 (+233418us) service_pool.cc:224] Handling call
0709 13:03:42.205638 (+  8746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.264509 (+ 58871us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:42.264866 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.443639 (+178773us) log.cc:844] Serialized 14 byte log entry
0709 13:03:42.443835 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.443891 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:42.443973 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.505053 (+ 61080us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b1800441b20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:42.858849 (+353796us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:43.138392 (+279543us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b1800441b20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c81522 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:43.163122 (+ 24730us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b5400026cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:43.163177 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.163309 (+   132us) inbound_call.cc:177] Queueing success response
0709 13:03:43.169078 (+  5769us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b1800441b20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
Metrics: {"mutex_wait_us":142073,"prepare.queue_time_us":1571,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":105,"raft.queue_time_us":154922,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":38238,"spinlock_wait_cycles":175776896,"thread_start_us":35930,"threads_started":1,"wal-append.queue_time_us":67248}
I20260709 13:03:43.306746  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:43.337345  1720 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:43.338093  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:43.333256  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 139317us
I20260709 13:03:43.339701  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:43.341965  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.342850  1568 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:43.344578  7360 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:43.345112  7360 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.345523  7360 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.345897  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:43.347915  1719 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:43.349042  7653 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:43.350838  7653 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:43.352603  1572 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.305929  7650 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.358182  1721 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:43.357223  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:43.262739  7518 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.380739  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:43.386242  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:43.389043  7739 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.004s
I20260709 13:03:43.389870  7739 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:03:43.392549  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.395056  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.361943  1717 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:43.400524  7687 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:43.402573  7751 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:03:43.265376  7525 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:03:43.352222  1645 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:43.314574  7743 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:43.371011  1718 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:43.374436  2375 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.408691  7621 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:43.413190  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:43.414402  7333 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:43.415071  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.415622  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.276921  7735 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:43.416719  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.418627  7735 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.420663  7639 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.271339  7510 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.427520  7510 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.428959  7743 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.431612  7743 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.434772  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.435637  1574 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:03:43.437968  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.364236  1725 raft_consensus.cc:2468] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:43.364723  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:43.442163  1568 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.364735  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:43.445133  2377 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.361075  1648 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:03:43.335706  7527 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:43.449915  7703 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:43.265812  7081 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.355348  7704 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.406129  7525 consensus_queue.cc:1243] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:43.392042  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.373972  7746 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.469046  7746 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:43.356492  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 49890.7us
I20260709 13:03:43.447150  1646 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:43.470976  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:43.388146  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.473026  1573 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:43.473273  7743 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:43.473496  1573 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:43.473809  7743 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.474232  7743 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.474613  1573 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.475904  1573 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.446576  7650 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.482625  7650 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.456156  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:43.447800  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:43.480212  1642 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:03:43.485152  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:43.486344  1616 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:43.450234  7527 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.461028  6447 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:43.464632  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 116499us
I20260709 13:03:43.490007  1650 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:43.490178  1570 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:43.493445  1647 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:43.494704  1724 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:43.502841  1568 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:43.507340  1568 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: [14.9-17.10]
I20260709 13:03:43.490900  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:43.495647  6447 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.537767  1567 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:43.486104  1716 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:43.544595  1569 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.540230  1643 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:43.536090  7739 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:43.551096  7598 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:43.553054  6447 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.553429  7693 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.554080  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.555054  1645 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:43.551594  7739 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:43.561383  1568 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:43.561718  7704 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.564110  6923 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.589406  6923 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.374025  7744 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:43.575038  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.590950  1644 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.591138  7744 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.545449  7689 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.592309  7689 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.592727  7689 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.593159  7464 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:43.576093  1568 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:43.578331  7384 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:43.579334  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.582026  7081 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:43.582432  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.556061  7693 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.461829  7412 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.584245   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:43.571506  1651 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.457389  7576 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:43.569180  1571 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:43.589736  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:43.590384  7739 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.651979  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.542599  7517 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:43.617023  1647 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:43.617941  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.662789  1643 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:43.618323  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.624675  1574 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:43.626387  1576 raft_consensus.cc:2410] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:03:43.638984  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 106763us
I20260709 13:03:43.646570  6923 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.649482  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.675223  1648 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.539117  7510 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.613705  1572 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:43.678771  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.681306  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:43.682389  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.682732  7510 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.700130  7510 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.686489  7621 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:43.690039  1576 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:43.795329  7517 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:43.796911  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.797712  1646 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:43.798228  1688 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:43.690776  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.538949  7755 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:43.676622  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:43.699278  1572 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:43.800665  1567 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:43.695950  1648 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:43.703608  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.658334  7743 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:43.758495  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.643237  7690 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.111s
W20260709 13:03:43.793865  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.791603  7525 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.678885  7412 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:43.794566  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.682704  7650 raft_consensus.cc:2764] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:03:43.756310  7450 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.811035  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134490us
W20260709 13:03:43.813540  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 360291us
I20260709 13:03:43.813393  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:43.819217  1648 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:43.830101  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 240330us
I20260709 13:03:43.831348  1643 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:43.678534  7384 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.850292  1644 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.815352  7738 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.858392  2374 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:43.861752  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:43.862213  7743 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:43.862939  2374 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 18: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:43.863303  7743 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:43.864648  7412 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.865622  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:43.866755  1643 raft_consensus.cc:2368] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:03:43.851110  7450 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.846098  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:43.800431  7621 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:03:43.885313  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.890399  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44477.6us
I20260709 13:03:43.891530  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:43.891549  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.890477  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.895640  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:43.900960  1643 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:43.894744  1573 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:03:43.903368  1573 pending_rounds.cc:85] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:03:43.896914  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.907289  1569 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:43.897460  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:43.903792  1573 pending_rounds.cc:107] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:43.855444  7738 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.910044  1573 raft_consensus.cc:2889] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:43.910894  1575 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:03:43.917377  1614 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:43.836432  7525 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:43.895483  7747 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:43.923745  7738 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.922592  7412 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:43.921043  7703 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:43.961536  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.968175  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.973555  1644 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:43.974129  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.974385  1725 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:43.940979  7747 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.973294  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.975914  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.976547  1717 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.976696  1724 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:43.862821  7517 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:43.988080  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 99376.7us
I20260709 13:03:43.980298  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.990778  1721 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:43.970968  7360 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.000s
I20260709 13:03:43.984668  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.985831  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.997591  1720 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:43.964972  7509 raft_consensus.cc:2804] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:43.991796  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:43.984709  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:43.983983  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:44.006781  1725 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.954849  7450 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:43.982622  7747 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:44.031610  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.022s	sys 0.021s
W20260709 13:03:44.032284  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.022s	sys 0.021s
I20260709 13:03:44.000643  7333 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.047571  6447 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:44.048264  6447 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:43.985165  7576 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:44.052568  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:44.053951  1716 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:44.056113  1725 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:44.049719  6447 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.086761  1539 leader_election.cc:400] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:44.091535  6447 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:44.029722  7509 raft_consensus.cc:697] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:44.052052  7333 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.009193  7360 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.030288  7676 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
W20260709 13:03:44.115478  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 85564.8us
I20260709 13:03:44.116983  7576 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:44.131124  2377 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
I20260709 13:03:44.131669  2377 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:44.102774  7450 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.121768  7759 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.212s	user 0.000s	sys 0.002s
W20260709 13:03:44.154121  7762 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:03:44.169782  7384 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:44.133754  7765 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:03:44.171566  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46353.8us
I20260709 13:03:44.169827  6447 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:44.126912  7464 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.002s	sys 0.019s
W20260709 13:03:44.193632  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 402861us
I20260709 13:03:44.197018  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:44.198325  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 218987us
W20260709 13:03:44.214145  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.218575  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:44.221428  6447 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:44.222949  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 424560us
I20260709 13:03:44.224104  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:44.231416  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.200675  7464 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.002s	sys 0.019s
I20260709 13:03:44.221594  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:44.213589  7525 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.242354  7703 raft_consensus.cc:2764] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:44.253489  7081 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:44.254172  7081 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:44.255434  7081 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:44.265420  7771 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:03:44.270711  7689 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.002s
I20260709 13:03:44.271296  7689 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:44.273239  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51703.7us
W20260709 13:03:44.278030  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54123.6us
I20260709 13:03:44.279232  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:44.215267  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:44.245201  6447 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:44.243058  1717 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
W20260709 13:03:44.250335  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53450.7us
I20260709 13:03:44.319115  6447 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:44.323897  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.325700  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:44.326493  1650 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.326853  7639 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.327666  7639 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.330369  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:44.331851  7765 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:44.332489  7755 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:03:44.333017  7755 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.335376  7755 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:44.272838  7739 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.565s	user 0.014s	sys 0.000s
I20260709 13:03:44.336778  7739 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.339409  7739 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:44.339634  7744 consensus_meta.cc:287] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.748s	user 0.011s	sys 0.005s
I20260709 13:03:44.190711  7450 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.344260  7518 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.346210  7450 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:44.350497  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 135305us
W20260709 13:03:44.356498  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.357095  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.366055  1650 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:44.368124  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:44.281729  7690 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:44.372716  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:44.330681  7639 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.352938  7464 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:44.369426  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:44.354399  7757 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:44.383277  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.383644  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 104490us
I20260709 13:03:44.384245  6447 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.371407  7765 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:44.406546  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 318514us
I20260709 13:03:44.375535  7755 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:03:44.425350  7766 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.296s	user 0.003s	sys 0.001s
I20260709 13:03:44.427537  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:44.443993  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74598.9us
I20260709 13:03:44.428774  7360 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.428930  6447 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.408694  7765 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.429090  7772 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:03:44.431419  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.447904  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.434649  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:44.448696  1570 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.437971  7689 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.376883  7518 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:44.438860  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:44.409924  7757 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.372660  7779 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:03:44.442302  1573 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:44.276988  7509 consensus_queue.cc:237] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.386821  7703 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:44.382408  7690 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.478540  1570 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:44.480067  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:44.446174  7755 consensus_queue.cc:1243] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:03:44.457037  7510 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.008s
I20260709 13:03:44.445446  7450 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.451289  7744 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:44.507519  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:44.447376  7765 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.508925  2377 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:44.464051  7757 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.466436  7518 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 18 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:44.511726  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:44.514094  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:44.482504  7690 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:44.484735  7464 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:44.513505  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.513410  7782 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:44.515736  7792 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:44.517022  7621 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.518725  7639 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:44.494647  7689 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.507020  7510 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.560959  2377 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:03:44.507682  7744 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.552497  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:44.506443  7360 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.530428  7384 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:44.526733  7676 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:44.562219  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:44.571514  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62020.6us
I20260709 13:03:44.549181  7782 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.573321  7384 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:44.583828  7384 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.576959  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:44.583917  7518 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.559666  7687 raft_consensus.cc:2764] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:44.595453  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:44.513069  7450 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.598101  2377 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:44.586475  2374 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:44.598763  1645 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:44.597110  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:44.598999  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 171666us
I20260709 13:03:44.570399  7744 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.572435  7621 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:44.615403  7795 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.615832  7779 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.616216  7795 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.617050  7779 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.624799  1650 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:03:44.585676  7782 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:44.636050  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50051.2us
W20260709 13:03:44.619180  7789 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:03:44.571033  7689 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.601354  7384 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:44.587844  7360 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.670580  7799 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.671517  7799 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:44.673064  7790 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.051s	user 0.001s	sys 0.001s
I20260709 13:03:44.674527  7799 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.641963  7687 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.692546  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3003) took 1066 ms. Trace:
I20260709 13:03:44.694063  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:44.693248  1576 rpcz_store.cc:276] 0709 13:03:43.626529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.796282 (+169753us) service_pool.cc:224] Handling call
0709 13:03:43.798793 (+  2511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.799090 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:44.177330 (+378240us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.493597 (+316267us) log.cc:844] Serialized 14 byte log entry
0709 13:03:44.519798 (+ 26201us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b180041ca40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1
0709 13:03:44.521310 (+  1512us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b180041ca40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:44.657675 (+136365us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180041ca40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287 00007fc833d89208
0709 13:03:44.690902 (+ 33227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:44.691002 (+   100us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:44.691091 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:44.691213 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.692447 (+  1234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20650,"prepare.queue_time_us":24635,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":3186,"raft.queue_time_us":15157,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":15057,"spinlock_wait_cycles":86557312,"thread_start_us":706373,"threads_started":3,"wal-append.queue_time_us":303779}
I20260709 13:03:44.693467  7795 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.700016  7765 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.703708  7779 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.704190  7765 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.704829  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:44.705951  2374 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:44.718544  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2268) took 1100 ms. Trace:
W20260709 13:03:44.719334  7787 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:03:44.677034  7792 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.719187  1645 rpcz_store.cc:276] 0709 13:03:43.617889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.618068 (+   179us) service_pool.cc:224] Handling call
0709 13:03:44.718241 (+1100173us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400412654. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:44.718458 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130016000}
W20260709 13:03:44.731838  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65034.4us
W20260709 13:03:44.733465  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.733299  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 851) took 1452 ms. Trace:
I20260709 13:03:44.739928  1722 rpcz_store.cc:276] 0709 13:03:43.281370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.281588 (+   218us) service_pool.cc:224] Handling call
0709 13:03:43.314347 (+ 32759us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:43.329807 (+ 15460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.329857 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:43.359922 (+ 30065us) log.cc:844] Serialized 52 byte log entry
0709 13:03:43.731519 (+371597us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:43.731597 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.731697 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.721868 (+990171us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b24001fe5d8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830b079c4 00007fc830b05acb 00007fc830cc0dc9 00007fc830cc0a55 00007fc830cbffb0 00007fc833936488 00007fc833936be8 00007fc83390e398 00007fc83390976a 00007fc833908bb6
0709 13:03:44.727033 (+  5165us) raft_consensus.cc:1685] finished
0709 13:03:44.727057 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:44.733225 (+  6168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20166,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":4384,"spinlock_wait_cycles":32030592,"thread_start_us":295343,"threads_started":1,"wal-append.queue_time_us":504374}
I20260709 13:03:44.746783  7795 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:44.747066  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:44.748631  7802 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.748865  7796 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:44.749440  7802 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.750329  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:44.737346  7800 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:44.754944  7765 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:44.761910  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:44.763254  1573 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.752156  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:44.731040  7747 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.004s
I20260709 13:03:44.766469  7802 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.766867  7747 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:44.767851  7793 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:03:44.768890  7510 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.740026  7792 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.771095  7747 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.764110  7800 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.772578  7800 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.774263  7450 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.684499  7464 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.775095  7800 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:44.775607  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:44.774151  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:44.779103  2377 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.780767  7360 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:44.783335  1573 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:44.694686  7687 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.790578  7687 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.796309  7757 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.799083  7757 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:44.799726  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.790577  7690 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.801739  2374 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:44.802392  7690 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.804688  2377 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
W20260709 13:03:44.804993  7738 consensus_meta.cc:287] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.816s	user 0.013s	sys 0.004s
W20260709 13:03:44.806105  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54063.9us
I20260709 13:03:44.805859  7738 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:44.806708  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.890s (SENT)
I20260709 13:03:44.807411  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:44.808072  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:44.809042  1644 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:44.809731  7795 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:44.810421  7795 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:44.810520  7765 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:44.811194  7738 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:44.811803  7765 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:44.811683  7795 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.813055  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:44.814208  7690 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:44.814363  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:44.815442  7690 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:44.817431  1688 leader_election.cc:400] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:03:44.817771  1688 leader_election.cc:403] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:44.819814  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:44.800123  7333 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.003s
I20260709 13:03:44.821611  7360 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:03:44.820745  7765 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:03:44.825366  7800 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.826050  7800 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.827353  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:44.816972  7690 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.822552  7805 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:44.833014  7690 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.829156  7333 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.817209  7621 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:44.836230  7804 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.097s	user 0.002s	sys 0.000s
W20260709 13:03:44.839551  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89295.5us
I20260709 13:03:44.840598  7800 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:44.838733  7796 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.851554  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:44.852416  7412 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:44.864084  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.856573  7779 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.858994  7333 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:44.864522  2374 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:44.866117  7412 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.868409  7805 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:44.869109  7360 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.868848  7687 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:44.870229  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:44.871094  7800 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:44.877161  7509 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.879390  7805 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:44.876216  7799 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.889063  7805 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.877938  7796 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.892450  7360 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.893872  7796 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.878078  7779 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.894945  7360 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.888784  7509 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.896428  7779 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.897672  7509 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.898185  7689 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.889423  7799 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.900703  7799 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:44.901579  7687 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:44.902093  7687 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:44.691468   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:44.904911  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:44.889063  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 649409us
I20260709 13:03:44.905990  1725 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:44.907428  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:44.907881  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:44.909235  1717 raft_consensus.cc:2410] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:44.914489  7800 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:44.903088  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:44.914733  7687 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.917450  7772 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:44.918160  7772 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.918911  1716 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:44.920593  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:44.921252  1719 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:44.922792  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:44.923916  7802 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:44.920694  7412 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:44.924516  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:44.927964  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:44.953126  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:44.954324  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.957509  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.959829  7812 raft_consensus.cc:2764] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:44.961182  7812 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:44.962811  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:44.963104  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:44.964017  1573 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 10.
I20260709 13:03:44.971665  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:44.972839  1570 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:44.975963  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:44.985625  6447 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.986272  6447 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:44.988463  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:44.990139  1573 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:44.862718  7782 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.918187  7687 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.994035  6447 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:44.962158  7763 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.953s	user 0.004s	sys 0.000s
I20260709 13:03:44.989483  1575 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:45.009809  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.010990  1575 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:44.931368  7333 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:45.016953  1642 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:45.018808  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.020718  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:44.878747  7744 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:44.988282  7746 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.025106  7782 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.026623  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.037706  1574 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:44.988802  7800 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:45.030661  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.034130  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.051443  1569 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:45.035986  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:44.963699  7738 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:45.053399  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.054205  1573 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:45.056721  1573 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:44.988415  7772 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:45.059574  7802 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:45.020031  7743 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.060899  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:45.046094  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.065595  1568 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:45.071183  1649 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:45.068444  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 203098us
I20260709 13:03:45.076035  6447 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:45.075126  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.080480  7743 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.083206  7743 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:45.053927  7738 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.084151  7746 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.085621  7746 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.090375  7746 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.091948  7746 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.093871  7746 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.097680  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:45.098453  1647 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.098353  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.099684  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.102707  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:45.104681  1721 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:45.108192  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:45.095403  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:45.111315  1645 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:45.111855  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.113067  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.114923  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.116981  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.119719  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.128283  1647 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:45.050104  7800 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:45.131095  1575 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:45.132963  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:45.134233  1645 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:45.120800  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.137337  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.137493  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.138175  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:45.147830  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.154001  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.156677  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.157280  1568 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.162029  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:45.165297  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.084192  7738 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.166066  1572 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.162671  1651 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.189353  7792 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:45.168581  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.168706  1571 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:45.161697  7333 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.170311  1724 raft_consensus.cc:1217] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[12.8-18.11]   Dedup: 14.9->[16.10-18.11]
I20260709 13:03:45.192054  1724 pending_rounds.cc:85] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 9
I20260709 13:03:45.192371  1724 pending_rounds.cc:107] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:45.192636  1724 raft_consensus.cc:2889] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.172441  7598 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:45.173815  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:45.112836  7795 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:45.175773  1644 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:03:45.176450  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55765.6us
I20260709 13:03:45.207384  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.208360  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.176951  7689 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:45.177402  7744 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:45.178274  7780 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.925s	user 0.005s	sys 0.000s
I20260709 13:03:45.179080  7746 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:45.181778  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 357268us
I20260709 13:03:45.223183  1572 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:45.185114  7802 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.163097  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.174381  7621 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:45.202775  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:45.169078  7806 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.264s
I20260709 13:03:45.209502  6447 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:45.232901  6447 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.233465  6447 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:45.209142  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.216912  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 853) took 1803 ms. Trace:
I20260709 13:03:45.220718  1651 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:03:45.224480  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.994s (SENT)
I20260709 13:03:45.237687  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:45.224496  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 116382us
I20260709 13:03:45.227893  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.206593  1575 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:03:45.251219  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.264640  7525 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:45.265349  1647 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:45.210206  7693 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:45.201274  7598 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:45.235950  1718 rpcz_store.cc:276] 0709 13:03:43.413312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.413565 (+   253us) service_pool.cc:224] Handling call
0709 13:03:43.414109 (+   544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.414632 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.414674 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.415427 (+   753us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.773176 (+357749us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:43.773250 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.773371 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:44.961704 (+1188333us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b24001fe5d8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830b079c4 00007fc830b05acb 00007fc830cc0dc9 00007fc830cc0a55 00007fc830cc009d 00007fc8338f210e 00007fc83390988c 00007fc833908bb6 00007fc8339087d8 00007fc833912221
0709 13:03:45.201865 (+240161us) raft_consensus.cc:1685] finished
0709 13:03:45.201890 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.216805 (+ 14915us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1582,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113,"spinlock_wait_cycles":60431232,"thread_start_us":357161,"threads_started":1,"wal-append.queue_time_us":523725}
I20260709 13:03:45.273139  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.284425  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.276376  1568 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:45.155067  7774 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 13:03:45.223237  7360 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:45.287886  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78765.8us
I20260709 13:03:45.284890  1646 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:45.221824  7795 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:45.294103  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.221588  7703 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.300331  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.289005  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.295785  6923 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.205420  7333 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.300992  1647 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:45.301440  6923 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.302671  7703 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.306638  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.307011  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.303840  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.307585  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:45.307866  1646 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:45.209082  7805 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.265211  7525 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:45.316758  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 356582us
I20260709 13:03:45.306638  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:45.310937  6923 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.298413  6447 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.283820  7693 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.286806  7806 consensus_queue.cc:1243] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:45.287381  7689 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:45.308214  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.318655  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.283295  7809 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:45.290189  7360 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.318540  1647 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:45.323225  7693 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.304376  7690 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.306344  7792 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.307478  7796 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:45.332166  1573 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:45.320595  7525 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.234387  7800 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.328394  7806 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.371616  7806 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.374723  7806 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.375399  7806 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:45.375947  7806 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.377439  7800 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:45.378897  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.380879  7813 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:45.354267  7808 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:03:45.396412  6447 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:45.355551  1568 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:03:45.399914  1568 pending_rounds.cc:85] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 10
I20260709 13:03:45.400300  1568 pending_rounds.cc:107] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:03:45.400766  1568 raft_consensus.cc:2889] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.354091  7796 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.339876  7360 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.419268  6447 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:45.422158  6447 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.422699  6447 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.432615  1569 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:45.439527  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.440821  1568 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:45.442363  1574 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:45.408954  7806 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.410197  7809 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:45.460827  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.420157  7796 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:45.465008  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.465764  1718 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:45.467670  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:45.465878  6923 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.358264  7792 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.470120  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.466830  1721 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.466926  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.339809  7795 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.445005  7621 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.480674  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.481128  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.469089  6923 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.481768  1722 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.458405  7774 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.458410  7081 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:45.489944  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.490531  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.491855  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.492871  1568 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
W20260709 13:03:45.495353  7807 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:03:45.496583  6923 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.465381  7744 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.497318  7744 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.499715  7744 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:45.500281  7081 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.501150  7690 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:45.502722  7081 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.502997  7792 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.467484  7689 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.465358  7809 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.504652  7809 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.505307  7765 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.505889  7765 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.506620  1716 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:45.474680  7742 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:45.515007  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:45.521273  7703 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:45.522509  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:45.523486  1725 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:03:45.523520  7806 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.408835  7598 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.531158  6923 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:45.531808  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.533406  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:45.539680  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 251927us
I20260709 13:03:45.532406  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:45.548327  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:45.549346  1716 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:45.554832  6447 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.479583  7450 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:03:45.559746  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.560389  1725 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.561201  1721 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:45.563687  1721 raft_consensus.cc:1275] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:03:45.555544  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 258818us
I20260709 13:03:45.568316  6447 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:45.568586  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 260323us
I20260709 13:03:45.547557  7742 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:45.500226  7738 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.601490  1725 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:45.593698  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:45.566895  7687 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:03:45.561375  7744 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:45.612195  1722 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:45.607537  1716 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:45.610440  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:45.572213  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 347563us
I20260709 13:03:45.603358  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:45.617311  1721 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.567718  7081 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:45.624380  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.625013  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 284713us
W20260709 13:03:45.635537  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.568192  7792 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:45.644021  1690 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.600834  7333 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:45.653224  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:45.639467  7791 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.984s	user 0.004s	sys 0.000s
I20260709 13:03:45.618886  7598 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.665308  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.674619  6447 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:45.675165  6447 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.675596  6447 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.618589  7805 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:45.679829  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.652434  7779 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:45.641111  7687 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:45.681568  1576 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:45.682025  1539 leader_election.cc:400] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 19.
I20260709 13:03:45.681589  7779 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.684719  1721 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:45.688983  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.692831  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:45.696691  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 164297us
I20260709 13:03:45.718770  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.719703  1575 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:45.725796  7792 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.677806  7825 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:45.687968  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 131017us
I20260709 13:03:45.730188  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.729491  6447 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.743778  6447 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:45.789286  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.824s (SENT)
W20260709 13:03:45.790896  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61542.7us
I20260709 13:03:45.740811  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:45.717463  7779 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.872136  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:45.869932  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 215540us
I20260709 13:03:45.732359  7825 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.743701  7799 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.886886  7792 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.885824  7765 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:45.888068  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 147285us
I20260709 13:03:45.892989  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:45.718191  7687 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:45.918645  7806 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:45.950067  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 160718us
W20260709 13:03:45.950814  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.202s (SENT)
W20260709 13:03:45.919265  7830 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:03:45.919667  7824 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
W20260709 13:03:45.921684  7800 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 13:03:45.923550  7842 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.889285  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:45.970752  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 81512.5us
I20260709 13:03:45.951236  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:45.953156  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60211.7us
W20260709 13:03:45.905064  7693 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.531s	user 0.003s	sys 0.007s
I20260709 13:03:45.883332  7825 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:45.789083  7827 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.183s
I20260709 13:03:45.888396  7833 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.974187  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:45.976747  1571 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:03:45.507226  7774 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:45.984838  1571 pending_rounds.cc:85] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 10
I20260709 13:03:45.979373  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.980466  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:45.982527  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:45.791662  7742 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.005921  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:45.985221  1571 pending_rounds.cc:107] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:03:46.008170  1571 raft_consensus.cc:2889] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:45.986034  1575 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:03:45.986961  6447 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:46.004256  6923 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.019634  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:45.925143  7799 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.029808  6923 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
W20260709 13:03:46.029865  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79046.9us
I20260709 13:03:46.052947  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 926) took 1529 ms. Trace:
W20260709 13:03:46.057283  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.030264  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 147078us
W20260709 13:03:46.065721  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.056180  1649 rpcz_store.cc:276] 0709 13:03:44.524000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.539240 (+ 15240us) service_pool.cc:224] Handling call
0709 13:03:46.052654 (+1513414us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b540038b8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:46.052845 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124589440}
W20260709 13:03:46.066152  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.069018  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:45.968381  7802 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:03:46.073482  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:46.074324  2375 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:46.073963  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:46.076910  2377 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.078286  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.078967  2371 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:46.079155  7687 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:46.079365  7525 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.000s
I20260709 13:03:46.080060  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:46.071028  7738 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:45.984118  7693 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.082765  7693 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:45.982055  7842 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.085305  2375 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
W20260709 13:03:46.085577  7796 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.000s
I20260709 13:03:46.085987  7796 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.086490  7842 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.027814  7509 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.079844  7525 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.088318  7796 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.092947  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.093268  7796 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:46.093724  2378 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:46.052417  7840 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:46.027913  7822 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.089625  7799 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.073561  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:46.093856  7796 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.096179  2373 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.027434  7792 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.096300  2374 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:46.105043  2379 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 19: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:46.047144  7823 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:45.984995  7839 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:03:46.047638  7847 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:46.052958  7851 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.091923  7525 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:46.137439  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.131114  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.012210  7833 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:46.148274  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140661us
I20260709 13:03:46.092291  7774 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.070452  7838 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 13:03:46.193115  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 2987) took 1042 ms. Trace:
I20260709 13:03:46.126415  7792 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:46.173731  7081 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
W20260709 13:03:46.197484  7826 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
I20260709 13:03:46.198658  7799 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:46.196676  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 127794us
W20260709 13:03:46.199656  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.147835  2377 raft_consensus.cc:2410] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:03:46.067806  7806 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:03:46.065445  7827 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:03:46.104374  7796 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:46.171171  7809 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.666s	user 0.005s	sys 0.003s
I20260709 13:03:46.211706  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:46.196375  7845 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:03:46.192436  7776 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:46.206538  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:46.197547  7814 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:46.216455  7081 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.192574  7509 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:46.230696  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:46.201514  1724 rpcz_store.cc:276] 0709 13:03:45.150659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.150806 (+   147us) service_pool.cc:224] Handling call
0709 13:03:45.170110 (+ 19304us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:03:45.193350 (+ 23240us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:45.472603 (+279253us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.740809 (+268206us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.741386 (+   577us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:45.744254 (+  2868us) log.cc:844] Serialized 52 byte log entry
0709 13:03:45.828789 (+ 84535us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:45.828860 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.828973 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:46.192834 (+363861us) raft_consensus.cc:1685] finished
0709 13:03:46.192860 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:46.193047 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21515,"prepare.queue_time_us":191616,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":13626,"raft.queue_time_us":133492,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":5448,"thread_start_us":434766,"threads_started":2,"wal-append.queue_time_us":682312}
I20260709 13:03:46.152416  7800 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:46.202222  7851 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.236444  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:46.201663  7847 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:46.242683  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:46.244367  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.226202  2373 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:46.213332  7703 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:46.270402  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63885.9us
I20260709 13:03:46.217295  7799 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:46.306914  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 521584us
I20260709 13:03:46.308043  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:46.315251  1644 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:46.317461  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 86953.7us
I20260709 13:03:46.240839  7776 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:46.307411  7826 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:03:46.315255  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:46.241382  7814 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:46.233111  7809 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:46.337873  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.524s (SENT)
I20260709 13:03:46.513621  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:46.514599  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.453735  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:46.429085  6447 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:46.517921  6447 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.460151  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:46.519399  1725 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:46.458174  1724 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:46.527297  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:46.305884  7464 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:46.530642  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 215350us
I20260709 13:03:46.531414  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:46.301395  7509 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:46.542735  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 204814us
I20260709 13:03:46.539544  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:46.308017  7841 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.308610  7802 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:46.361546  7360 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.917s	user 0.012s	sys 0.000s
I20260709 13:03:46.553056  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:46.381901  7598 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.000s
I20260709 13:03:46.312515  7525 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:46.353695  7848 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:03:46.538277  7412 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:46.548777  6447 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.310125  7851 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.554299  7839 consensus_queue.cc:1048] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:03:46.538022  7813 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:46.555514  7854 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.383615  7792 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.428999  7772 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:46.336295  7835 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:46.392364  7833 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.439435  7822 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.538264  7690 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.356587  7827 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:46.551857  7693 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.537885  7687 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.562538  1614 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:46.546378  7806 raft_consensus.cc:2764] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:03:46.592154  7860 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:46.318404  7703 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.566298  6447 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.357257  7844 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.205s
I20260709 13:03:46.558506  7842 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.357851  7847 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:46.570907  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.007s	sys 0.078s
W20260709 13:03:46.592736  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 280843us
I20260709 13:03:46.556200  7360 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.361377  7776 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:46.659853  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 97340.3us
I20260709 13:03:46.592198  7792 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:46.674221  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.007s	sys 0.078s
I20260709 13:03:46.556658  7841 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.675099  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:46.597834  7822 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.596832  7772 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.359505  7799 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.671705  7860 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:46.569754  7813 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.666136  7687 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.666950  7827 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.596563  7509 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.658340  7690 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.687098  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:46.690344  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:46.687422  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 679532us
I20260709 13:03:46.714844  1616 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:46.668992  7643 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:46.680101  7840 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:46.595288  7835 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.722218  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:46.686453  7776 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.690551  7598 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:46.737371  7598 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:46.737922  7822 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:46.679462  7809 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.684965  7850 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:46.724850  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:46.738338  7598 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.739848  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:46.740507  6447 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.740904  6447 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.741233  6447 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.722095  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:46.744851  7844 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:46.721535  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 175442us
I20260709 13:03:46.746263  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:46.693423  7360 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.750192  1724 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:46.751348  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:46.752861  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:46.753453  1722 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:03:46.691416  7772 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.731606  1721 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:46.724774  7840 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:46.756831  7840 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.758113  7840 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:46.727895  7863 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.760691  7863 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.761085  7863 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.762064  7687 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:46.728112  7643 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:46.764043  6447 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.765210  7643 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.766973  6447 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.734251  7860 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.769810  7813 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:46.772787  2374 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: [17.13-18.14]
I20260709 13:03:46.772953  7822 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.773595  7827 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.774518  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.776661  7690 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.592689  7814 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.777834  7814 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.776074  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:46.774451  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:46.779464  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57447.5us
W20260709 13:03:46.783186  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.783229  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.783186  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:46.803946  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.677492  7842 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:46.809775  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87630.8us
I20260709 13:03:46.695724  7693 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:03:46.791635  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:46.730873  7866 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:46.818353  7822 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:46.778249  7839 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:46.687403  7792 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.820518  1616 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:46.825763  7835 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:46.821086  7690 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.835058  7851 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:46.839143  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.808326  7856 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:46.790365  7799 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:46.867717  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47244.9us
I20260709 13:03:46.886229  1568 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:03:46.853910  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62318.5us
W20260709 13:03:46.862737  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 53607.4us
I20260709 13:03:46.881503  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:46.892201  1573 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:46.883263  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:46.849296  7850 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:46.856660  7839 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.890779  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:46.846138  7779 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.957s	user 0.010s	sys 0.001s
I20260709 13:03:46.894518  1575 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.894652  7779 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.896754  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:46.897388  1574 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:46.847934  7825 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.000s
I20260709 13:03:46.898015  7825 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.899255  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.899876  2373 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:46.884516  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:46.900734  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:46.900859  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.884521  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:46.901700  7851 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:46.901717  1569 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:46.856873  7869 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:46.861339  7841 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:46.887885  7856 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:46.904579  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.904407  7856 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.852592  7871 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:03:46.905263  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:46.902959  2374 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:46.906009  1721 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:46.907016  7853 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:46.882182  7829 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.859004  7792 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.896852  7779 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:46.894110  7839 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.900043  7825 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:46.927459  1567 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.928339  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:46.930588  2374 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.911090  1574 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:46.920938  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:46.928730  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:46.931432  1569 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
W20260709 13:03:46.849238  7796 consensus_meta.cc:287] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:03:46.935956  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:46.937004  1569 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:46.936431  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 377000us
I20260709 13:03:46.938856  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:46.867987  7742 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:46.902962  7869 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.957362  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:46.866845  7703 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:46.975453  7795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.006s	sys 0.015s
W20260709 13:03:47.018738  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.871295  7690 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.957939  7742 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.908304  7829 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:46.947819  7796 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.840291  7863 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:46.973284  7869 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:46.996732  7850 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:46.958434  7840 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:03:47.125705  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:46.947400  7779 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:46.930436  7744 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
W20260709 13:03:47.128820  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 189941us
I20260709 13:03:47.131450  1567 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:47.131848  1569 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:46.926846  7831 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 13:03:47.130111  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 238981us
I20260709 13:03:47.128842  1575 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:47.127784  7809 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.099105  7765 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.006s	sys 0.010s
I20260709 13:03:47.129539  7812 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:47.129315  7792 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:47.128415  7825 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:47.147657  1570 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:47.148873  1725 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:47.163767  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:47.168713  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 281626us
I20260709 13:03:47.188670  1571 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:47.146495  7690 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:47.172695  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:47.192577  7812 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:47.192968  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:47.193686  7812 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.195281  7800 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:47.196013  7800 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:47.171981  7779 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.196552  7800 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:47.196714  7779 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.197532  7796 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:47.197474  7800 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.174098  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:47.199366  7863 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:47.199960  7863 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:47.144471  7742 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:47.201210  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:47.176021  7809 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:47.203545  7809 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:47.204489  7792 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:47.205428  7412 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:47.206543  7792 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:47.207523  6923 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:47.207988  6923 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:47.161295  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.157553  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.185004  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:47.194090  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:47.141999  7689 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.214238  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.141709  2374 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:47.175048  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:47.148381  7795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.007s	sys 0.020s
I20260709 13:03:47.217465  1575 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.218214  1725 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:47.218117  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.240644  1648 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.217902  7689 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.218756  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.218766  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.220322  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:47.220824  7703 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:47.253748  7703 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:47.254706  7703 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.259073  7742 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:47.222724  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.260751  2378 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:47.224290  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.208293  7850 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.208956  7746 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
W20260709 13:03:47.288945  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.285640  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:47.293232  2380 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:47.208973  7842 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:47.294618  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.294829  7746 consensus_queue.cc:1243] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer da3b57321031416cba9db75450ef302a log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:47.215286  7825 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.296633  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.296944  7825 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:47.297397  1573 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:47.298722  1576 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:47.287307  7779 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:47.285640  7742 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:47.300565  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.301404  1570 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:47.289022  7870 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:47.302381  7870 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:47.302546  7779 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:47.302623  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.308698  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:47.309463  1573 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.288968  7689 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:47.314154  7870 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:47.315778  7742 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:47.316538  7742 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.225765  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:47.226603  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.227011  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.227262  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.228093  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:47.229856  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:47.230494  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.230729  1651 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:47.231312  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:47.232856  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.234054  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:47.234483  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:47.234714  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.216374  7822 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:03:47.208766  6923 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:47.223934  1649 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:47.147271  7829 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:47.241463  2379 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:47.244071  1718 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:47.181945  7765 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.006s	sys 0.010s
I20260709 13:03:47.234994  7690 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:03:47.249341  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.249456  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.250268  1723 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:47.250213  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:47.251046  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.261955  2377 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:47.328051  1573 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:47.388940  2373 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.357698  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:47.389837  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:47.390288  2380 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.390498  1573 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.390863  1574 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:47.391429  1721 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:47.351598  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.267587  1575 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:47.390549  1569 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.393345  1650 raft_consensus.cc:2376] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16. Re-sending same reply.
I20260709 13:03:47.359221  1717 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:47.392931  2375 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
W20260709 13:03:47.400408  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 169071us
I20260709 13:03:47.392786  1716 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:03:47.400800  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 149195us
W20260709 13:03:47.395941  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 167165us
I20260709 13:03:47.407408  2371 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:47.407562  1646 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:47.301683  7814 consensus_meta.cc:287] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.005s
I20260709 13:03:47.410844  7643 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.411549  7643 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.413709  7863 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:47.414283  7863 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:47.268477  7743 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:47.411286  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.411716  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:47.419122  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 193310us
I20260709 13:03:47.429932  1570 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:47.411336  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.410923  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.430055  1648 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:47.439155  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.444902  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.452316  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:47.445444  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.453094  1725 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.450026  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.453661  1645 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:47.454010  1651 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:03:47.448014  1567 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:47.456369  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.481624  1569 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:47.455744  1573 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:47.485833  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.487207  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:47.488245  1717 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 15: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:47.485901  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.489600  2371 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:47.493363  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.288443  7509 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:47.506275  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.523762  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.523640  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:47.603982  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.463883  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.654258  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.733944  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:47.734992  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:47.456108  7743 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:47.595237  7621 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.712s	user 0.016s	sys 0.032s
I20260709 13:03:47.284788  7412 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:47.943069  7863 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:47.944887  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:48.132020  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 724572us
I20260709 13:03:47.456455  7779 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:47.313023  7839 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:47.924714  1720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.007s	sys 0.007s
I20260709 13:03:47.456744  7825 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:47.962201  7802 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.965786  7746 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.927583  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.004s	sys 0.011s
W20260709 13:03:48.005479  1716 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.039s	sys 0.084s
W20260709 13:03:48.160720  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.004s	sys 0.011s
W20260709 13:03:48.160917  1716 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.039s	sys 0.084s
I20260709 13:03:48.010103  1717 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:48.027827  7643 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.164057  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:48.053004  7850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.002s	sys 0.020s
W20260709 13:03:48.056306  1725 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.012s
I20260709 13:03:48.079748  1646 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.166339  1725 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:48.079914  1649 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:48.083783  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:48.089346  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 632905us
I20260709 13:03:48.197129  1690 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:48.118506  1722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 875) took 1213 ms. Trace:
W20260709 13:03:48.119393  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:48.079293  2373 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.003s
I20260709 13:03:48.199615  1651 raft_consensus.cc:2368] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17.
I20260709 13:03:48.199810  2373 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
W20260709 13:03:48.147178  1647 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.000s
I20260709 13:03:48.202659  1647 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
W20260709 13:03:48.080281  1572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.011s	sys 0.001s
I20260709 13:03:48.119743  1648 raft_consensus.cc:2410] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:03:48.203783  1572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.011s	sys 0.001s
W20260709 13:03:48.128259  2380 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.005s
I20260709 13:03:48.204706  2380 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:48.081131  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.198477  1722 rpcz_store.cc:276] 0709 13:03:46.904598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.906707 (+  2109us) service_pool.cc:224] Handling call
0709 13:03:46.907062 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.907768 (+   706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.907798 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:46.908235 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:03:47.170690 (+262455us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:47.170749 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.170833 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.002879 (+832046us) raft_consensus.cc:1685] finished
0709 13:03:48.002903 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.118423 (+115520us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1012,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":12553,"thread_start_us":261990,"threads_started":1,"wal-append.queue_time_us":1091183}
I20260709 13:03:48.205978  1645 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
W20260709 13:03:48.127864  1724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.006s	sys 0.005s
W20260709 13:03:48.207275  1724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.005s
I20260709 13:03:48.150974  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.149341  2371 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:48.152906  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:48.052091  7869 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.882s	user 0.015s	sys 0.012s
W20260709 13:03:48.155761  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.007s	sys 0.007s
W20260709 13:03:48.128093  7621 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.252s	user 0.016s	sys 0.033s
I20260709 13:03:48.158751  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.159283  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:48.121569  7703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.019s	sys 0.079s
I20260709 13:03:48.158810  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:48.074549  7870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.002s	sys 0.003s
I20260709 13:03:48.162700  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:48.213403  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:48.154743  7509 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.002s
I20260709 13:03:48.167600  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.082553  7598 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.148722  7823 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:48.069046  7742 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.127282  2379 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:48.157558  7825 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.158985  7779 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:48.162014  7866 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.166107  7850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.002s	sys 0.020s
I20260709 13:03:48.230701  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:48.218710  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.226497  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.219792  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.219969  1719 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:48.216360  7870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.002s	sys 0.003s
I20260709 13:03:48.234539  1723 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
W20260709 13:03:48.216698  7703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.019s	sys 0.079s
I20260709 13:03:48.215160  7690 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:48.166091  7746 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.224586  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:48.264773  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:48.265702  1648 raft_consensus.cc:2393] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:48.236141  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.233577  1645 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:48.152735  7412 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.238508  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:48.208992  7871 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.268244  1723 raft_consensus.cc:2368] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:48.268790  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:48.269709  1642 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:48.242782  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.242611  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:48.271181  1647 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.235261  1721 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:48.242621  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:48.220724  7825 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.273442  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.217026  7823 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:48.164287  7802 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.275833  7802 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.276742  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 714) took 1055 ms. Trace:
I20260709 13:03:48.277007  7823 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.278100  1717 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:48.277149  2374 rpcz_store.cc:276] 0709 13:03:47.221690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.228755 (+  7065us) service_pool.cc:224] Handling call
0709 13:03:47.229021 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.486249 (+257228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.486279 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.486747 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.137828 (+651081us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:48.137872 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.137946 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.276435 (+138489us) raft_consensus.cc:1685] finished
0709 13:03:48.276458 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.276673 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":714501,"raft.queue_time_us":294129,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":36389,"thread_start_us":190191,"threads_started":1,"wal-append.queue_time_us":474686}
W20260709 13:03:48.214345  7812 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.026s	sys 0.026s
I20260709 13:03:48.216872  7869 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.281361  7869 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.267202  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.210145  7643 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:03:48.278321  7823 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.230895  7509 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:48.292714  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.297308  7839 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:48.297535  1719 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:48.297993  7879 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.298635  7887 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.298s	user 0.003s	sys 0.003s
I20260709 13:03:48.298599  7879 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.299026  1719 raft_consensus.cc:1217] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 12.6->[13.7-15.8]   Dedup: 13.7->[15.8-15.8]
I20260709 13:03:48.299623  1719 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 13:03:48.276172  1644 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 11.
I20260709 13:03:48.269553  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:48.293025  7850 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:48.306692  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.307463  1651 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:03:48.309444  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.305876  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:48.226519  7598 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.285136  1725 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:48.270089  7893 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:03:48.286434  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.281375  7412 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.316941  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.317294  1647 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:48.234139  7866 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.287336  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.287498  7746 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.271746  7765 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:48.288424  1646 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:48.293555  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.294920  1722 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:03:48.300395  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 878) took 1088 ms. Trace:
I20260709 13:03:48.245841  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.311308  7743 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.310686  7879 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.303238  7850 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.315243  1719 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:48.316653  1723 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:48.284176  7812 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.027s	sys 0.027s
I20260709 13:03:48.284854  1717 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:03:48.328202  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.379240  1645 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:48.324909  1642 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:03:48.325426  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:48.325781  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3011) took 1065 ms. Trace:
I20260709 13:03:48.327085  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2310) took 1065 ms. Trace:
I20260709 13:03:48.298810  7509 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.379992  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2308) took 1131 ms. Trace:
I20260709 13:03:48.380424  1724 rpcz_store.cc:276] 0709 13:03:47.260395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.260561 (+   166us) service_pool.cc:224] Handling call
0709 13:03:47.285014 (+ 24453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.327355 (+ 42341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.327399 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.327999 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.207819 (+879820us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:48.207877 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.207998 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.325519 (+117521us) raft_consensus.cc:1685] finished
0709 13:03:48.325541 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.325711 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56107,"raft.queue_time_us":42830,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":39043,"thread_start_us":640513,"threads_started":1,"wal-append.queue_time_us":667585}
I20260709 13:03:48.288784  7871 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.380715  1647 rpcz_store.cc:276] 0709 13:03:47.261280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.326045 (+1064765us) service_pool.cc:224] Handling call
0709 13:03:48.326443 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.326664 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.326706 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.326753 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:48.326798 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.326856 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.326996 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.384155  1651 rpcz_store.cc:276] 0709 13:03:47.260753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.332123 (+1071370us) service_pool.cc:224] Handling call
0709 13:03:48.379264 (+ 47141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.379540 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.379572 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.379620 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:48.379697 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.379761 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.379916 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:48.380488  1649 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:48.379197  1720 rpcz_store.cc:276] 0709 13:03:47.211594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.211804 (+   210us) service_pool.cc:224] Handling call
0709 13:03:47.212122 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.212470 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.212508 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.235843 (+ 23335us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.208954 (+973111us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:48.208982 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.209025 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.298184 (+ 89159us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b24001fe6f8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830b079c4 00007fc830b05acb 00007fc830cc0dc9 00007fc830cc0a55 00007fc830cc009d 00007fc8338f210e 00007fc83390988c 00007fc833908bb6 00007fc8339087d8 00007fc833912221
0709 13:03:48.299969 (+  1785us) raft_consensus.cc:1685] finished
0709 13:03:48.299995 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.300283 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2604,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":26672,"spinlock_wait_cycles":19661696,"thread_start_us":972734,"threads_started":1,"wal-append.queue_time_us":763783}
I20260709 13:03:48.407843  1650 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 13:03:48.408229  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 82864.8us
I20260709 13:03:48.407909  1642 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:48.408365  7870 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:48.411526  1650 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:48.407831  1646 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:48.315052  7839 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:48.421849  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.418077  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.424129  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.424888  1649 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:03:48.422885  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:48.385883  7850 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.428126  1647 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:48.432778  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.433055  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.429754  7598 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:48.427094  7746 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:03:48.434996  1642 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:48.435686  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.436888  1720 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:48.314662  7882 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:03:48.420238  7765 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:48.438482  1723 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:48.440184  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.419027  7743 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:48.442016  6923 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:48.442446  6923 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:48.443297  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.444815  1717 raft_consensus.cc:2368] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
I20260709 13:03:48.446987  7742 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:48.448758  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.451385  7825 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.432916  7412 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:48.424947  7870 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.314303  7779 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.422099  7839 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.447115  7743 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:48.460780  6923 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.478840  6923 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:48.483129  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.477279  7839 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.492879  7509 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:48.495426  7598 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.507849  7412 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:48.571547  1539 connection.cc:256] RPC call timeout handler was delayed by -3.3303580289999672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:03:48.584908  1539 connection.cc:256] RPC call timeout handler was delayed by -3.1521580289999633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 13:03:48.585534  1539 connection.cc:256] RPC call timeout handler was delayed by -2.4213580289999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2541s for a response
W20260709 13:03:48.585963  1539 connection.cc:256] RPC call timeout handler was delayed by -1.7436580289999597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3294s for a response
I20260709 13:03:48.586983  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:48.572392  7898 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:48.589632  7886 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.589s	user 0.005s	sys 0.000s
I20260709 13:03:48.591112  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2692) took 1362 ms. Trace:
W20260709 13:03:48.591748  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.544730  7765 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.575968  7802 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:48.596493  7802 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.591535  1568 rpcz_store.cc:276] 0709 13:03:47.228641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.232420 (+  3779us) service_pool.cc:224] Handling call
0709 13:03:47.232707 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.233015 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.233048 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.233508 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.161292 (+927784us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:48.161349 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.161437 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.588983 (+427546us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:48.590779 (+  1796us) raft_consensus.cc:1685] finished
0709 13:03:48.590802 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.591036 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":394,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":3554688,"thread_start_us":927446,"threads_started":1,"wal-append.queue_time_us":767235}
I20260709 13:03:48.588876  7898 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.599403  7802 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.600410  7898 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:48.601215  7412 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.602160  7820 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.584374  7598 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.603143  7820 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.605706  7598 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:48.607053  7894 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:03:48.613674  7870 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.614825  7825 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:48.613561  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:48.600239  7802 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:48.606815  7820 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:48.617763  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.618019  7870 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:48.618884  7870 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:48.620832  7870 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.622087  7746 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:48.622588  7746 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:48.625864  7888 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.361s	user 0.005s	sys 0.001s
I20260709 13:03:48.628670  7743 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.633663  1568 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:03:48.634464  1568 pending_rounds.cc:85] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:03:48.566793  7812 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:48.634825  1568 pending_rounds.cc:107] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:03:48.635226  1568 raft_consensus.cc:2889] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:48.635131  7812 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.633698  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2296) took 1421 ms. Trace:
I20260709 13:03:48.637095  1644 rpcz_store.cc:276] 0709 13:03:47.212181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.212377 (+   196us) service_pool.cc:224] Handling call
0709 13:03:48.633461 (+1421084us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54003fcd54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:48.633622 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2248355584}
I20260709 13:03:48.637857  7812 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.642289  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.572880  7703 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.644399  7703 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:48.645865  7891 log.cc:927] Time spent T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.375s	user 0.001s	sys 0.002s
I20260709 13:03:48.646111  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.647495  1568 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:48.647773  7703 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:48.648783  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.649248  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3044) took 1336 ms. Trace:
I20260709 13:03:48.650048  1573 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:48.653689  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.655025  1568 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:48.657380  1573 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:48.661813  7866 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:48.664201  7883 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:48.665019  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.666033  1569 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:48.668889  7873 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.665s	user 0.004s	sys 0.000s
W20260709 13:03:48.671341  7897 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.403s	user 0.003s	sys 0.000s
I20260709 13:03:48.649881  1572 rpcz_store.cc:276] 0709 13:03:47.312959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.359402 (+ 46443us) service_pool.cc:224] Handling call
0709 13:03:47.359762 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.360167 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.360196 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.360733 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.204380 (+843647us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:48.204442 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.204541 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.644849 (+440308us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:48.648749 (+  3900us) raft_consensus.cc:1685] finished
0709 13:03:48.648780 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.649058 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":531,"raft.queue_time_us":2715,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":18310,"spinlock_wait_cycles":13692160,"thread_start_us":843214,"threads_started":1,"wal-append.queue_time_us":853259}
I20260709 13:03:48.675897  7871 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:48.664978  7883 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:48.677733  7883 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.679211  7883 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.679782  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.680554  1572 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.690418  7866 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:48.691329  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.691355  7866 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:48.692453  1573 raft_consensus.cc:2368] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17.
I20260709 13:03:48.691975  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.623641  7746 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:48.694125  7746 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:48.694268  7866 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:48.696761  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.699100  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.699904  1567 raft_consensus.cc:2376] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 16. Re-sending same reply.
I20260709 13:03:48.701618  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.702373  1567 raft_consensus.cc:2376] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 16. Re-sending same reply.
I20260709 13:03:48.704376  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.705516  1568 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:03:48.676754  1569 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:03:48.715139  1572 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:48.724387  1574 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:48.720983  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.726258  1573 raft_consensus.cc:2368] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17.
W20260709 13:03:48.721844  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.050s	sys 0.042s
W20260709 13:03:48.727506  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.050s	sys 0.042s
W20260709 13:03:48.728036  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 568463us
W20260709 13:03:48.731205  2344 connection.cc:256] RPC call timeout handler was delayed by -1.2687162839999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
W20260709 13:03:48.731805  2344 connection.cc:256] RPC call timeout handler was delayed by -1.2156162840000206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
W20260709 13:03:48.732532  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.695s (SENT)
W20260709 13:03:48.732805  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.001s	sys 0.010s
W20260709 13:03:48.733465  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.001s	sys 0.010s
W20260709 13:03:48.733642  2344 connection.cc:256] RPC call timeout handler was delayed by -1.1208305530000189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 13:03:48.734402  2344 connection.cc:256] RPC call timeout handler was delayed by -1.0212162839999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:03:48.729336  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.005s
W20260709 13:03:48.735970  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.006s	sys 0.006s
W20260709 13:03:48.736737  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 652680us
W20260709 13:03:48.738829  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.011s
W20260709 13:03:48.733875  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 537079us
I20260709 13:03:48.747742  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.727615  7879 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:48.750097  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.772217  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:48.774752  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.783265  2375 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:48.748364  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.002s	sys 0.010s
W20260709 13:03:48.784210  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.003s	sys 0.010s
I20260709 13:03:48.784554  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:48.760555  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.005s	sys 0.008s
W20260709 13:03:48.786154  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.006s	sys 0.009s
W20260709 13:03:48.778858  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.003s	sys 0.011s
W20260709 13:03:48.786849  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 589446us
W20260709 13:03:48.761957  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.022s	sys 0.061s
I20260709 13:03:48.780714  2371 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:48.788496  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.024s	sys 0.061s
W20260709 13:03:48.760468  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.013s
I20260709 13:03:48.787704  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.787621  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.783502  1572 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:03:48.788674  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:48.789500  1569 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:48.788969  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.000s	sys 0.014s
I20260709 13:03:48.762679  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:48.794678  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.795881  1717 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:03:48.797654  2379 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:03:48.731258  7703 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:48.676240  7839 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:48.785130  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 635726us
I20260709 13:03:48.798249  2379 pending_rounds.cc:85] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 8
W20260709 13:03:48.864910  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 102359us
I20260709 13:03:48.874955  2379 pending_rounds.cc:107] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:03:48.875468  2379 raft_consensus.cc:2889] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:48.836177  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.832777  1573 raft_consensus.cc:2393] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:48.870324  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71565.1us
I20260709 13:03:48.869485  2380 pending_rounds.cc:85] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 11
I20260709 13:03:48.877600  2380 pending_rounds.cc:107] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:03:48.877907  2380 raft_consensus.cc:2889] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:48.835348  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:48.857101  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.880257  1717 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:48.862339  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:48.858139  1720 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:48.882838  1724 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:48.754403  7866 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.884473  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:48.876247  1569 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:48.888298  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.888890  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:48.894028  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:48.879840  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 55126.4us
W20260709 13:03:48.899887  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.703s (SENT)
I20260709 13:03:48.900913  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.902132  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.906566  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:48.906610  1650 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:48.909538  1568 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:48.909263  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:49.046762  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 162712us
I20260709 13:03:49.445896  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:49.120770  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.029s	sys 0.066s
W20260709 13:03:49.448447  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.035s	sys 0.066s
I20260709 13:03:49.449105  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:48.988910  7866 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:49.450899  1723 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:49.452214  7907 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:49.213066  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2326) took 1268 ms. Trace:
W20260709 13:03:49.453562  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 546786us
I20260709 13:03:49.454543  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:49.454959  7866 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:49.453787  1651 rpcz_store.cc:276] 0709 13:03:47.944833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.405529 (+460696us) service_pool.cc:224] Handling call
0709 13:03:48.423851 (+ 18322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.424084 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:48.777302 (+353218us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.212454 (+435152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.212547 (+    93us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:49.212629 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.212754 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.212966 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109877,"raft.queue_time_us":148689,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":164410,"thread_start_us":658423,"threads_started":2}
I20260709 13:03:48.884461  7799 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:48.987959  1569 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:49.457069  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:49.173633  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.457828  1568 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.864748  7509 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.459466  7509 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.460901  1567 raft_consensus.cc:2368] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
I20260709 13:03:49.461910  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.461982  7509 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:49.252522  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.007882  7690 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.455s
W20260709 13:03:49.047991  7823 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.000s
I20260709 13:03:49.466272  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:48.872231  7800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.166s	user 0.014s	sys 0.006s
W20260709 13:03:49.467438  7800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.858s	user 0.024s	sys 0.022s
W20260709 13:03:49.307021  2374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.008s	sys 0.010s
W20260709 13:03:49.468676  2374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.014s	sys 0.010s
W20260709 13:03:49.469148  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 568974us
W20260709 13:03:49.317233  7743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.019s
W20260709 13:03:49.471401  7743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.000s	sys 0.025s
I20260709 13:03:48.907308  7703 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.473335  7909 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.474828  7703 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:49.475718  1571 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:03:49.203065  2375 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:49.476758  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.476986  2371 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:49.477496  1571 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:49.041565   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:48.868530  7621 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.016s	sys 0.016s
W20260709 13:03:49.484414  7621 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.036s	sys 0.023s
I20260709 13:03:49.000240  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.047685  7742 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.436s
I20260709 13:03:48.864564  7825 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:49.501706  7850 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.060s	user 0.014s	sys 0.001s
I20260709 13:03:48.906908  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:49.094434  7779 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:49.000293  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:48.823119  7879 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:49.489602  1716 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.011s	sys 0.010s
W20260709 13:03:49.518286  1716 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.012s	sys 0.011s
I20260709 13:03:49.092474  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:49.466419  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.447620  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:49.455703  7866 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:49.450982  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:49.532630  1648 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
W20260709 13:03:49.471747  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.921s (SENT)
I20260709 13:03:49.511946  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:49.534103  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:49.508468  1567 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:49.534274  2375 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.988656  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:49.521446  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73873.8us
W20260709 13:03:49.534986  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 80489.2us
I20260709 13:03:49.535844  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:49.498835  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.499797  1573 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:49.517674  1570 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:49.524170  1568 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:49.519203  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2325) took 1574 ms. Trace:
I20260709 13:03:49.496418  7914 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:49.535830  1690 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:49.492866  1650 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:49.539155  1573 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:49.547025  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.548163  1573 raft_consensus.cc:2410] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:03:49.536736  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3014) took 2110 ms. Trace:
I20260709 13:03:49.555409  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:49.507383  7825 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:49.557291  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 85579.5us
W20260709 13:03:49.560761  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.980s (SENT)
I20260709 13:03:49.560621  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:49.561444  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:49.561719  1649 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:03:49.575213  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.581355  1570 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:49.510418  7800 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:49.538865  1645 rpcz_store.cc:276] 0709 13:03:47.944456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.397079 (+452623us) service_pool.cc:224] Handling call
0709 13:03:48.404482 (+  7403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.404779 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:48.583064 (+178285us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.518681 (+935617us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.518760 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:49.518836 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.518934 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.519121 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34329,"prepare.queue_time_us":710567,"prepare.run_wall_time_us":41,"raft.queue_time_us":34558,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"spinlock_wait_cycles":62494464,"thread_start_us":549220,"threads_started":2,"wal-append.queue_time_us":965517}
W20260709 13:03:49.582367  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 593477us
I20260709 13:03:49.582260  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:49.585335  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.997s (SENT)
I20260709 13:03:49.615978  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:49.617480  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:49.617857  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.000s (SENT)
I20260709 13:03:49.618486  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:49.582929  1568 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 5 to 12, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:49.534307  7907 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:49.608004  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2622) took 1432 ms. Trace:
I20260709 13:03:49.634948  1718 rpcz_store.cc:276] 0709 13:03:48.150542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.150676 (+   134us) service_pool.cc:224] Handling call
0709 13:03:49.583143 (+1432467us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54002959d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:49.583325 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843699584}
I20260709 13:03:49.541409  7812 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:49.541013  7779 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:49.586901  1651 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:49.637614  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.040s (SENT) [suppressed 15 similar messages]
I20260709 13:03:49.577060  7825 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:49.575639  7831 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:49.646761  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:03:49.645272  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 89898.6us
I20260709 13:03:49.643621  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:49.581313  7598 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:49.576393  7914 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:49.660061  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:49.666867  1690 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:49.527465  7850 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:49.672097  1614 leader_election.cc:400] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
I20260709 13:03:49.613745  7870 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.666061  2375 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:49.634680  7899 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:49.673174  6923 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:03:49.613690  7871 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.676396  1614 leader_election.cc:403] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:49.607282  7916 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:49.669471  7509 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:49.688482  1474 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 208) took 71139 ms (client timeout 15000 ms). Trace:
W20260709 13:03:49.689764  7913 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:03:49.654522  7823 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.689550  7598 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:49.775854  7598 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.776448  7598 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:49.777482  7746 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.083s	user 0.016s	sys 0.000s
W20260709 13:03:49.691254  7920 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:03:49.651401  7812 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.648774  7908 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:49.731630  7693 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.474227  7909 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:49.733157  7912 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 13:03:49.675285  7918 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 13:03:49.639078  7765 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:49.653132  7450 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.024s
W20260709 13:03:49.741525  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 69545us
I20260709 13:03:49.740569  1723 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:03:49.669778  7799 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:49.634042  7800 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:49.738639  2380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.008s	sys 0.039s
I20260709 13:03:49.669631  7906 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:49.613741  7866 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:49.756292  7879 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:49.735245  7899 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.761030  7932 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:49.769595  7850 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:49.734663  7870 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:49.771176  7703 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:49.732211  7928 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:03:49.663631  7744 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:03:49.784586  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:49.732189  7922 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:03:49.691689  7882 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 5 to 12, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:03:49.751381  1474 rpcz_store.cc:269] 0709 13:02:38.369236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.369460 (+   224us) service_pool.cc:224] Handling call
0709 13:03:49.503302 (+71133842us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:40.594850 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:40.645180 (+ 50330us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:40.645236 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:03:40.645267 (+    31us) tablet.cc:662] Decoding operations
0709 13:03:40.645999 (+   732us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:40.646092 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:40.646117 (+    25us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:03:40.646708 (+   591us) tablet.cc:701] Row locks acquired
0709 13:03:40.646762 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:03:40.646940 (+   178us) write_op.cc:270] Start()
0709 13:03:40.647095 (+   155us) write_op.cc:276] Timestamp: P: 1783602220646878 usec, L: 0
0709 13:03:40.647115 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:03:40.715403 (+ 68288us) log.cc:844] Serialized 1696 byte log entry
0709 13:03:41.986330 (+1270927us) spinlock_profiling.cc:243] Waited 6.56 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:42.265521 (+279191us) op_driver.cc:464] REPLICATION: finished
0709 13:03:42.373042 (+107521us) write_op.cc:301] APPLY: starting
0709 13:03:42.373169 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:42.396140 (+ 22971us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:42.396191 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:42.398668 (+  2477us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:42.399015 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:03:42.399061 (+    46us) write_op.cc:312] APPLY: finished
0709 13:03:42.545482 (+146421us) log.cc:844] Serialized 49 byte log entry
0709 13:03:42.589614 (+ 44132us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:42.622855 (+ 33241us) write_op.cc:454] Released schema lock
0709 13:03:42.650705 (+ 27850us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:42.766242 (+115537us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:42.769214 (+  2972us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6136391,"reported_tablets":52,"spinlock_wait_cycles":2304,"child_traces":[["op",{"apply.queue_time_us":95935,"apply.run_cpu_time_us":9756,"apply.run_wall_time_us":382123,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20871,"num_ops":4,"prepare.queue_time_us":13918,"prepare.run_cpu_time_us":23159,"prepare.run_wall_time_us":270541,"raft.queue_time_us":211591,"raft.run_cpu_time_us":19067,"raft.run_wall_time_us":64146,"replication_time_us":1618169,"spinlock_wait_cycles":15083776,"thread_start_us":363362,"threads_started":4,"wal-append.queue_time_us":147052,"wal-append.run_cpu_time_us":26867,"wal-append.run_wall_time_us":2694119}]]}
W20260709 13:03:49.857048  7934 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:03:49.795075  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 157406us
W20260709 13:03:49.860944  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.196s (SENT)
I20260709 13:03:49.861532  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:49.755443  7916 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:49.800153  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140129us
I20260709 13:03:49.871488  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:49.123502  1466 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.751194  7509 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:03:49.805704  2380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.008s	sys 0.039s
I20260709 13:03:49.835376  7899 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:49.875372  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 870) took 1099 ms. Trace:
I20260709 13:03:49.790424  7908 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:49.791862  7779 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.749148  7917 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
I20260709 13:03:49.809571  7909 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:49.795895  7693 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:49.802999  7799 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:49.766551  7914 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:49.893813  7799 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:49.803277  7450 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.008s	sys 0.036s
W20260709 13:03:49.831380  1614 connection.cc:256] RPC call timeout handler was delayed by -1.1554303419999883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:03:49.901432  1614 connection.cc:256] RPC call timeout handler was delayed by -1.0272804900000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
I20260709 13:03:49.902868  7916 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:49.827129  7746 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.903873  7746 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.906021  7879 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:49.906719  7746 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:49.907289  7746 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:49.909699  1466 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 5 to 13, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:49.916589  7917 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:49.917304  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60218 (request call id 881) took 1133 ms. Trace:
I20260709 13:03:49.917694  1723 rpcz_store.cc:276] 0709 13:03:48.783975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.784137 (+   162us) service_pool.cc:224] Handling call
0709 13:03:49.917080 (+1132943us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400232b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:49.917220 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528926592}
I20260709 13:03:49.920078  7902 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:49.920698  7902 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:49.921713  7902 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.923002  7902 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:49.874612  7906 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:49.876631  7509 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:49.930840  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.822355  7765 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:49.853581  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 70633.4us
I20260709 13:03:49.933269  1539 leader_election.cc:400] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:49.933754  1539 leader_election.cc:403] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:03:49.871618  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.627s (SENT)
I20260709 13:03:49.800587  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:49.873546  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 206672us
W20260709 13:03:49.937111  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65487.3us
W20260709 13:03:49.937899  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.665s (SENT)
I20260709 13:03:49.876384  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:49.875769  2380 rpcz_store.cc:276] 0709 13:03:48.776149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.776329 (+   180us) service_pool.cc:224] Handling call
0709 13:03:48.869144 (+ 92815us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:48.895960 (+ 26816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.895993 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:48.896865 (+   872us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.874965 (+978100us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:49.875010 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.875087 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.875139 (+    52us) raft_consensus.cc:1685] finished
0709 13:03:49.875153 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.875307 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17189,"raft.queue_time_us":29303,"raft.run_wall_time_us":51145,"spinlock_wait_cycles":107401600,"thread_start_us":850229,"threads_started":1,"wal-append.queue_time_us":915254}
W20260709 13:03:49.938773  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.694s (SENT)
I20260709 13:03:49.845651  7800 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:49.850976  7936 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:49.887754  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 866) took 1152 ms. Trace:
I20260709 13:03:49.876832  7932 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:49.949545  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:49.950601  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 880) took 1167 ms. Trace:
I20260709 13:03:49.948868  2374 rpcz_store.cc:276] 0709 13:03:48.735625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.748600 (+ 12975us) service_pool.cc:224] Handling call
0709 13:03:48.749234 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.749735 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.749776 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.751152 (+  1376us) log.cc:844] Serialized 501 byte log entry
0709 13:03:49.469166 (+718014us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:49.469247 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.469352 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.887323 (+417971us) raft_consensus.cc:1685] finished
0709 13:03:49.887364 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.887645 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":717543,"threads_started":1,"wal-append.queue_time_us":766667}
W20260709 13:03:49.952696  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 76369.4us
I20260709 13:03:49.851315  7703 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.892936  7693 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:49.944288  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:49.948179  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:49.967464  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 167007us
I20260709 13:03:49.890533  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2307) took 2641 ms. Trace:
I20260709 13:03:49.954110  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:49.968796  1573 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:49.954934  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:49.891283  7908 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:49.956260  2380 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:49.951138  1725 rpcz_store.cc:276] 0709 13:03:48.783595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.786650 (+  3055us) service_pool.cc:224] Handling call
0709 13:03:48.786976 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.787220 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:49.256182 (+468962us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.942590 (+686408us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b18002cc560. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a
0709 13:03:49.950181 (+  7591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.950240 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:49.950295 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.950383 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.950531 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7313,"prepare.queue_time_us":818396,"prepare.run_cpu_time_us":464,"prepare.run_wall_time_us":7683,"spinlock_wait_cycles":5526784,"thread_start_us":737824,"threads_started":2,"wal-append.queue_time_us":508030}
I20260709 13:03:49.907838  7746 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:49.966609  2371 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:49.971931  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:49.970820  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:49.977058  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:49.977869  2374 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:49.983116  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.984782  7946 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:03:50.004223  2374 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:49.947067  7831 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.968127  1643 rpcz_store.cc:276] 0709 13:03:47.248689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.559171 (+310482us) service_pool.cc:224] Handling call
0709 13:03:47.983310 (+424139us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:47.986313 (+  3003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.986530 (+   217us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:48.132032 (+145502us) log.cc:844] Serialized 52 byte log entry
0709 13:03:48.789015 (+656983us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:49.310586 (+521571us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.311904 (+  1318us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.312399 (+   495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.312561 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:49.890182 (+577621us) raft_consensus.cc:1685] finished
0709 13:03:49.890208 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.890407 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1130568,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":4358,"raft.queue_time_us":16450,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":341,"thread_start_us":989192,"threads_started":2,"wal-append.queue_time_us":1263860}
I20260709 13:03:50.011204  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:50.013067  7911 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:50.013651  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:50.018257  7943 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:50.028739  2380 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:49.945303  7412 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:50.026904  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84354.4us
I20260709 13:03:49.968768  7690 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:49.950443  7901 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:03:49.968475  7871 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.966907  7907 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:50.048542  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:50.010160  7908 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.000517  7866 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.056763  7908 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.012243  7939 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:49.961687  7800 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:03:50.062525  7907 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:49.993626  7909 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:50.010525  7932 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:50.066315  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:50.079288  1571 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:50.001186  7916 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:50.081043  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:50.076838  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:50.084707  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51001.4us
I20260709 13:03:50.088573  1570 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:50.092723  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:50.042695  7941 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.075714  7943 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:50.096884  7916 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:50.072461  7871 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.101824  2374 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.101187  7916 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.103348  7871 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:50.109407  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:50.085158  7932 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:50.117744  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 140769us
I20260709 13:03:50.120195  1573 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:50.121429  1643 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:50.124441  2374 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:50.096560  7621 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:50.047412  7693 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:03:50.048813  7412 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.032886  7831 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.084766  7450 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:03:50.199160  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 112745us
I20260709 13:03:50.107282  7936 raft_consensus.cc:697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:50.115497  7871 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.116825  7800 consensus_queue.cc:1243] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:50.040309  7911 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.044744  7746 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.201773  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:50.133734  7932 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.226370  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:50.061599  1466 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:50.192848  7621 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:50.227813  2371 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:03:50.236467  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 120133us
I20260709 13:03:50.231014  1571 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:50.105017  7952 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:03:50.129946  7901 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:03:50.104588  7765 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.099874  7943 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:50.252626  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 87461.9us
I20260709 13:03:50.204550  7948 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:03:50.257296  1688 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:50.240774  7621 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.236634  7908 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:50.243029  7870 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:50.256654  7871 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:50.256438  7742 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.300810  7935 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.307s	user 0.002s	sys 0.000s
I20260709 13:03:50.300074  7908 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:50.313984  7598 consensus_meta.cc:287] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.003s
I20260709 13:03:50.311636  7800 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:50.281723  7942 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:03:50.315166  7914 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:50.309980  7703 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:50.338162  7800 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:50.344760  7800 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.346889  7800 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:50.349819  7954 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:03:50.352674  7914 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:50.352943  7412 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:50.354001  7914 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.300791  7953 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:03:50.362540  7839 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:50.376876  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 119596us
I20260709 13:03:50.381055  7957 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.381908  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:50.382860  7643 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:50.384944  7643 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:50.386546  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:50.388762  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.405287  1688 connection.cc:256] RPC call timeout handler was delayed by -1.6876572600000372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
W20260709 13:03:50.409147  1688 connection.cc:256] RPC call timeout handler was delayed by -1.2133572600000662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
W20260709 13:03:50.409596  1688 connection.cc:256] RPC call timeout handler was delayed by -1.0023171500000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:03:50.332589  7952 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:50.408206  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 214879us
I20260709 13:03:50.410929  7952 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.412145  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:50.413689  7779 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.414749  7952 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:50.310294  7742 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.311926  7871 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:50.420444  7899 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.001s
I20260709 13:03:50.275695  7936 consensus_queue.cc:237] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.371959  7882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36598 (request call id 4) took 7785 ms. Trace:
I20260709 13:03:50.429704  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:50.435273  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:50.452194  2373 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:50.431540  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:50.425040  7839 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:50.324226  7959 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 13:03:50.426010  7743 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.368834  7598 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:50.513381  7927 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.823s	user 0.000s	sys 0.008s
I20260709 13:03:50.514500  7961 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:50.513367  7643 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:50.515261  7643 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.516510  7972 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:50.468549  7866 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:50.469462  7799 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.003s
I20260709 13:03:50.473582  7911 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:50.486112  7742 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:50.550706  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 120595us
I20260709 13:03:50.429333  7971 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:03:50.515198  7961 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.469389  7871 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:50.519506  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:50.337204  1466 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 12, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:03:50.522315  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.464359  7825 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.793s	user 0.016s	sys 0.000s
I20260709 13:03:50.443133  7831 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:50.493050  7839 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:50.518551  7509 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.005s
I20260709 13:03:50.556326  1725 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:49.550868  1716 rpcz_store.cc:276] 0709 13:03:47.409047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.409600 (+   553us) service_pool.cc:224] Handling call
0709 13:03:47.443037 (+ 33437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.443333 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:48.161470 (+718137us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.519168 (+1357698us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.519456 (+   288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:49.519520 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:49.519595 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.519711 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.519850 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124817,"prepare.queue_time_us":675032,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":6301,"raft.queue_time_us":11093,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":58092,"thread_start_us":1950403,"threads_started":3,"wal-append.queue_time_us":1064300}
I20260709 13:03:50.558306  1717 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:50.559228  1721 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:50.597021  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.597759  1721 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:50.598994  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:50.599146  7866 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:50.559898  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.600270  7866 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.560974  1720 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:50.561683  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.605047  1722 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:50.562482  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:50.564203  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.012s	sys 0.032s
I20260709 13:03:50.577804  1616 leader_election.cc:400] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
I20260709 13:03:50.454516  7899 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.609753  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:50.607928  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:50.610921  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.013s	sys 0.032s
I20260709 13:03:50.592420  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.584852  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:50.611127  2375 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:50.570498  7693 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:50.555002  7953 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.614087  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.615108  1718 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:03:50.614832  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 388344us
I20260709 13:03:50.615738  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.616663  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:50.555474  7765 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.617609  6923 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.614933  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.520646  7743 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.551811  7799 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.585758  7825 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.575155  7943 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.449064  7882 rpcz_store.cc:276] 0709 13:03:42.552622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.558707 (+  6085us) service_pool.cc:224] Handling call
0709 13:03:43.171867 (+613160us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b54002f1aa0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ceb520 00007fc833d74b42 00007fc833d73800 00007fc833d6c9fa 00007fc838d344c9 00007fc8322b0c54 00007fc8322b0be2 00007fc8322b0b11 00007fc8322b0a8c 00007fc8322afd42
0709 13:03:43.570123 (+398256us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b18000577e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833da973d 00007fc833d74f3f 00007fc833d746b3 00007fc833d6c9fa 00007fc838d344c9 00007fc8322b0c54 00007fc8322b0be2 00007fc8322b0b11 00007fc8322b0a8c 00007fc8322afd42
0709 13:03:43.683901 (+113778us) log.cc:844] Serialized 501 byte log entry
0709 13:03:44.655101 (+971200us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b54002f1aa0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce7142 00007fc833cc7eaa 00007fc833cced53 00007fc833ccecd9 00007fc833ccec69 00007fc833ccec31 00007fc833ccddfd 000000000038f354 000000000038f199 00007fc830c81475
0709 13:03:44.892004 (+236903us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b180013a9e0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833cc7b0b 00007fc833d4cb63 00007fc833d74752
0709 13:03:50.327045 (+5435041us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":177198,"raft.run_cpu_time_us":4525,"raft.run_wall_time_us":554785,"spinlock_wait_cycles":274054912,"thread_start_us":591245,"threads_started":2,"wal-append.queue_time_us":590595,"wal-append.run_cpu_time_us":5009,"wal-append.run_wall_time_us":1270069}
I20260709 13:03:50.635023  6923 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:50.622527  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:50.623066  1725 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:50.726090  1576 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:50.582916  7969 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.705529  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88958.8us
I20260709 13:03:50.621526  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:50.770262  1576 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:50.773748  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:50.689513  7902 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.766s	user 0.018s	sys 0.002s
W20260709 13:03:50.784372  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 162837us
W20260709 13:03:50.733760  7929 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.955s	user 0.000s	sys 0.007s
W20260709 13:03:50.689611  7950 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.662s	user 0.000s	sys 0.006s
W20260709 13:03:50.681161  7919 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.933s	user 0.006s	sys 0.000s
I20260709 13:03:50.788072  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.792521  1716 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:50.660571  7765 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.787675  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:50.679996  7870 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:50.668399  7693 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:50.679436  7961 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:50.800546  1570 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:50.810487  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.687668  7598 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:50.688421  7901 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:50.681037  7812 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:50.694115  7939 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:50.791201  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:50.679664  7643 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:50.788908  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:50.609383  7971 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.177s
I20260709 13:03:50.609787  7509 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.806281  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 62836cb7c4a548b5a6c44c89782a958c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 5)
I20260709 13:03:50.807723  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:50.783564  7959 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.689527  7742 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:50.834993  1576 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.683015  7932 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.785408  7943 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:50.660771  7953 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.971541  1721 raft_consensus.cc:606] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:50.737048  7839 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:50.962502  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:50.837599  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46507.7us
I20260709 13:03:50.918238  1724 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:50.905489  1458 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 99408.2us
I20260709 13:03:50.969291  7850 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:51.003876  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:50.804435  7870 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:51.010023  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 202315us
I20260709 13:03:51.018750  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:50.786394  7977 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:51.024688  1576 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:50.975678  7779 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.000s
I20260709 13:03:50.810283  7879 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:50.916752  7923 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:03:51.032142  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3052) took 2133 ms. Trace:
I20260709 13:03:50.836719  7901 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:50.793710  7969 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:50.837908  7978 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.305s
I20260709 13:03:50.837078  7939 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.104235  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3057) took 2206 ms. Trace:
I20260709 13:03:51.155138  6923 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 13:03:51.151014  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.037003  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.102995  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:51.101065  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 97318.3us
I20260709 13:03:51.044257  1719 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:51.034700  1723 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
W20260709 13:03:51.169166  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.170532  1723 pending_rounds.cc:85] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 13
I20260709 13:03:51.171097  1723 pending_rounds.cc:107] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:03:51.170629  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.163988  1574 rpcz_store.cc:276] 0709 13:03:48.886535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.889517 (+  2982us) service_pool.cc:224] Handling call
0709 13:03:48.889881 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.987961 (+ 98080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.988001 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.988690 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.969763 (+981073us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x7b180000e860. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:03:50.026889 (+ 57126us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:50.026964 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.027115 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.686617 (+659502us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:51.019446 (+332829us) raft_consensus.cc:1685] finished
0709 13:03:51.019471 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.019637 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23244,"raft.queue_time_us":110391,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":43831,"spinlock_wait_cycles":23587456,"thread_start_us":56820,"threads_started":1,"wal-append.queue_time_us":985684}
I20260709 13:03:51.171423  1723 raft_consensus.cc:2889] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.173501  1718 raft_consensus.cc:2410] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:50.870749  7948 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.999148  7941 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.977625  7942 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.165108  1575 rpcz_store.cc:276] 0709 13:03:48.897605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.898670 (+  1065us) service_pool.cc:224] Handling call
0709 13:03:48.899130 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:49.044886 (+145756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.044919 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.045715 (+   796us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.520736 (+475021us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:49.520803 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.520905 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.732771 (+1211866us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:51.087020 (+354249us) raft_consensus.cc:1685] finished
0709 13:03:51.087059 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.104132 (+ 17073us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145104,"raft.queue_time_us":147014,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":125416,"spinlock_wait_cycles":5190400,"thread_start_us":474385,"threads_started":1,"wal-append.queue_time_us":628808}
I20260709 13:03:51.179536  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:50.976681  7742 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:51.009763  7850 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:51.032387  7643 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:51.046386  7882 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:51.180955  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64176.4us
I20260709 13:03:51.227644  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:51.099331  7982 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:03:51.228399  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 877) took 1205 ms. Trace:
I20260709 13:03:51.229369  2374 rpcz_store.cc:276] 0709 13:03:50.012572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.012750 (+   178us) service_pool.cc:224] Handling call
0709 13:03:51.217258 (+1204508us) spinlock_profiling.cc:243] Waited 72.9 ms on lock 0x7b540064f354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:51.217431 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167558400}
I20260709 13:03:51.027741  7977 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.032083  7812 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:51.052106  7779 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:51.221202  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.192541  1576 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 13:03:51.237536  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 448476us
I20260709 13:03:51.223899  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.164315  7879 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.102284  7899 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.102005  7953 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.180073  7971 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:03:51.275303  1717 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:51.238770  1576 pending_rounds.cc:85] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 9
I20260709 13:03:51.276065  1576 pending_rounds.cc:107] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:51.276495  1576 raft_consensus.cc:2889] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:51.098984  7968 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 13:03:51.087966  7961 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.145s
I20260709 13:03:51.273167  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.015450  7959 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.220188  7643 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:51.208071  7911 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:03:51.297303  7765 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.241747  7799 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:51.241506  7509 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.299006  7812 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.234560  7882 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:51.272465  7879 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.281436  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2702) took 1328 ms. Trace:
I20260709 13:03:51.232901  7901 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.007383  1466 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 11, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.279446  7742 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.284288  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2697) took 1345 ms. Trace:
I20260709 13:03:51.316778  1569 rpcz_store.cc:276] 0709 13:03:49.939051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.939332 (+   281us) service_pool.cc:224] Handling call
0709 13:03:49.939752 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.940135 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.940175 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.941098 (+   923us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.612604 (+671506us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:50.612672 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.612782 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.283970 (+671188us) raft_consensus.cc:1685] finished
0709 13:03:51.283998 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.284206 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":600647,"threads_started":1,"wal-append.queue_time_us":701006}
W20260709 13:03:51.287972  7946 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:03:51.291859  7866 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.239957  7977 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.292606  7943 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:51.295194  7621 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.010s	user 0.008s	sys 0.006s
W20260709 13:03:51.296159  7800 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.949s	user 0.011s	sys 0.007s
W20260709 13:03:51.331808  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 161217us
I20260709 13:03:51.296432  7412 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:51.333200  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.300101  1576 rpcz_store.cc:276] 0709 13:03:49.952737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.954666 (+  1929us) service_pool.cc:224] Handling call
0709 13:03:49.954991 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.725917 (+770926us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400279aa0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce5dc0 00007fc833d701c0 00007fc833d71231 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c
0709 13:03:51.281068 (+555151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.281347 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591842176}
I20260709 13:03:51.111822  7693 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.300689  7643 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:51.280802  7743 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.207513  7870 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.207823  7941 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.299750  7882 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.334506  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.335358  7901 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.349076  7988 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:51.349956  7941 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.350517  7941 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:51.350567  7799 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.351076  7941 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.326402  7951 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:51.352018  7951 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:51.352639  7932 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.352864  7951 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.339355  7742 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.353953  7951 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.336549  7621 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.357862  7621 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.358080  2371 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:51.338737  7943 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:03:51.358808  7870 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.362071  7866 raft_consensus.cc:993] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:51.362629  7866 raft_consensus.cc:1081] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
W20260709 13:03:51.362803  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.320832  7831 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.351193  7799 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.363997  7831 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.333591  7412 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.338755  7946 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:51.388154  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53714.5us
I20260709 13:03:51.392266  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:51.394294  7799 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.394871  7746 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.398319  1466 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 3 to 11. New cstate: current_term: 11 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.377565  7969 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.390146  7812 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.381311  7412 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.410295  7831 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:51.413717  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 186019us
W20260709 13:03:51.413066  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 190242us
I20260709 13:03:51.424175  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:51.429044  1466 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 14 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:51.433907  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.442850  1466 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 10, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.450073  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:51.439069  7990 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:03:51.435818  7800 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:51.455163  8000 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.455499  7941 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:51.455786  8000 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.446182  7870 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.456398  7901 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.458065  8000 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.449617  7988 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:51.458906  7988 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.460572  7800 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:51.461012  7800 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.461253  7988 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.468174  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:51.426614  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:51.475240  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:51.481587  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57712.6us
W20260709 13:03:51.483191  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 56658.8us
I20260709 13:03:51.502409  7957 raft_consensus.cc:2764] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:51.504129  7959 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:51.503618  1466 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.505985  7825 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.505590  1466 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:51.507055  7977 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.509333  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:51.491801  7744 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:51.491143  8002 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:51.517701  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.486796  7870 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:51.536793  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:51.535599  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.553367  1539 leader_election.cc:400] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17.
I20260709 13:03:51.556389  1539 leader_election.cc:403] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:51.556766  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.563287  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:51.566769  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 84049us
I20260709 13:03:51.568571  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:51.520648  7957 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:51.529232  7744 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.588601  7744 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.577173  7943 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:03:51.596834  7943 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:51.597234  7943 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:51.598054  7943 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.599221  7943 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.581982  7812 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:51.507711  1466 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 6 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.601665  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.578835  7902 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:51.602406  1719 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:51.602414  7902 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:51.603039  7902 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.596688  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.530203  8002 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:51.619729  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134259us
I20260709 13:03:51.621773  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:51.559058  7946 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:51.628096  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:51.575675  7746 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:03:51.625291  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:51.630856  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:51.631210  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:51.638154  1570 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:51.638558  1572 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:51.573305  7831 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.630060  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:51.639979  1575 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:51.639838  7831 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.630503  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:51.641295  1574 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:51.642180  7831 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.632151  1569 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:51.631171  7899 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:03:51.642923  7879 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.623513  8002 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.638978  7643 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.622960  7971 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:03:51.626513  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96553.3us
I20260709 13:03:51.645737  1567 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.646353  7971 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:51.646808  7971 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:51.628685  7939 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.647025  7643 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:51.637120  7992 log.cc:927] Time spent T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:03:51.602623  1466 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.649004  1719 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:51.636441  7984 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.575531  7998 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:51.652240  7998 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:51.651566  7800 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.652681  7998 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.573637  7999 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:51.654583  7621 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.655076  7621 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.655499  1567 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:51.657332  7621 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.658519  7953 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.659032  7953 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.661648  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.662698  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46374 (request call id 37) took 1045 ms. Trace:
I20260709 13:03:51.663348  1721 rpcz_store.cc:276] 0709 13:03:50.589270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.598758 (+  9488us) service_pool.cc:224] Handling call
0709 13:03:51.615276 (+1016518us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400087854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d68853 00007fc838d3660a 00007fc8322bc7d4 00007fc8322bc762 00007fc8322bc691 00007fc8322bc60c 00007fc8322bb8c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:51.615419 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854697216}
I20260709 13:03:51.665532  1466 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 10, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.666965  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3035) took 1099 ms. Trace:
I20260709 13:03:51.667351  1466 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 11 to 12, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.668372  7850 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.669253  7744 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.669819  7850 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:51.667491  1723 rpcz_store.cc:276] 0709 13:03:50.568108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.601567 (+ 33459us) service_pool.cc:224] Handling call
0709 13:03:50.601842 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.195207 (+593365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.666627 (+471420us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54001a6dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:51.666876 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993218816}
I20260709 13:03:51.672837  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.673090  7744 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.675638  7885 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:51.673717  7831 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:51.674515  7909 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:51.673112  7953 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:51.676915  7909 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.676702  7971 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.677361  7909 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.678421  7971 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.679358  7882 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:51.629832  7996 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.175s	user 0.000s	sys 0.004s
W20260709 13:03:51.629732  8001 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.128s	user 0.002s	sys 0.002s
W20260709 13:03:51.691390  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.692818  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.683712  7412 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.697844  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.699080  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:51.693080  2380 raft_consensus.cc:2955] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 5 to 12, VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:03:51.702298  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:51.703179  2375 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:51.534662  7936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.003s	sys 0.022s
I20260709 13:03:51.706902  1567 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:51.712046  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.706743  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:51.707650  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.706851  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:51.708693  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.714010  1718 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:51.709275  1724 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:51.715131  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:51.716657  2379 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:51.717312  2373 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:51.717415  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:51.717907  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:51.718001  1718 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.718663  1722 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:51.721340  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 168083us
I20260709 13:03:51.713753  2372 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:51.724988  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:51.725878  2377 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.726486  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:51.724980  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:51.728082  1717 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.728880  2375 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:51.676733  7831 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:51.686728  7901 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:51.734849  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:51.735812  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.736565  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:51.741307  1643 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:51.742794  1718 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:51.743945  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:51.748731  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.730093  7984 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:51.749387  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:51.686569  7909 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.750169  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.732569  7743 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.734382  7831 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:51.725921  7870 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:51.733028  7977 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.756580  2377 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:03:51.730319  7951 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17.
I20260709 13:03:51.736764  8003 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.736629  7965 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.770705  1717 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:51.737720  7643 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:51.778702  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:51.772400  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.742920  7939 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:51.730147  7839 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:03:51.753296  7899 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:03:51.753623  7953 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:03:51.787216  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:51.772110  7743 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.793632  7412 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.794127  7743 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.794832  1688 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:51.790580  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.784874  2373 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:03:51.776870  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.802312  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:51.803002  1647 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.774843  7879 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.765650  7984 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:51.805336  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:51.802537  2373 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 16. Ops: []
I20260709 13:03:51.777796  7800 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:51.784729  8005 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:03:51.818488  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:51.833081  1724 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.791917  7909 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.784143  7690 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:51.785512  7882 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:51.794989  7942 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:03:51.824760  7936 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:51.781240  7643 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.849732  8009 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:51.781303  8003 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.843729  7690 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.781033  7965 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.853487  7936 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.857939  8009 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:51.859248  7984 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.869122  8009 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.854900  8002 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.874264  8002 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:51.669539  1466 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 10, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:51.758963  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:51.914283  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:51.915102  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 156009us
W20260709 13:03:51.917011  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.917867  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.800918  1643 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:03:51.930179  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.930251  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.917532  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:51.935032  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:51.937603  1650 raft_consensus.cc:1240] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:03:51.941347  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:51.944468  1642 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:51.860543  7902 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:51.948429  1644 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:51.950848  2373 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:51.951100  7902 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:51.960879  8023 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:03:51.872130  7998 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.861213  7936 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.969928  7998 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:51.987335  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.003979  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:52.005266  7939 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.007614  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:51.841302  7831 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:51.846866  7870 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.002990  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:52.010145  7939 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.015568  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:52.015976  1717 raft_consensus.cc:1240] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:52.016739  7939 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:52.017207  1719 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:52.018306  1643 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:52.011879  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:52.012096  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 94773.1us
I20260709 13:03:52.018668  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:52.018993  6923 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:52.018568  7643 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:52.048904  1650 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:03:52.060103  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.052991  1642 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:52.057380  8009 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:52.061625  1643 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:52.066649  7977 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:52.069835  7951 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:52.071678  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:52.071964  7831 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.019356  7800 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:52.068068  7899 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:52.078450  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.082919  7965 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:52.084988  7951 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:52.085714  7951 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:52.093114  1644 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:03:52.093636  1644 pending_rounds.cc:85] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 12
I20260709 13:03:52.093920  1644 pending_rounds.cc:107] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:03:52.094223  1644 raft_consensus.cc:2889] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.095559  1647 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
W20260709 13:03:52.096660  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.097456  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:52.080971  7977 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:52.081063  7800 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:52.111232  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.040s (SENT)
I20260709 13:03:52.111943  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:52.112144  1643 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:52.110917  7977 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.087520  7951 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.115695  7800 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.133431  7951 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:52.134081  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 62615.1us
W20260709 13:03:52.163023  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.127610  7939 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:52.226756  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.229242  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.229853  1574 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:52.230118  1567 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:52.173877  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.181732  8003 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:52.183056  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.192287  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 179420us
I20260709 13:03:52.193215  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.231998  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.194084  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.195755  1650 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:03:52.239989  8012 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.397s	user 0.000s	sys 0.005s
I20260709 13:03:52.196897  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.240968  7914 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:52.200802  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.241737  7914 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.144433  7879 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:52.202682  1721 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:52.208461  1644 raft_consensus.cc:606] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:52.243417  1644 raft_consensus.cc:614] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:52.244840  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.208655  6923 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:52.247483  6923 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.250301  7914 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:52.209396  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 156727us
I20260709 13:03:52.253927  7866 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:52.254281  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:52.254954  7850 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.009s
I20260709 13:03:52.211058  7899 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:52.214859  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.215994  7946 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.216751  7870 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.132445  7978 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.012s	sys 0.011s
I20260709 13:03:52.225703  1724 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:52.193847  7621 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:52.242704  7879 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:52.222909  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.255630  1648 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:52.276144  7946 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.282137  7946 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:52.284677  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:52.285246  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:52.285538  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:52.285987  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38bef56447d743db91bd1f59274db995"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:03:52.286630  1645 raft_consensus.cc:606] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:52.287145  1645 raft_consensus.cc:614] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:52.289147  2344 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.290268  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.290586  7899 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.291215  8002 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.291927  7643 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.292501  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.293144  1567 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.291178  1573 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:52.294852  7800 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.295305  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.294116  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:03:52.274806  7978 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.012s	sys 0.012s
I20260709 13:03:52.300400  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.300872  1570 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:52.301316  1569 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:52.287293  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:52.302390  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.311230  1645 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:52.311543  2372 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:52.311611  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:52.312505  1717 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.312775  1645 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:52.311281  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 153805us
I20260709 13:03:52.318174  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.325973  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.321565  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:52.333263  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:52.339007  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 53587.3us
I20260709 13:03:52.291945  8002 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.349428  1574 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:52.358803  7948 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:52.359344  7948 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:52.359863  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.360615  1647 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:52.360821  7879 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.362071  7879 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:52.362378  1642 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.365141  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.349525  1572 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:52.366626  1573 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:52.368664  1651 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:52.326288  7870 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.339116  7939 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:52.370651  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:52.348510  7978 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:03:52.371115  1614 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:52.379575  1649 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:52.371842  7800 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:52.381096  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.378062  1576 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:52.387040  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.386304  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:52.389916  7850 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:52.390605  7850 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:52.391287  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.392870  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.397116  8033 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:52.397792  8033 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:52.398280  8033 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.400472  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:52.401197  1576 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:52.401708  1717 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:52.391534  7850 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.409214  7690 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.410669  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:52.411401  7870 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:52.412019  1568 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:52.412364  8009 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.412489  7690 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:52.412938  1574 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:52.413239  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.413942  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.414755  1574 raft_consensus.cc:2410] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:52.414516  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.415478  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:52.415614  1574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:03:52.417296  1576 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.383402  7977 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.419173  1576 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:52.414496  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:52.417520  8032 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:03:52.422911  6923 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.404739  7914 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.419765  7977 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:52.406339  7939 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:52.433457  8038 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.161s	user 0.003s	sys 0.000s
I20260709 13:03:52.434481  7825 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:52.438023  1568 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:52.435094  7939 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:52.436682  1643 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:52.445513  8002 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:52.445686  7936 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.447250  6923 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:52.461390  1572 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:52.449007  7936 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:52.478598  1567 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:52.435217  7825 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.486480  1717 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:03:52.486888  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:52.494585  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.491734  1575 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:03:52.505002  1575 pending_rounds.cc:85] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 10
I20260709 13:03:52.506579  1575 pending_rounds.cc:107] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:03:52.506867  1575 raft_consensus.cc:2889] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.482754  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.495031  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.490674  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:52.495106  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:52.511957  1718 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:03:52.518883  8035 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:52.494318  7902 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:03:52.498185  7948 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.511650  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:52.483290  8034 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:03:52.480300  7971 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.782s	user 0.005s	sys 0.016s
I20260709 13:03:52.527768  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.484297  7825 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:52.528687  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:03:52.530383  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110226us
I20260709 13:03:52.538121  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:52.531404  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:52.551568  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:52.559123  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:52.560509  7984 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:52.565958  7825 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:52.567675  7825 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:52.551841  1646 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:52.566037  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:52.580518  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 69639.5us
I20260709 13:03:52.549062  7936 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:52.585373  1574 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:52.585968  1574 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:52.536309  7643 consensus_queue.cc:1059] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:03:52.587265  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.568261  7825 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:52.547720  7914 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:52.577371  7948 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.593690  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:52.603055  1720 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:52.604537  1724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:52.607069  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.605371  1724 raft_consensus.cc:606] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:52.628908  7866 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:52.629608  7866 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.630189  7866 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.632871  1718 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:03:52.629777  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.634424  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 143822us
I20260709 13:03:52.623948  1569 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 13:03:52.637408  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 78355.5us
I20260709 13:03:52.592243  7936 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:52.623775  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:52.603693  7984 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:52.644445  1574 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.664934  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.634327  7690 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:52.633338  7825 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.676532  7839 raft_consensus.cc:3055] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:52.677106  7839 raft_consensus.cc:740] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:52.680953  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:52.682041  7914 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:52.682595  7914 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:52.682971  7914 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.669889  7690 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.684710  7690 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.689080  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:52.690198  2372 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:52.692366  7839 consensus_queue.cc:260] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.693722  1569 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:52.697221  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 76743us
I20260709 13:03:52.695252  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:52.646090  7831 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:52.677126  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:52.699985  1570 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.700289  1567 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:03:52.713186  1567 pending_rounds.cc:85] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:03:52.713557  1567 pending_rounds.cc:107] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:03:52.713873  1567 raft_consensus.cc:2889] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:52.642712  7936 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.700788  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.716092  1568 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:52.760030  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57166.6us
I20260709 13:03:52.660799  7977 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:52.783160  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.783901  1720 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.790894  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.793896  1643 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.796959  1649 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:52.800084  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:52.803838  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.806355  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 136129us
I20260709 13:03:52.811535  1570 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:03:52.782682  7831 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:52.816004  1646 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 11: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:52.820205  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.822870  1575 raft_consensus.cc:1275] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:03:52.823342  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.824860  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:52.826913  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.836611  1568 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:52.845587  1574 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:52.788703  7879 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:03:52.859061  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 163811us
I20260709 13:03:52.869277  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.870579  7988 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.870898  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.871606  8052 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:52.864979  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.880007  1575 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.867992  1570 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:52.872092  8052 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:52.882515  8052 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.882766  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:52.884063  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.883893  8036 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 13:03:52.885787  8036 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:52.886142  8036 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:52.887065  8036 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.888331  8036 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:52.888751  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.882864  1724 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:52.893884  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:52.872107  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.850270  7825 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.839149  8009 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.815932  8035 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:03:52.863234  7839 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.873178  7850 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:52.847229  7977 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:52.924417  1574 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.934409  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:52.935081  1643 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:03:52.937261  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 157625us
I20260709 13:03:52.919571  7850 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.955354  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.952327  7984 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:52.961758  8046 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.302s	user 0.003s	sys 0.000s
I20260709 13:03:52.962385  7984 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:52.964498  1721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:03:52.968422  1571 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:52.969121  1722 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:52.969225  7831 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:52.973068  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44319.9us
W20260709 13:03:52.941473  8033 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.005s
I20260709 13:03:52.977026  1723 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:03:52.978654  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 379282us
I20260709 13:03:52.983345  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:52.987846  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.978173  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:52.959348  7850 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:52.983520  8050 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:03:52.994465  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.996621  1716 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:52.984988  8021 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:03:52.998867  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:53.000052  1568 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.933254  7988 consensus_queue.cc:1059] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:03:52.988766  7450 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:03:53.006129  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 328958us
I20260709 13:03:53.014096  1723 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:03:53.006923  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:53.007644  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:52.977651  8033 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.025142  1569 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:53.029986  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.030896  1569 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:53.032120  1725 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:53.047324  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53583.9us
I20260709 13:03:53.009503  7450 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:53.049899  1716 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:03:53.007228  7936 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.065300  1576 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:53.033833  7988 raft_consensus.cc:3055] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:53.082962  1575 raft_consensus.cc:2410] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:53.083853  7693 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 13:03:53.088809  8066 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:53.089149  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:53.092818  8020 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:03:53.117537  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.022269  7899 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:03:53.127635  1720 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.101138  7977 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:53.101562  8063 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:53.102094  8003 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:53.111351  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:53.114590  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 97780.8us
W20260709 13:03:53.111310  8021 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:53.020720  7866 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.079464  7951 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.134857  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.135403  1569 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:53.139736  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:53.140618  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 79899.8us
I20260709 13:03:53.137928  8063 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:53.142184  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.107267  7988 raft_consensus.cc:740] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:53.158993  7866 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:53.146960  7977 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:53.159919  7977 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.162055  7977 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.162586  8063 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:53.164440  7951 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.166707  7951 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.167260  8009 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:53.142411  7450 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.168162  7450 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.168246  1568 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 12.
I20260709 13:03:53.169382  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:53.170686  1568 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:53.171756  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.171780  1568 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.173372  8033 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.175225  1575 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 11.
I20260709 13:03:53.100193  7690 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.177714  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:53.108894  7914 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.195338  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:53.196220  1643 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:53.201105  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 194206us
I20260709 13:03:53.204277  1721 raft_consensus.cc:606] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:53.204813  1721 raft_consensus.cc:614] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:53.204434  1717 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:53.210685  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99398.5us
W20260709 13:03:53.212015  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.212553  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:53.213251  1643 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.214809  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:53.214489  7988 consensus_queue.cc:260] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.216002  7988 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.147760  8066 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 13:03:53.204931  7948 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.006s
I20260709 13:03:53.172060  7850 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.225818  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:53.225800  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:53.227705  1573 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:03:53.229641  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:53.232789  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:53.267479  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.287348  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:53.292793  2378 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:53.232402  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3195487429999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 13:03:53.293083  8069 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.293931  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1197487430000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:03:53.294413  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9919487429999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 13:03:53.295084  1539 connection.cc:256] RPC call timeout handler was delayed by -1.6958487429999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
I20260709 13:03:53.294286  8069 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.295801  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:03:53.296705  8073 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:03:53.297298  7914 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:53.236209  8083 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:53.299329  8056 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:03:53.226965  7965 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.074s
W20260709 13:03:53.303028  8070 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:53.305850  7850 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.308543  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:53.310015  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:53.234655  7948 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.332002  2375 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:53.308688  7690 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:53.334162  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:53.233809  8071 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.180s
I20260709 13:03:53.318013  8083 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:53.335929  2378 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:53.334993  1649 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:53.322660  7914 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:53.327193  8033 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.330502  7951 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.331125  7850 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:53.425437  7850 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:53.426028  7850 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.354378  1646 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:03:53.431295  8074 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:03:53.355840  8059 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.381s	user 0.004s	sys 0.000s
I20260709 13:03:53.361032  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:53.362655  8083 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:53.366462  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.434263  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.368587  8033 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.372784  7825 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:53.374596  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:53.332999  7948 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.337721  7936 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:53.388568  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 92793.7us
I20260709 13:03:53.399139  1575 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:53.400152  7902 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.406605  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 177287us
W20260709 13:03:53.408267  8077 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:03:53.409922  8069 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:53.412567  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:53.349030  7866 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
W20260709 13:03:53.449930  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.458943  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 226159us
W20260709 13:03:53.404117  8060 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.371s	user 0.005s	sys 0.007s
I20260709 13:03:53.462656  1643 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:53.453233  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3092) took 1051 ms. Trace:
I20260709 13:03:53.455363  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2738) took 1087 ms. Trace:
I20260709 13:03:53.477922  1572 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:03:53.478688  1572 pending_rounds.cc:85] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 10
I20260709 13:03:53.479041  1572 pending_rounds.cc:107] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:03:53.479368  1572 raft_consensus.cc:2889] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:53.480998  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44746 (request call id 3091) took 1101 ms. Trace:
I20260709 13:03:53.459954  7902 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.491575  7902 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.492877  7882 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:53.492151  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:53.454017  7899 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:53.481604  1573 rpcz_store.cc:276] 0709 13:03:52.368221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.374792 (+  6571us) service_pool.cc:224] Handling call
0709 13:03:53.466832 (+1092040us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:53.469180 (+  2348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1918898048}
I20260709 13:03:53.474901  1571 rpcz_store.cc:276] 0709 13:03:52.361056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.583890 (+222834us) service_pool.cc:224] Handling call
0709 13:03:52.618713 (+ 34823us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b5800003398. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49eee 0000000000375939 00007fc838cbd1e0 00007fc838d860c4 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:03:52.820685 (+201972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.968898 (+148213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.448060 (+479162us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54000c2450. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:53.448334 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483255424}
I20260709 13:03:53.463300  7825 consensus_queue.cc:1243] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer 4f737473248e4c059fa74af3b8e1f77e log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:53.481658  8066 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:53.514251  2377 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.460883  7984 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 13:03:53.519655  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.519383  7899 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.472996  1570 rpcz_store.cc:276] 0709 13:03:52.387484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.895094 (+507610us) service_pool.cc:224] Handling call
0709 13:03:52.895416 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.895889 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.895927 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.896510 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.961175 (+ 64665us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:52.961215 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.961282 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.435220 (+473938us) raft_consensus.cc:1685] finished
0709 13:03:53.435248 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.435443 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"thread_start_us":64213,"threads_started":1,"wal-append.queue_time_us":392919}
I20260709 13:03:53.500811  8069 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.531030  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 156539us
I20260709 13:03:53.531632  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:53.533005  1722 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.517336  7977 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.540617  1721 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:53.546602  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:53.550494  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:53.529321  7831 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.516571  7902 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:53.550433  7951 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:53.563869  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.565833  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.567682  1568 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:53.537708  8066 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:53.539961  7984 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:53.579397  1717 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:03:53.580406  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:53.584924  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:53.593122  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.607378  1575 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:03:53.521727  8063 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:53.612497  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47128.2us
W20260709 13:03:53.625424  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.552616  7977 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.612005  7450 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:53.631270  1651 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:53.562973  7831 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.637739  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:53.548769  8069 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.646497  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:53.648972  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.661814  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.566453  8009 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:03:53.663565  1724 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:53.670643  7866 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:03:53.611894  7984 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.680302  7977 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.620275  8071 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:03:53.681164  7984 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.684827  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:53.689062  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:53.674384  7450 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.676406  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:53.688800  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:53.671854  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:53.678970  8033 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.690644  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.687942  7831 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.693704  1567 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:53.700155  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.696178  1719 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:03:53.639426  7902 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:53.677239  7948 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:53.707234  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:53.738572  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.730334  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.744660  7984 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.745504  7984 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.749001  7984 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:53.749872  7984 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:53.751647  7984 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:53.753750  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.754225  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3099) took 1174 ms. Trace:
I20260709 13:03:53.754603  1567 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:03:53.756098  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44394.4us
I20260709 13:03:53.754633  1569 rpcz_store.cc:276] 0709 13:03:52.583201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.196842 (+613641us) service_pool.cc:224] Handling call
0709 13:03:53.207752 (+ 10910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.221151 (+ 13399us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:53.330360 (+109209us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.753558 (+423198us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.753799 (+   241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.753858 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:53.753918 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.753998 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.754152 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":199090,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":81678,"raft.run_wall_time_us":13094,"spinlock_wait_cycles":46902016,"thread_start_us":508277,"threads_started":3,"wal-append.queue_time_us":369598}
I20260709 13:03:53.758164  1720 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
W20260709 13:03:53.775822  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.790053  1576 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:03:53.737552  7902 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.805051  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.814612  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.854516  1567 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:03:53.760124  7948 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:53.872898  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 198334us
I20260709 13:03:53.683650  8066 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:53.892956  1722 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:53.906680  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:53.892876  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:53.920504  1567 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:03:53.933204  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 229352us
I20260709 13:03:53.901335  7899 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:53.961439  7850 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.006s
I20260709 13:03:53.962273  7850 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:53.965086  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.924754  8094 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:53.968652  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:53.952463  8052 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.069s	user 0.015s	sys 0.001s
I20260709 13:03:53.925480  8098 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:53.929956  8083 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:53.909439  7948 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:53.987722  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:53.989907  1719 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:03:53.994725  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.863s (SENT)
I20260709 13:03:53.914422  7951 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:53.943140  8097 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:53.973280  7909 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.144s
I20260709 13:03:54.020586  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:54.012044  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:53.969630  8094 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:54.029629  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 136882us
I20260709 13:03:53.984416   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:53.978710  8090 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.978323  8100 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:54.032305  7621 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.025472  8097 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:53.978029  8052 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.055056  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 119298us
I20260709 13:03:54.036273  8106 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:54.069720  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.038892  8104 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:54.010346  8098 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.077548  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 82833us
I20260709 13:03:54.060271  8069 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:54.074307  8009 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:03:54.031469  8099 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:03:54.046664  8090 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.094980  8090 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.095613  7948 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.096531  7948 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:54.100399  8052 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.101082  7850 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.055821  8002 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:54.101976  8002 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:54.063338  8100 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.102989  8002 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.105165  8100 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.092144  8112 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:54.106324  8064 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:54.106884  8064 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:54.106536  8112 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.108913  7951 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:54.114220  8095 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:03:54.115347  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.609s (SENT)
I20260709 13:03:54.099471  8097 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.117396   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:54.117664  8097 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.120676  8098 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:54.121785  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.018s	sys 0.021s
W20260709 13:03:54.122323  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.018s	sys 0.021s
I20260709 13:03:54.116048  2344 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:54.123008  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 571492us
I20260709 13:03:54.097255  7948 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.116156  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.145622  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
W20260709 13:03:54.149900  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 98506.4us
I20260709 13:03:54.157586  1648 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:03:54.161312  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 381408us
I20260709 13:03:54.165202  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:54.165364  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:54.167734  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:54.167791  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:54.174829  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:54.158483  7690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.007s
W20260709 13:03:54.179730  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.160940  8098 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:54.181304  1720 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:03:54.181774  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.151301  8099 consensus_queue.cc:1048] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:03:54.185747  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:54.178088  7690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.003s	sys 0.008s
I20260709 13:03:54.185237  7909 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:54.074846  8106 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.212908  7909 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.215103  8106 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.198982  8080 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.199579  8110 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:03:54.200594  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:54.200393  8098 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.220711  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 105462us
I20260709 13:03:54.220978  7450 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.222765  8098 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:54.185226  8088 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.300s	user 0.000s	sys 0.002s
I20260709 13:03:54.201242  8113 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.208786  7831 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:54.226814  7831 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.216197  7909 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.210088  7914 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:54.242542  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:54.238883  1574 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:54.271343  1568 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:54.272061  1574 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:54.233860  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.249518  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:54.244612  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:54.233966  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 76850.9us
I20260709 13:03:54.238996  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:54.108017  8064 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.177327  8090 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.229504  7831 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.285480  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:54.286535  1576 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.293987  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:54.295143  1569 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:54.296787  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 607645us
I20260709 13:03:54.297109  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:54.121909  8094 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.235180  8112 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.241765  8100 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.237783  7948 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.231263  7914 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:54.328444  1574 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:54.326002  8103 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.254s	user 0.004s	sys 0.000s
I20260709 13:03:54.347601  8118 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:54.329840  7825 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:03:54.351434  7914 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.350942  8109 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:03:54.352988  1574 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:54.377871  7914 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.234056  8009 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:03:54.380249  7825 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:54.380733  7825 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:54.381868  7825 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.353813  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.354153  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:54.374748  8118 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.338343  8121 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.384299  8099 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.364779  8002 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.369822  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2745) took 2006 ms. Trace:
I20260709 13:03:54.372871  8064 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.316701  8104 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.399855  1572 rpcz_store.cc:276] 0709 13:03:52.363173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.688075 (+324902us) service_pool.cc:224] Handling call
0709 13:03:53.467098 (+779023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.496990 (+ 29892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.369328 (+872338us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54001a9fd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:54.369654 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1789642496}
I20260709 13:03:54.384618  8121 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.431555  7936 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:54.432169  7936 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:54.437311  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.445750  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.298980  1570 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:54.458287  1690 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:54.463608  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 148324us
W20260709 13:03:54.464395  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 270596us
W20260709 13:03:54.464841  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 123403us
I20260709 13:03:54.470628  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:03:54.473519  7902 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.021s
I20260709 13:03:54.474152  7902 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.433820  7936 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.477330  7936 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.448632  8113 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:54.478793  8113 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.448470  8065 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.482465  8065 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.467975  8110 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:54.484833  8110 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.485306  8110 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.447911  7690 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.449067  8104 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:54.462165  7971 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.519557  1574 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:03:54.364781  8100 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.523556  1574 pending_rounds.cc:85] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:03:54.519649  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:54.524302  1574 pending_rounds.cc:107] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:03:54.525745  1574 raft_consensus.cc:2889] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:54.468428  8063 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.531436  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:03:54.532203  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:54.500882  7948 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.558995  1569 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:03:54.522099  8071 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.010s	sys 0.022s
W20260709 13:03:54.561599  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.800s (SENT)
W20260709 13:03:54.564191  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.565749  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:54.561798  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:54.568186  1568 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:54.566159  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:54.569155  1648 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:03:54.578426  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:54.581966  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:54.583033  1716 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:54.583262  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.583387  1725 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:54.585824  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.581128  7902 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.589974  8121 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.573279  7866 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:03:54.591229  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78448.9us
I20260709 13:03:54.593243  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:54.594313  7902 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:54.595193  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:54.600838  8065 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.603461  8118 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:54.609017  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.618072  8113 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:54.621733  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 90363.9us
I20260709 13:03:54.625682  8063 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.506980  8112 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:03:54.629724  8065 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:54.629676  7902 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.630718  8112 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:54.629745  8100 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.631780  8112 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:54.630720  8065 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:54.634441  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:54.637279  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99692.5us
I20260709 13:03:54.524274  7690 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:54.643651  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.641942  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65748.7us
I20260709 13:03:54.573539  8098 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.539609  8106 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.505587  8119 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.154s
I20260709 13:03:54.625174  7831 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.529124  7971 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.658525  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:54.533124  8122 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:03:54.531711  8104 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.587272  7909 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:54.675290  8104 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.653237  8033 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:03:54.677534  7909 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.685930  7971 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.689672  8112 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.691231  8112 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.693024  8065 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.680001  7909 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:54.700343  1568 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:54.702777  1568 raft_consensus.cc:1217] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Deduplicated request from leader. Original: 12.6->[13.7-16.9]   Dedup: 13.7->[14.8-16.9]
I20260709 13:03:54.703600  1568 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: [13.7-16.9]
I20260709 13:03:54.706900  7831 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.708091  7909 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:54.709558  7825 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:54.710901  8033 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:54.717523  8110 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.713843  1576 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
W20260709 13:03:54.723249  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88833.3us
I20260709 13:03:54.658424  8097 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:54.778481  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 105681us
I20260709 13:03:54.670799  8116 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:54.757731  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.786003  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.785996  1688 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:54.785009  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:54.786563  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.738221  8009 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:54.798156  8009 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.740984  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:03:54.805589  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64724.8us
I20260709 13:03:54.826056  7690 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:54.834199  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.839171  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:54.813076  8002 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:54.861311  8009 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:54.864185  8119 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:54.814586  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 80510.9us
I20260709 13:03:54.870658  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:54.778282  8033 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:54.876390  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 138734us
I20260709 13:03:54.738602  8106 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:03:54.882624  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.883215  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:54.886003  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52828.8us
I20260709 13:03:54.728749  8080 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.151s
I20260709 13:03:54.890053  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:54.738648  7831 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.894716  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:54.873895  8064 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.897697  2377 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:54.899700  2379 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:54.900641  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:54.901482  2378 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:54.891606  8052 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:03:54.906006  8063 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:54.906803  8063 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.879631  8131 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:54.911660  8131 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.911929  8063 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:54.912104  8131 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.885684  8116 raft_consensus.cc:2764] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:03:54.895464  1688 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.767s (SENT) [suppressed 7 similar messages]
W20260709 13:03:54.940066  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 155076us
W20260709 13:03:54.945449  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.905006  8002 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:54.932013  8098 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:54.957352  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:54.893358  8106 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:54.980957  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:54.982414  2378 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:54.914311  8033 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:54.988050  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:55.029531  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.029664  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:55.031441  1725 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:54.966838  7909 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.978749  8098 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:03:55.042227  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 146704us
I20260709 13:03:55.037600  2377 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:55.051497  2378 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:55.045830  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:03:55.061909  8139 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:55.065201  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:55.095405  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:55.096612  2378 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:03:55.099143  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.105465  8033 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:55.106606  8135 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:03:55.107123  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:55.108425  2376 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:55.110016  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:55.112754  8134 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:03:55.114430  8138 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:03:55.115866  7866 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:55.116639  7866 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.117022  8098 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.118881   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:55.121918  7831 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:55.123004  8110 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:55.124795  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:55.064163  8063 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:55.089332  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43584.5us
I20260709 13:03:55.133920  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:55.125833  2375 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:55.048897  7909 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.159777  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:55.138882  7977 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.170682  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:55.148658  8065 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:55.180737  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:55.179575  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:55.183419  2375 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.149231  8063 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:55.143064  8110 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:03:55.089417  8002 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.165406  7825 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.173779  8146 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:55.169687  7984 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.185324  7839 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:55.197693  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88097.1us
I20260709 13:03:55.164670  8116 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:55.145607  8119 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:03:55.210672  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.176172  8097 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:03:55.199681  7971 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:55.178848  7977 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:55.237268  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.203047  7825 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.214051  7914 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:55.217123  8137 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:55.225165  7971 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.246722  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:55.213608  8064 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:03:55.214363  8116 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:55.253865  8133 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 13:03:55.254416  8106 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:55.254510  8137 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:55.253201  8140 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:03:55.274832  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.270068  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.294494  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.209229  7984 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.284278  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:55.257676  7971 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.250020  8063 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.304924  2375 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:55.309124  7902 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:55.311038  7825 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:55.296708  8097 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:55.312208  8097 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.315887  8097 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:55.249931  7914 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:55.317353  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66855.7us
I20260709 13:03:55.318481  8098 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.334272  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:55.305249  8153 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.354465  7839 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.358039  7936 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:55.358927  7936 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:55.360268  7936 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.363232  7914 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:55.320307  8119 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:55.370146  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:55.392254  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:55.394644  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:55.396572  8110 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:55.397192  7914 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:55.397779  8106 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:55.396476  8009 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:55.398429  8066 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:55.336520  7909 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:55.401064  7825 raft_consensus.cc:993] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:55.332130  7850 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.400473  8119 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:55.389364  8147 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:03:55.409228  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2760) took 1157 ms. Trace:
W20260709 13:03:55.452399  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50669.8us
I20260709 13:03:55.465988  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:55.414532  7951 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.417413  7977 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:55.367568  8098 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.451155  7914 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:55.479713  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2768) took 1188 ms. Trace:
I20260709 13:03:55.457302  1572 rpcz_store.cc:276] 0709 13:03:54.251566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.435976 (+184410us) service_pool.cc:224] Handling call
0709 13:03:54.462948 (+ 26972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.463224 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:54.766945 (+303721us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.375806 (+608861us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b180012f100. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc830c83a40 00007fc830c7e4e4 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:03:55.407529 (+ 31723us) log.cc:844] Serialized 14 byte log entry
0709 13:03:55.408211 (+   682us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:55.408300 (+    89us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:55.408548 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.408866 (+   318us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.409128 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516592,"prepare.queue_time_us":948625,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":75,"raft.queue_time_us":8349,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6820,"spinlock_wait_cycles":3507584,"thread_start_us":338698,"threads_started":2,"wal-append.queue_time_us":187246}
I20260709 13:03:55.412236  8122 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:03:55.492487  7914 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.495252  8098 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:55.496062  8098 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.390583  8153 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:55.502616  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 100588us
I20260709 13:03:55.454378  7850 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.489159  1574 rpcz_store.cc:276] 0709 13:03:54.442922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.718331 (+275409us) service_pool.cc:224] Handling call
0709 13:03:54.718922 (+   591us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:54.719522 (+   600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.719566 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:54.725419 (+  5853us) log.cc:844] Serialized 52 byte log entry
0709 13:03:54.933649 (+208230us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:54.933702 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.933793 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.479333 (+545540us) raft_consensus.cc:1685] finished
0709 13:03:55.479370 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.479616 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":216,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":146,"thread_start_us":144272,"threads_started":1,"wal-append.queue_time_us":247753}
I20260709 13:03:55.512630  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:55.517900  8161 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.523389  8165 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:55.523896  8165 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:55.524646  8153 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:55.525146  8165 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.527000  8165 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.527275  8157 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.528474  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:55.529130  8115 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.176s	user 0.007s	sys 0.008s
W20260709 13:03:55.530985  8160 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:03:55.528275  8157 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.532816  8164 raft_consensus.cc:1081] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:03:55.533818  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:55.534132  8164 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.534806  8164 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:55.535770  8159 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:03:55.522358  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:55.538050  8164 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:55.539708  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:55.540105  8131 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.627s	user 0.014s	sys 0.000s
I20260709 13:03:55.540462  1650 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:55.544742  8116 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:55.549197  8137 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:55.549400  8125 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.978s	user 0.002s	sys 0.002s
I20260709 13:03:55.550522  7984 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:55.552776  7850 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:55.525684  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:55.582134  1643 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:03:55.554359  8119 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:55.508555  7902 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.560148  8066 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:55.560793  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:55.565167  1651 raft_consensus.cc:2410] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:03:55.482542  7951 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.584048  1648 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:55.569466  1644 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: [13.7-16.9]
I20260709 13:03:55.578845  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2769) took 1134 ms. Trace:
I20260709 13:03:55.591275  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:55.578723  1649 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:55.592543  1568 rpcz_store.cc:276] 0709 13:03:54.444684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.729796 (+285112us) service_pool.cc:224] Handling call
0709 13:03:54.784151 (+ 54355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.803442 (+ 19291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.803479 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.841938 (+ 38459us) log.cc:844] Serialized 26 byte log entry
0709 13:03:55.556214 (+714276us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b18002f6aa0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1
0709 13:03:55.562284 (+  6070us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:55.562328 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:55.562403 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.562449 (+    46us) raft_consensus.cc:1685] finished
0709 13:03:55.562459 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.578766 (+ 16307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32467,"raft.queue_time_us":6849,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":27323,"spinlock_wait_cycles":4539264,"thread_start_us":35873,"threads_started":1,"wal-append.queue_time_us":91197}
I20260709 13:03:55.610549  8106 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:55.612098  8106 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.614352  8009 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:55.509164  8098 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:55.648800  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:55.655890  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:55.656889  1575 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.649458  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:55.663793  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 157168us
I20260709 13:03:55.665616  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.665939  1573 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:03:55.668279  1572 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:55.681486  1572 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:55.607761  7902 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.666587  1573 pending_rounds.cc:85] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:03:55.762295  1573 pending_rounds.cc:107] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:03:55.762725  1573 raft_consensus.cc:2889] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.769155  1646 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.769500  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.770541  1568 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:03:55.771636  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.581516  8131 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.615759  8009 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:55.797140  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:55.783756  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 110374us
I20260709 13:03:55.801404  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:55.806834  1568 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:55.818814  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2752) took 1581 ms. Trace:
I20260709 13:03:55.836097  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:55.819437  1567 rpcz_store.cc:276] 0709 13:03:54.237543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.239232 (+  1689us) service_pool.cc:224] Handling call
0709 13:03:54.239676 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.240328 (+   652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.240382 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.241009 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.439228 (+198219us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:54.439279 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.439365 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:55.548553 (+1109188us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:55.800466 (+251913us) raft_consensus.cc:1685] finished
0709 13:03:55.800490 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:55.818707 (+ 18217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21772,"raft.queue_time_us":39648,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9028,"spinlock_wait_cycles":10342656,"thread_start_us":197728,"threads_started":1,"wal-append.queue_time_us":249066}
I20260709 13:03:55.789973  1572 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:55.838455  8033 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.732s	user 0.014s	sys 0.004s
W20260709 13:03:55.863325  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.862453  1571 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:55.868459  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.875046  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:55.879489  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:55.881623  1567 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:03:55.885386  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.886973  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:55.896723  1576 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:03:55.901453  1576 pending_rounds.cc:85] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:03:55.901870  1576 pending_rounds.cc:107] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:03:55.902201  1576 raft_consensus.cc:2889] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:55.859378  8157 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:55.915812  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:55.916903  1567 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:55.920519  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:55.926599  1650 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:55.927774  1643 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:55.931828  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:55.933902  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:55.828653  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.955840  1568 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:55.987502  1575 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:55.988973  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.990852  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:55.991641  1568 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:55.859042  8178 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:03:55.999178  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:56.034675  1569 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:56.034979  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:56.042593  1642 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:56.047881  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:56.049362  1575 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.960697  8033 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.055193  1642 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:03:56.060695  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 259258us
W20260709 13:03:56.055368  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64168.4us
I20260709 13:03:55.996858  8122 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:55.996482  8065 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.073295  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:03:55.996806  8116 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.028153  8002 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:55.960697  8166 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:56.059970  8161 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.001s
I20260709 13:03:56.092115  8181 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.093541  8181 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:56.098824  8179 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:03:56.092375  8131 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:56.112389  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 172437us
I20260709 13:03:56.114434  1575 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:56.117676  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:56.125705  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:56.082928  8170 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:56.065251  8033 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:56.132997  8165 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.004s
I20260709 13:03:56.135358  1645 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:56.136474  8002 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:56.137288  8002 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.137737  8002 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.139561  8181 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.086640  8116 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.141475  1568 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 19 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:56.105644  8161 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.089812  7951 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.114547  8157 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:56.476639  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 350866us
W20260709 13:03:56.483192  1567 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.565s	user 0.016s	sys 0.000s
W20260709 13:03:56.124836  8163 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.456s	user 0.001s	sys 0.001s
I20260709 13:03:56.483789  1567 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
W20260709 13:03:56.486785  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 413406us
I20260709 13:03:56.117424  8066 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:56.365813  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:56.493904  1646 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.716s	user 0.016s	sys 0.000s
W20260709 13:03:56.266145  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.002s	sys 0.057s
W20260709 13:03:56.497249  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.032s	sys 0.081s
W20260709 13:03:56.235742  8152 log.cc:927] Time spent T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.951s	user 0.008s	sys 0.008s
I20260709 13:03:56.495393  1646 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
W20260709 13:03:56.169054  7914 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.002s	sys 0.029s
I20260709 13:03:56.512645  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:56.515152  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 1.04873e+06us
W20260709 13:03:56.516892  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 444094us
I20260709 13:03:56.518905  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:56.519486  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:56.474756  8116 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:56.462235  8131 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:56.496446  8110 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.421581  8161 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:56.510766  7914 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.031s	sys 0.064s
W20260709 13:03:56.521950  7971 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.218s	user 0.011s	sys 0.007s
W20260709 13:03:56.561940  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.569164  8189 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:03:56.510630  8066 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:56.577026  8169 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.577723  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:56.557893  8131 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:56.578564  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55764.6us
I20260709 13:03:56.560114  8110 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.608565  1573 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:56.609519  8033 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.513983  8157 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:56.611564  8110 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:56.612713  8157 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.577011  8122 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.594143  7971 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.614861  8136 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:56.615384  8136 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:56.615943  7971 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:56.616752  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:56.620668  1647 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:56.621848  1573 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:56.622450  8094 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.597085  8009 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:56.625262  1645 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:03:56.629922  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:56.087000  8166 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.168790  8170 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:56.625761  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:56.666450  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:56.689185  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:56.691212  1718 raft_consensus.cc:2368] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:56.687826  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.640568  8009 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:56.692291  1716 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:56.683676  7936 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:03:56.649505  8106 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:56.694968  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:56.695165  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.581275  8161 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.697661  7643 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:56.697948  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.652201  8110 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:56.698961  8189 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:56.604892  8169 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.739955  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.743418  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.649803  7914 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:56.744612  8066 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.745620  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.746371  1721 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:56.746198  8106 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:56.711279  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 105401us
I20260709 13:03:56.747743  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:56.737042  8189 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:56.710402  8136 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:56.675729  8165 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.660220  7971 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.714792  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.709980  8009 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:56.693432  7825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.012s	sys 0.026s
I20260709 13:03:56.737260  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.738854  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.745014  7914 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:56.655048  8122 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:56.667055  8119 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.755373  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.692642  8157 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:56.766327  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2373) took 1244 ms. Trace:
I20260709 13:03:56.768846  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.752277  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:56.762480  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.731779  8110 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:56.716578  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 394120us
W20260709 13:03:56.748325  8002 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.003s
W20260709 13:03:56.802634  7825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.013s	sys 0.026s
I20260709 13:03:56.756846  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.750073  8170 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.837891  1646 rpcz_store.cc:276] 0709 13:03:55.521774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.521919 (+   145us) service_pool.cc:224] Handling call
0709 13:03:56.765923 (+1244004us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54003fd754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:56.766180 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":534185344}
W20260709 13:03:56.858570  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.790066  8161 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:56.762073  8166 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:56.861141  7866 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:56.767400  8165 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:56.756807  8106 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.751893  8136 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.753238  8169 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:56.767685  8175 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:56.759065  8098 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.753181  8033 raft_consensus.cc:2764] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:56.961238  8175 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:03:56.947660  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 195377us
I20260709 13:03:56.751741  8189 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:56.962535  8175 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.769622  7971 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.953663  8157 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:56.973042  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.978520  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:56.978937  1725 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:03:56.979420  7971 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:56.980187  8033 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.980824  8193 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.981421  8193 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.983640  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:56.987320  7882 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.985234  8193 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:56.985966  8187 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.197s	user 0.001s	sys 0.002s
I20260709 13:03:56.976945  8175 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.999284  8131 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.002421  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.003515  1721 raft_consensus.cc:2410] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:03:57.008884  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.012135  8033 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:56.868958  8065 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.034665  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.035873  1722 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:57.049616  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.050956  1722 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
W20260709 13:03:57.068529  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.111442  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:56.836612  8122 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:56.847656  8119 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.017066  8065 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.118703  1723 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:03:57.103403  8094 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:57.120776  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.161437  1718 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:03:56.992048  7882 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.119911  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:56.903967  8002 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.153905  1722 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:57.124536  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:03:57.164561  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.019s	sys 0.032s
W20260709 13:03:57.173134  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.019s	sys 0.033s
W20260709 13:03:57.174046  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 661140us
I20260709 13:03:57.172092  1724 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:03:57.176541  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.014s	sys 0.014s
I20260709 13:03:56.995563  8110 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.192720  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:57.193660  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.008s	sys 0.038s
W20260709 13:03:57.182548  1575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.028s	sys 0.061s
W20260709 13:03:57.194149  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.038s
I20260709 13:03:57.189152  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:57.188232  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.014s	sys 0.014s
I20260709 13:03:57.195158  1722 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
W20260709 13:03:57.196775  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 176985us
I20260709 13:03:57.197450  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3134) took 1377 ms. Trace:
W20260709 13:03:57.194603  1575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.029s	sys 0.062s
I20260709 13:03:57.193568  1723 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:03:57.174077  8174 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.381s	user 0.024s	sys 0.024s
W20260709 13:03:57.210139  1642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.015s	sys 0.060s
W20260709 13:03:57.194603  8197 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.157s	user 0.003s	sys 0.000s
W20260709 13:03:57.201972  1617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.014s	sys 0.015s
W20260709 13:03:57.202610  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 722388us
W20260709 13:03:57.234385  1617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.015s	sys 0.017s
I20260709 13:03:57.237371  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:57.239187  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.016s	sys 0.071s
W20260709 13:03:57.241066  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.016s	sys 0.071s
I20260709 13:03:57.198668  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:03:57.223057  1642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.017s	sys 0.061s
W20260709 13:03:57.222035  1650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.018s	sys 0.040s
I20260709 13:03:57.240994  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:57.248960  1650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.018s	sys 0.040s
W20260709 13:03:57.254767  1571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.026s	sys 0.056s
W20260709 13:03:57.270318  1571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.028s	sys 0.057s
W20260709 13:03:57.250211  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51628.3us
W20260709 13:03:57.223587  8178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.019s	sys 0.058s
W20260709 13:03:57.215366  8183 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.523s	user 0.007s	sys 0.005s
I20260709 13:03:57.170728  8094 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.284672  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:57.222500  8174 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.430s	user 0.025s	sys 0.025s
W20260709 13:03:57.304900  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.283949  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2379) took 1700 ms. Trace:
I20260709 13:03:57.287724  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3117) took 1658 ms. Trace:
I20260709 13:03:57.308764  2379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 741) took 2395 ms. Trace:
I20260709 13:03:57.227572  8110 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.361694  1650 rpcz_store.cc:276] 0709 13:03:55.569322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.928579 (+359257us) service_pool.cc:224] Handling call
0709 13:03:55.929230 (+   651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.930057 (+   827us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:56.041226 (+111169us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.269433 (+1228207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.269507 (+    74us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:57.269569 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.269652 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.269802 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112193,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":219944,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":1227594,"threads_started":2,"wal-append.queue_time_us":573624}
I20260709 13:03:57.366963  1570 rpcz_store.cc:276] 0709 13:03:55.629128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.629279 (+   151us) service_pool.cc:224] Handling call
0709 13:03:55.629844 (+   565us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:55.759427 (+129583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:55.759460 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:55.800367 (+ 40907us) log.cc:844] Serialized 78 byte log entry
0709 13:03:56.161117 (+360750us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:56.215148 (+ 54031us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.215471 (+   323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.286939 (+1071468us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540030a054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:57.287037 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.287155 (+   118us) raft_consensus.cc:1685] finished
0709 13:03:57.287313 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.287614 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74539,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":972,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":17340,"spinlock_wait_cycles":314214016,"thread_start_us":325312,"threads_started":1,"wal-append.queue_time_us":742774}
I20260709 13:03:57.265655  8189 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:57.406183  8065 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.276715  8033 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:57.306574  8178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.022s	sys 0.060s
I20260709 13:03:57.415243  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:57.432297  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 391185us
I20260709 13:03:57.432178  1570 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:57.434676  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.435721  1570 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:03:57.403688  8119 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:57.420560  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.030s	sys 0.059s
W20260709 13:03:57.423272  1651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.034s	sys 0.037s
W20260709 13:03:57.479615  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.035s	sys 0.062s
W20260709 13:03:57.479702  1651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.034s	sys 0.038s
I20260709 13:03:57.437474  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.284898  8195 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.447067  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:57.368621  2379 rpcz_store.cc:276] 0709 13:03:54.937560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.942423 (+  4863us) service_pool.cc:224] Handling call
0709 13:03:54.954305 (+ 11882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.954579 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:54.978269 (+ 23690us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.290901 (+2312632us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b1800422260. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d78716 00007fc833d89287
0709 13:03:57.291097 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.291198 (+   101us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:57.291286 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.291408 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.291645 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23406,"prepare.queue_time_us":23861,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":51258,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":17065,"spinlock_wait_cycles":14065152,"thread_start_us":166123,"threads_started":2,"wal-append.queue_time_us":228470,"wal-append.run_cpu_time_us":3765,"wal-append.run_wall_time_us":1263458}
W20260709 13:03:57.452265  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.490149  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 205492us
I20260709 13:03:57.482837  8182 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:57.478416  7866 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.489377  8157 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:57.483490  7825 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:57.487507  8204 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.503643  2376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 743) took 2361 ms. Trace:
W20260709 13:03:57.511615  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 2.26626e+06us
I20260709 13:03:57.513574  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:57.505496  2376 rpcz_store.cc:276] 0709 13:03:55.141999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.142182 (+   183us) service_pool.cc:224] Handling call
0709 13:03:55.156012 (+ 13830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.156306 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:55.212895 (+ 56589us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.503204 (+2290309us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.503269 (+    65us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:57.503319 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.503457 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.503584 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2235083,"prepare.queue_time_us":47577,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"thread_start_us":97285,"threads_started":2,"wal-append.queue_time_us":199603,"wal-append.run_cpu_time_us":2616,"wal-append.run_wall_time_us":1108352}
I20260709 13:03:57.517516  7450 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:57.518246  7450 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:57.518951  7450 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:57.511638  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.523164  7831 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:57.516759  8181 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.523536  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:57.523777  7831 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:57.523555  8002 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.529294  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.529919  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.530197  1717 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:57.524992  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 742) took 2387 ms. Trace:
I20260709 13:03:57.483592  8193 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:57.529372  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.531483  8193 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:57.532806  7909 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:57.533226  7909 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:57.533627  7909 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:57.534256  2344 connection.cc:256] RPC call timeout handler was delayed by -2.1119390289999842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 13:03:57.534734  2344 connection.cc:256] RPC call timeout handler was delayed by -1.8226059549999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 13:03:57.535065  2344 connection.cc:256] RPC call timeout handler was delayed by -1.7129086299999585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 13:03:57.523967  8181 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:57.535329  2344 connection.cc:256] RPC call timeout handler was delayed by -1.6750059550000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 13:03:57.539883  2344 connection.cc:256] RPC call timeout handler was delayed by -1.4357086299999651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
I20260709 13:03:57.537361  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2776) took 1620 ms. Trace:
I20260709 13:03:57.540501  1568 rpcz_store.cc:276] 0709 13:03:55.920817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.034552 (+113735us) service_pool.cc:224] Handling call
0709 13:03:57.536358 (+1501806us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54002a76d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:57.537274 (+   916us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241408896}
I20260709 13:03:57.478173  8202 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:57.497931  8178 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:57.542953  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.499850  8199 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.540380  2344 connection.cc:256] RPC call timeout handler was delayed by -1.2872086300000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
W20260709 13:03:57.561888  2344 connection.cc:256] RPC call timeout handler was delayed by -1.0627040319999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
I20260709 13:03:57.560395  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.536056  2372 rpcz_store.cc:276] 0709 13:03:55.138146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.138309 (+   163us) service_pool.cc:224] Handling call
0709 13:03:55.152225 (+ 13916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.152517 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:55.170875 (+ 18358us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.524660 (+2353785us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.524725 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:57.524768 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.524831 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.524932 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2203391,"prepare.queue_time_us":37386,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":2214394,"thread_start_us":149827,"threads_started":1,"wal-append.queue_time_us":299980,"wal-append.run_cpu_time_us":1790,"wal-append.run_wall_time_us":1018584}
W20260709 13:03:57.552120  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.555902  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:57.560590  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.566984  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:57.567682  1718 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:57.563129  1568 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:03:57.570868  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.573001  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.573621  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:57.572980  1568 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 13: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:03:57.577958  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.579072  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1684858670000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:03:57.579569  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1684858670000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:03:57.579944  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1603858670000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:03:57.580317  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1603858670000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:03:57.580660  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1234858669999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 13:03:57.580955  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1234858669999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 13:03:57.581200  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0163858669999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:03:57.581450  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0163858669999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:03:57.581696  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0046858670000347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260709 13:03:57.581948  1539 connection.cc:256] RPC call timeout handler was delayed by -1.0046858670000347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:03:57.574234  2377 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:03:57.530938  7831 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.541404  8118 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:57.538863  8094 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.594082  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:57.501564  7825 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:57.600605  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.638141  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:57.558763  8178 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:57.498936  7866 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:57.649104  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.668295  1539 leader_election.cc:400] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
W20260709 13:03:57.668812  1614 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.023s	sys 0.108s
W20260709 13:03:57.670112  1614 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.023s	sys 0.108s
W20260709 13:03:57.670653  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 669277us
I20260709 13:03:57.670796  2378 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:57.572360  8146 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:57.668277  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 69955us
I20260709 13:03:57.498440  8182 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:57.676798  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.668982  1568 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:57.684058  1646 raft_consensus.cc:2410] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:03:57.675311  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.682056  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.588016  8033 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.602110  8066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.007s	sys 0.021s
I20260709 13:03:57.668848  1539 leader_election.cc:403] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:57.684967  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.700376  1721 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:03:57.701344  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 118805us
I20260709 13:03:57.703299  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:57.704734  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:57.700829  1568 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:57.706331  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.707629  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.708535  1568 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:03:57.710110  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.710928  1568 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:03:57.645337  8094 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.712301  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.715377  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:57.669308  8178 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:57.715660  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.718215  1568 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:57.719516  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.722787  1646 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:57.561394  8202 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:57.712116  8066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.009s	sys 0.024s
I20260709 13:03:57.673398  8002 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:57.674952  8146 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.719395  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:57.734372  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 287234us
I20260709 13:03:57.728746  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.722975  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:03:57.735680  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 494530us
I20260709 13:03:57.742254  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:57.741441  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:57.745282  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:57.575573  8199 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.748889  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2382) took 2178 ms. Trace:
I20260709 13:03:57.749945  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:57.755441  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.976s (SENT)
I20260709 13:03:57.756074  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:57.756778  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.715708  8112 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16.
I20260709 13:03:57.760035  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:57.725458  8204 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.773458  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.767004  1616 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:57.770165  1725 raft_consensus.cc:2410] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:57.751789  1642 rpcz_store.cc:276] 0709 13:03:55.570342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.057947 (+487605us) service_pool.cc:224] Handling call
0709 13:03:56.071030 (+ 13083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.071492 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:56.461847 (+390355us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.748458 (+1286611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.748516 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:57.748586 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.748657 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.748832 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22466,"prepare.queue_time_us":15057,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":226451,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":329,"thread_start_us":1653121,"threads_started":3,"wal-append.queue_time_us":1037302}
I20260709 13:03:57.771778  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.776535  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.778959  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.771807  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:57.781575  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:57.712909  8033 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.777275  1645 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:57.790659  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:57.765195  8170 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:57.767685  8157 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:57.799314  8157 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.745453  8146 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.727622  8094 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.203145  1567 rpcz_store.cc:276] 0709 13:03:55.819988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.516330 (+696342us) service_pool.cc:224] Handling call
0709 13:03:56.531573 (+ 15243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.553796 (+ 22223us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:56.625211 (+ 71415us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.196782 (+571571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:57.196925 (+   143us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:57.197002 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.197088 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.197221 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70882,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":1107,"thread_start_us":570086,"threads_started":1}
I20260709 13:03:57.805220  8094 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.806201  2380 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
W20260709 13:03:57.809158  8215 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:03:57.811012  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2774) took 1895 ms. Trace:
W20260709 13:03:57.813004  8207 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:03:57.811503  1573 rpcz_store.cc:276] 0709 13:03:55.915878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.654909 (+739031us) service_pool.cc:224] Handling call
0709 13:03:56.692416 (+ 37507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.692869 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.692904 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.693476 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.269695 (+576219us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:57.269736 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.269804 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.810716 (+540912us) raft_consensus.cc:1685] finished
0709 13:03:57.810744 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.810927 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5943,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"thread_start_us":458652,"threads_started":1,"wal-append.queue_time_us":884255}
I20260709 13:03:57.814371  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3121) took 2182 ms. Trace:
I20260709 13:03:57.814869  1571 rpcz_store.cc:276] 0709 13:03:55.632072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.055319 (+423247us) service_pool.cc:224] Handling call
0709 13:03:56.083753 (+ 28434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.084259 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:56.084294 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.112945 (+ 28651us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.271694 (+1158749us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.271764 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.271884 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.814160 (+542276us) raft_consensus.cc:1685] finished
0709 13:03:57.814179 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.814315 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2084,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":171,"thread_start_us":1030155,"threads_started":1,"wal-append.queue_time_us":1395790}
W20260709 13:03:57.818554  8205 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:03:57.818657  8219 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.002s
I20260709 13:03:57.793310  8066 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:57.892970  8066 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.893548  8066 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.893301  7450 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:03:57.819548  8201 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
I20260709 13:03:57.808059  8122 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:57.820292  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:57.820905  8224 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:57.822044  8209 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:03:57.822790  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:57.823312  8214 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:57.823951  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:57.826071  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.826864  8221 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:57.829783  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.834146  8164 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:57.835834  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:57.838920  1572 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:57.840384  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:57.841549  8213 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:03:57.842042  8210 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:03:57.842825  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87348.1us
I20260709 13:03:57.845882  7825 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:57.848256  8202 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:57.849700  8146 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:57.850382  8182 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:57.853070  8199 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.853988  8033 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.750492  8002 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.777863  8174 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:57.753290  7882 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.766250  8151 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.794438  8165 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:03:57.858057  8136 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.901s	user 0.015s	sys 0.006s
I20260709 13:03:57.794097  8137 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:57.860860  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:57.861711  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71109.9us
W20260709 13:03:57.864390  8009 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.035s	user 0.017s	sys 0.004s
I20260709 13:03:57.793725  8204 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.877007  8181 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.878017  8225 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:57.881879  2380 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:57.805261  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:57.883621  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:57.797646  8170 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:57.982735  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2383) took 2412 ms. Trace:
I20260709 13:03:57.910631  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3115) took 2283 ms. Trace:
I20260709 13:03:57.886610  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.774693  7971 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:57.984172  8193 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:57.988087  1647 rpcz_store.cc:276] 0709 13:03:55.570728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.101048 (+530320us) service_pool.cc:224] Handling call
0709 13:03:56.101565 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.626350 (+524785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.982276 (+1355926us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54003fcd54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:57.982647 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192478976}
I20260709 13:03:57.990430  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:57.990749  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.991159  1649 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:57.991751  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3133) took 2208 ms. Trace:
I20260709 13:03:57.992807  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2380) took 2423 ms. Trace:
W20260709 13:03:57.940198  8200 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.528s	user 0.000s	sys 0.003s
I20260709 13:03:57.994091  8146 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.993186  1643 rpcz_store.cc:276] 0709 13:03:55.569639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.929364 (+359725us) service_pool.cc:224] Handling call
0709 13:03:55.965572 (+ 36208us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:56.762542 (+796970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.762580 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:56.763992 (+  1412us) log.cc:844] Serialized 52 byte log entry
0709 13:03:57.482019 (+718027us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:57.482071 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.933549 (+451478us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b540038b3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:57.933620 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.992534 (+ 58914us) raft_consensus.cc:1685] finished
0709 13:03:57.992556 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.992754 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15320,"raft.queue_time_us":26392,"raft.run_cpu_time_us":1397,"raft.run_wall_time_us":13501,"spinlock_wait_cycles":1830976640,"thread_start_us":675746,"threads_started":1,"wal-append.queue_time_us":905727}
I20260709 13:03:57.926431  8178 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:57.995049  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.975698  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.996870  1642 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:57.997073  8178 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:57.997711  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.997874  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.998354  1574 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:57.886062  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:57.999513  1571 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:57.999764  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 983) took 1433 ms. Trace:
I20260709 13:03:57.999945  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2787) took 1197 ms. Trace:
I20260709 13:03:58.000468  1571 rpcz_store.cc:276] 0709 13:03:56.803046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.880873 (+1077827us) service_pool.cc:224] Handling call
0709 13:03:57.999888 (+119015us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.000144  1651 rpcz_store.cc:276] 0709 13:03:56.565933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.689297 (+123364us) service_pool.cc:224] Handling call
0709 13:03:56.689963 (+   666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.699933 (+  9970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.699978 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.717781 (+ 17803us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.490182 (+772401us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:57.490246 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.490350 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.999583 (+509233us) raft_consensus.cc:1685] finished
0709 13:03:57.999600 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.999712 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61861,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":114,"thread_start_us":735136,"threads_started":1,"wal-append.queue_time_us":981193}
I20260709 13:03:58.002316  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.002950  8002 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:58.003417  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.003747  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.004037  1651 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:03:58.004443  1644 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 15: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:58.005074  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.005201  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.031888  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.053701  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.931283  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.054700  1570 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:58.055253  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.055613  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.056854  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.057421  1570 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:58.058173  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2775) took 2142 ms. Trace:
I20260709 13:03:58.058265  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.058607  1570 rpcz_store.cc:276] 0709 13:03:55.916202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.056687 (+2140485us) service_pool.cc:224] Handling call
0709 13:03:58.057031 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.057907 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.058109 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.059926  1570 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:57.935295  7914 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:57.842832  7866 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:03:57.945588  8216 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:03:57.775413  8112 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:58.007383  8225 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.976595  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2679) took 1258 ms. Trace:
I20260709 13:03:58.007926  8174 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.928884  7902 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:58.008499  8009 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.009820  1645 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:57.988907  1650 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:58.010686  1646 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:03:57.896221  7450 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:57.992121  1575 rpcz_store.cc:276] 0709 13:03:55.784020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.473957 (+689937us) service_pool.cc:224] Handling call
0709 13:03:56.501840 (+ 27883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.502991 (+  1151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.503057 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.509853 (+  6796us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.205153 (+695300us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:57.206277 (+  1124us) log.cc:844] Serialized 14 byte log entry
0709 13:03:57.207639 (+  1362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.207739 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.991570 (+783831us) raft_consensus.cc:1685] finished
0709 13:03:57.991589 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.991701 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1875,"prepare.run_cpu_time_us":219,"prepare.run_wall_time_us":221,"raft.queue_time_us":4514,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":3734,"spinlock_wait_cycles":27727104,"thread_start_us":622729,"threads_started":1,"wal-append.queue_time_us":1045826}
I20260709 13:03:57.996332  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.005095  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:57.988510  8164 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:58.012434  8204 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:57.988940  8170 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:58.014003  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.083s (SENT)
I20260709 13:03:58.165795  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:58.015949  1567 raft_consensus.cc:2368] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16.
I20260709 13:03:58.167035  1643 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:58.167524  1567 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:58.154618  1724 rpcz_store.cc:276] 0709 13:03:56.718346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.718524 (+   178us) service_pool.cc:224] Handling call
0709 13:03:57.976218 (+1257694us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400223154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:57.976400 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766707968}
I20260709 13:03:58.168371  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2780) took 2230 ms. Trace:
I20260709 13:03:58.168946  1567 rpcz_store.cc:276] 0709 13:03:55.938456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.166928 (+2228472us) service_pool.cc:224] Handling call
0709 13:03:58.167217 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.168049 (+   832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.168280 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:57.885105  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.018008  2371 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:03:57.988456  8224 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.021507  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.175938  1574 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:58.021989  1572 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:03:57.988451  1576 rpcz_store.cc:276] 0709 13:03:55.627077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.627377 (+   300us) service_pool.cc:224] Handling call
0709 13:03:55.639881 (+ 12504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.902852 (+262971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.910257 (+2007405us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54000cbf54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:57.910532 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589244672}
I20260709 13:03:58.177131  1572 pending_rounds.cc:85] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 13
W20260709 13:03:57.906808  8110 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.000s
I20260709 13:03:57.926198  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.029826  8221 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:58.029737  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 146140us
I20260709 13:03:58.177536  1572 pending_rounds.cc:107] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:03:58.178750  1572 raft_consensus.cc:2889] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:58.030521  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3113) took 1027 ms. Trace:
I20260709 13:03:58.038482  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3131) took 2391 ms. Trace:
W20260709 13:03:58.178746  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.179694  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3114) took 2553 ms. Trace:
I20260709 13:03:58.181166  1572 rpcz_store.cc:276] 0709 13:03:55.626678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.626821 (+   143us) service_pool.cc:224] Handling call
0709 13:03:55.668076 (+ 41255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.838577 (+2170501us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5800478dd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833cca0aa 00007fc833d4c447 00007fc833d64fe7 00007fc833d667de 00007fc833d70b49 00007fc833d71416 00007fc833d6dd8f 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4
0709 13:03:58.179385 (+340808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.179627 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561139456}
I20260709 13:03:58.040647  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.043198  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:58.184274  1717 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.998739  1642 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:58.044123  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.044888  1649 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:03:58.045766  2379 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:58.045933  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.046303  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.188771  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.044736  7971 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:58.006202  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:58.191134  1571 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 19 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:03:58.179989  1569 rpcz_store.cc:276] 0709 13:03:55.647590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.059145 (+411555us) service_pool.cc:224] Handling call
0709 13:03:56.088250 (+ 29105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:56.238829 (+150579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.238878 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:56.240470 (+  1592us) log.cc:844] Serialized 26 byte log entry
0709 13:03:56.853265 (+612795us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:56.951023 (+ 97758us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.951384 (+   361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.946140 (+994756us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54000cba54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:57.946192 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.035362 (+ 89170us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b1800151120. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:03:58.038283 (+  2921us) raft_consensus.cc:1685] finished
0709 13:03:58.038304 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.038424 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2218,"prepare.queue_time_us":101167,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":93,"raft.queue_time_us":7413,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":19569,"spinlock_wait_cycles":515157760,"thread_start_us":373166,"threads_started":1,"wal-append.queue_time_us":864122}
W20260709 13:03:58.051039  7909 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:03:58.006068  8151 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:58.196877  8151 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:58.197623  2378 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:58.198525  2378 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:58.199299  7909 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:58.201237  2372 raft_consensus.cc:2393] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 19: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:03:58.204360  7909 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:58.174575  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.060209  1644 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:57.957738  1718 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
W20260709 13:03:58.065925  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66851us
I20260709 13:03:58.221731  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:58.222033  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:58.075464  8195 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:03:58.112691  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:58.101606  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2779) took 2179 ms. Trace:
W20260709 13:03:58.063253  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.166184  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.232501  1570 rpcz_store.cc:276] 0709 13:03:55.921837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.059514 (+2137677us) service_pool.cc:224] Handling call
0709 13:03:58.059777 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.060233 (+   456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.101526 (+ 41293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.164072  8174 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:58.230289  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 216210us
I20260709 13:03:58.233664  1719 raft_consensus.cc:2393] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:03:58.230944  1576 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:03:58.237074  1576 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:58.237452  1576 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.170130  1573 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:03:58.155460  8112 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:58.244925  2376 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:58.179836  1716 rpcz_store.cc:276] 0709 13:03:57.002594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.002748 (+   154us) service_pool.cc:224] Handling call
0709 13:03:57.003121 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.037900 (+ 34779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.037936 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.045683 (+  7747us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.312639 (+266956us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:57.312685 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.312770 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.817734 (+504964us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:57.900961 (+ 83227us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4800240858. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:03:58.030231 (+129270us) raft_consensus.cc:1685] finished
0709 13:03:58.030262 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.030446 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34177,"raft.queue_time_us":44867,"raft.run_cpu_time_us":14107,"raft.run_wall_time_us":807664,"spinlock_wait_cycles":35453056,"thread_start_us":265247,"threads_started":1,"wal-append.queue_time_us":593924}
I20260709 13:03:58.095687  8066 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.263233  1647 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:03:58.275252  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.248s (SENT)
I20260709 13:03:58.275815  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:03:58.175734  8225 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.213604  8169 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.213173  8221 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.284065  8151 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:58.284965  8169 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:58.286168  8221 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:58.287081  8066 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:58.222854  7902 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:03:58.297611  8009 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:58.301977  7902 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.302697  7902 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.265336  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:58.212435  8224 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.304445  8164 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.305151  7902 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:58.305650  8164 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.309571  8002 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:58.310986  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45743.3us
W20260709 13:03:58.316088  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.319159  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.325968  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2674) took 1654 ms. Trace:
I20260709 13:03:58.327929  1720 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:58.331010  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:58.262738  8112 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.310322  8002 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:58.327072  1719 rpcz_store.cc:276] 0709 13:03:56.672864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.673075 (+   211us) service_pool.cc:224] Handling call
0709 13:03:58.325757 (+1652682us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003dccd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:58.325904 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3473750784}
I20260709 13:03:58.335588  1717 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
W20260709 13:03:58.346828  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 234099us
I20260709 13:03:58.350307  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:58.350962  1574 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.344677  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:03:58.337126  1576 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.353099  1575 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:58.357577  1576 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:58.358625  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44760 (request call id 2786) took 1564 ms. Trace:
I20260709 13:03:58.359012  1576 rpcz_store.cc:276] 0709 13:03:56.793737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.230774 (+1437037us) service_pool.cc:224] Handling call
0709 13:03:58.358447 (+127673us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.348698  8223 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:58.360774  8223 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.361155  8223 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.367161  8174 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.373502  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2784) took 1580 ms. Trace:
I20260709 13:03:58.373898  1575 rpcz_store.cc:276] 0709 13:03:56.793025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.229540 (+1436515us) service_pool.cc:224] Handling call
0709 13:03:58.281314 (+ 51774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.353696 (+ 72382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.373276 (+ 19580us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b54000276d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:58.373446 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164562816}
I20260709 13:03:58.374601  8174 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:58.375595  1575 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:58.313544  8195 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:58.379091  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3104) took 1624 ms. Trace:
I20260709 13:03:58.379469  1720 rpcz_store.cc:276] 0709 13:03:56.754824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.756969 (+  2145us) service_pool.cc:224] Handling call
0709 13:03:58.327761 (+1570792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.335300 (+  7539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.378781 (+ 43481us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54003dccd0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:58.378997 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3609889152}
I20260709 13:03:58.380482  8226 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:58.376169  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2793) took 1478 ms. Trace:
I20260709 13:03:58.346199  7902 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:58.381035  8226 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:58.381435  1575 rpcz_store.cc:276] 0709 13:03:56.898225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.374881 (+1476656us) service_pool.cc:224] Handling call
0709 13:03:58.375208 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.375958 (+   750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.376122 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.372658  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:58.382573  1571 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:58.382831  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 145927us
I20260709 13:03:58.372179  7882 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:58.384606  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:58.385789  1570 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.381670  8226 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.386583  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:58.396284  8009 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:58.397007  8009 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:58.397418  8009 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.401543  8106 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:58.401528  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.041s	sys 0.041s
W20260709 13:03:58.405568  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.044s	sys 0.041s
W20260709 13:03:58.406018  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 702389us
W20260709 13:03:58.403988  2373 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.008s	sys 0.007s
W20260709 13:03:58.409016  2373 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.007s
I20260709 13:03:58.412191  1571 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:03:58.443714  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43786.9us
I20260709 13:03:58.449311  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.459311  1720 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:58.349597  8002 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:58.401806  8118 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.002s	sys 0.023s
I20260709 13:03:58.488744  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:58.465926  2377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.006s	sys 0.026s
W20260709 13:03:58.514364  2377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.006s	sys 0.026s
I20260709 13:03:58.514930  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:03:58.491333  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.022s	sys 0.024s
W20260709 13:03:58.488621  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 43722.7us
I20260709 13:03:58.515717  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.391395  7882 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:03:58.517424  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:58.533582  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.023s	sys 0.030s
I20260709 13:03:58.536329  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.537761  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.539815  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.518400  1721 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:58.517560  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71971.3us
I20260709 13:03:58.541221  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.516773  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.542719  1725 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:03:58.532672  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.543371  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:58.551497  1574 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:03:58.553785  1574 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:58.554385  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2796) took 1557 ms. Trace:
I20260709 13:03:58.554700  1574 rpcz_store.cc:276] 0709 13:03:57.029853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.553166 (+1523313us) service_pool.cc:224] Handling call
0709 13:03:58.553460 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.554154 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.554332 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.557013  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:58.578008  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.575482  1724 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:03:58.579586  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.580168  1646 raft_consensus.cc:2376] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18. Re-sending same reply.
I20260709 13:03:58.582921  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:58.558580  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.577183  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:03:58.556941  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.585517  1717 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
W20260709 13:03:58.586428  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.588815  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.589020  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.589648  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:58.589780  1725 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:03:58.592185  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2797) took 1554 ms. Trace:
I20260709 13:03:58.592597  1574 rpcz_store.cc:276] 0709 13:03:57.037678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.588520 (+1550842us) service_pool.cc:224] Handling call
0709 13:03:58.591206 (+  2686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.591584 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.591642 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.591712 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.591775 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.592040 (+   265us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.592137 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.488615  8164 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.589907  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.583788  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:58.600485  2374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.011s	sys 0.024s
W20260709 13:03:58.601006  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55795.7us
W20260709 13:03:58.615756  2374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.011s	sys 0.024s
I20260709 13:03:58.616084  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:58.592641  8066 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:03:58.608532  8169 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:58.612737  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:58.489300  7971 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:03:58.643465  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 66486us
I20260709 13:03:58.643792  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:58.557689  8204 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.645853  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:58.646631  1647 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:58.646642  1724 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.618412  1716 raft_consensus.cc:2410] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:58.620296  1643 raft_consensus.cc:2393] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:03:58.620860  1570 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
W20260709 13:03:58.513844  8118 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.003s	sys 0.023s
W20260709 13:03:58.623821  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.666265  1642 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
W20260709 13:03:58.625017  2345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.051s	sys 0.086s
I20260709 13:03:58.667238  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:58.613358  1648 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:58.459164  8174 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 13:03:58.574985  8131 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:58.557129  8199 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:03:58.586236  7882 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.655025  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:58.660455  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.660377  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.661350  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.659888  7988 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:58.531864  7902 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.157s
I20260709 13:03:58.710600  8211 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:58.627657  8193 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:03:58.614045  8231 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.138s
I20260709 13:03:58.654486  7914 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:58.597873  8221 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:58.715600  8221 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:03:58.716619  1650 raft_consensus.cc:2368] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21.
I20260709 13:03:58.662876  7866 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:58.717427  7866 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:58.718314  7866 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.663445  2372 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:58.664513  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:58.667307  2345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.051s	sys 0.086s
I20260709 13:03:58.624523  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:58.557879  7850 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:58.643283  8157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.043s	sys 0.051s
I20260709 13:03:58.678246  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.679867  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:58.726601  2345 connection.cc:256] RPC call timeout handler was delayed by -1.0949434439999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:03:58.693357  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2800) took 1501 ms. Trace:
I20260709 13:03:58.623004  8146 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:58.727598  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 103178us
I20260709 13:03:58.728694  2373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 905) took 3735 ms. Trace:
I20260709 13:03:58.655213  8169 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:58.740718  1570 rpcz_store.cc:276] 0709 13:03:57.191710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.649451 (+1457741us) service_pool.cc:224] Handling call
0709 13:03:58.692549 (+ 43098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.692864 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.692903 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.692964 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:58.693023 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.693099 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.693274 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.733062  1648 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:03:58.759548  1642 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:58.631978  8164 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.720031  8225 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.657425  7971 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:58.672494  8110 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 13:03:58.761425  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.065219  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2785) took 2271 ms. Trace:
I20260709 13:03:59.066139  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.066690  1648 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:59.067737  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:58.732852  8157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.044s	sys 0.052s
I20260709 13:03:59.066841  1567 rpcz_store.cc:276] 0709 13:03:56.793408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.230292 (+1436884us) service_pool.cc:224] Handling call
0709 13:03:58.230627 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.231037 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.231065 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.231558 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.654638 (+423080us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.654684 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.654749 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.065031 (+410282us) raft_consensus.cc:1685] finished
0709 13:03:59.065058 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.065155 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2245,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":17816,"thread_start_us":342561,"threads_started":1,"wal-append.queue_time_us":814942}
W20260709 13:03:58.974087  1576 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.013s	sys 0.042s
I20260709 13:03:58.809307  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.070180  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2802) took 1783 ms. Trace:
W20260709 13:03:59.070279  1576 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.013s	sys 0.042s
I20260709 13:03:58.954212  8106 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.072767  8106 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.074044  8225 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:59.070827  1644 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 13: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:59.172603  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:58.792080  7909 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:59.174199  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.070674  1567 rpcz_store.cc:276] 0709 13:03:57.286585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.069376 (+1782791us) service_pool.cc:224] Handling call
0709 13:03:59.069656 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.069818 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.069852 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.069901 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:59.069961 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.070019 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.070134 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.728258  8068 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:59.075397  8106 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:58.842118  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 745) took 1242 ms. Trace:
I20260709 13:03:58.675220  8131 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.763582  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.809993  8169 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:03:58.732947  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:58.806922  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:58.872581  2371 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:59.086123  7971 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:59.087443  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 442388us
I20260709 13:03:59.197145  1690 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:59.088478  8250 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.090711  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 506697us
I20260709 13:03:58.758315  2373 rpcz_store.cc:276] 0709 13:03:54.993879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.038274 (+ 44395us) service_pool.cc:224] Handling call
0709 13:03:55.038787 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:55.039143 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:55.130334 (+ 91191us) log.cc:844] Serialized 14 byte log entry
0709 13:03:56.140043 (+1009709us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:03:57.801906 (+1661863us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b18001566a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf 00007fc833d88038
0709 13:03:58.459774 (+657868us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.727403 (+267629us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.727461 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:58.728111 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.728439 (+   328us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.728620 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2304553,"prepare.queue_time_us":1069799,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":21707,"raft.queue_time_us":2331536,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":52005,"spinlock_wait_cycles":22949376,"thread_start_us":996629,"threads_started":4,"wal-append.queue_time_us":109404,"wal-append.run_cpu_time_us":35975,"wal-append.run_wall_time_us":2295140}
I20260709 13:03:58.868290  8230 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:58.832083  8170 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.005s
I20260709 13:03:58.812110  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2801) took 1526 ms. Trace:
I20260709 13:03:58.710615  7988 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:58.946810  1724 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:03:58.790674  7902 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:03:58.974645  1575 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.019s	sys 0.026s
W20260709 13:03:59.148501  1645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.016s	sys 0.019s
W20260709 13:03:59.275815  1575 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.019s	sys 0.027s
W20260709 13:03:59.050446  1723 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.045s	sys 0.056s
W20260709 13:03:59.276714  1723 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.240s	user 0.045s	sys 0.056s
I20260709 13:03:59.069430  8157 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.157481  1574 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.008s	sys 0.006s
W20260709 13:03:59.277755  1574 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.009s	sys 0.006s
I20260709 13:03:59.278692  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2798) took 2193 ms. Trace:
I20260709 13:03:59.278396  1570 rpcz_store.cc:276] 0709 13:03:57.285964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.770617 (+1484653us) service_pool.cc:224] Handling call
0709 13:03:58.811431 (+ 40814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.811671 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.811708 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.811764 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:58.811818 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.811888 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.812036 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:58.663609  8204 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:03:59.183336  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2804) took 1896 ms (client timeout 2139 ms). Trace:
W20260709 13:03:59.187870  1649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.000s	sys 0.018s
I20260709 13:03:58.758443  7850 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.195211  2378 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:03:59.298748  1649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.000s	sys 0.018s
I20260709 13:03:58.869110  8223 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.304725  1647 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:59.188108  2377 rpcz_store.cc:276] 0709 13:03:57.599596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.681048 (+ 81452us) service_pool.cc:224] Handling call
0709 13:03:57.681409 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.769291 (+ 87882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.769325 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.797079 (+ 27754us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.514848 (+717769us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.514904 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.514994 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.841834 (+326840us) raft_consensus.cc:1685] finished
0709 13:03:58.841856 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.842018 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5695,"raft.queue_time_us":36409,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":4108,"spinlock_wait_cycles":1222528,"thread_start_us":725154,"threads_started":2,"wal-append.queue_time_us":981184}
W20260709 13:03:59.226310  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:59.309213  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.194052  8131 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:59.252771  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:59.248029  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:59.243372  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:03:59.314133  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 66181.7us
I20260709 13:03:59.314352  2371 raft_consensus.cc:2410] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:03:59.243571  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:59.279246  1574 rpcz_store.cc:276] 0709 13:03:57.085160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.602837 (+1517677us) service_pool.cc:224] Handling call
0709 13:03:58.604733 (+  1896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.278133 (+673400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.278163 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.278213 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.278254 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.278519 (+   265us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.278638 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":239628,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":187,"thread_start_us":671049,"threads_started":1}
I20260709 13:03:59.315035  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:59.308930  2342 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:03:59.292452  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.010s	sys 0.004s
I20260709 13:03:59.295193  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:59.319913  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.011s	sys 0.004s
W20260709 13:03:59.298664  1568 rpcz_store.cc:269] 0709 13:03:57.287389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.722705 (+435316us) service_pool.cc:224] Handling call
0709 13:03:59.183028 (+1460323us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001aa9d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:59.183258 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3050259712}
I20260709 13:03:59.203826  1567 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:59.322170  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2809) took 1322 ms. Trace:
I20260709 13:03:59.322602  1567 rpcz_store.cc:276] 0709 13:03:57.999408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.191126 (+1191718us) service_pool.cc:224] Handling call
0709 13:03:59.203269 (+ 12143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.321854 (+118585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.322101 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:59.192617  7909 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:59.321040  1570 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:59.209335  8112 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.000s
I20260709 13:03:59.325942  2373 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 14: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
W20260709 13:03:59.320408  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 736423us
I20260709 13:03:59.329188  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:59.312249  2372 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:03:59.331465  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:03:59.216562  8225 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:59.343003  1651 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:03:59.346295  1648 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:59.331236  7450 raft_consensus.cc:2804] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:59.337248  8112 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.349186  8249 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.349906  8249 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.350677  8112 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:59.342664  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:59.351854  8112 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:59.351953  7971 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:59.352612  7971 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.352875  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:59.353138  7971 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.354101  8199 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:59.355226  8199 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.356082  8199 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:59.356994  8199 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:03:59.370867  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:59.275658  7902 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:03:59.357237  8238 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.401s	user 0.000s	sys 0.002s
W20260709 13:03:59.375898  1688 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.006s	sys 0.015s
W20260709 13:03:59.383927  1688 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.006s	sys 0.015s
W20260709 13:03:59.384383  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 687855us
I20260709 13:03:59.301842  8170 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.194501  8068 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:59.378465  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.386381  1645 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:03:59.380105  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:03:59.387140  1645 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:03:59.295925  8157 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.352620  8112 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:03:59.398236  1614 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:03:59.398319  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:59.398909  1614 leader_election.cc:403] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:59.399191  1567 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:59.224485  8230 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:03:59.242992  8250 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.365005  8249 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:03:59.378188  8224 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.073s	user 0.004s	sys 0.009s
I20260709 13:03:59.405027  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34482 (request call id 989) took 1344 ms. Trace:
I20260709 13:03:59.405524  1650 rpcz_store.cc:276] 0709 13:03:58.060788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.166901 (+106113us) service_pool.cc:224] Handling call
0709 13:03:59.404832 (+1237931us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540005c3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:59.404973 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135097088}
W20260709 13:03:59.407220  1573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.018s	sys 0.022s
W20260709 13:03:59.407682  1573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.018s	sys 0.022s
I20260709 13:03:59.412770  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:59.414551  1720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.011s	sys 0.029s
W20260709 13:03:59.415096  1720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.012s	sys 0.029s
W20260709 13:03:59.416376  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:59.424049  1572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.005s	sys 0.037s
W20260709 13:03:59.424517  1572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.005s	sys 0.037s
I20260709 13:03:59.424847  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:03:59.425319  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2790) took 2587 ms. Trace:
I20260709 13:03:59.425490  1724 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.425714  1572 rpcz_store.cc:276] 0709 13:03:56.837982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.238743 (+1400761us) service_pool.cc:224] Handling call
0709 13:03:58.239171 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.239402 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:58.622207 (+382805us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.424963 (+802756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.425019 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:59.425064 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.425132 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.425269 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41056,"prepare.queue_time_us":901641,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":1106320,"threads_started":2}
I20260709 13:03:59.427515  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:59.428087  1572 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:59.430059  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:03:59.444788  2377 raft_consensus.cc:2410] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:59.403406  8112 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:59.377295  8235 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:03:59.402170  8253 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:59.472460  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.472883  1567 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:03:59.422201  8254 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:59.476305  2380 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.000s
I20260709 13:03:59.457021  1724 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:03:59.405138  8230 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:59.502976  1645 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:03:59.515599  2380 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:03:59.446555  8110 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.449473  8249 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.529210  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 747) took 1828 ms. Trace:
I20260709 13:03:59.533629  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:59.536856  1567 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
W20260709 13:03:59.499428  8118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.012s	sys 0.037s
I20260709 13:03:59.450429  7988 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.542320  8157 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:59.543476  8252 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.540680  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:03:59.544777  1567 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:03:59.558077  1572 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:03:59.382551  8169 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.422258  8224 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.575748  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:03:59.535488  8009 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.133s	user 0.012s	sys 0.002s
I20260709 13:03:59.534247  2374 rpcz_store.cc:276] 0709 13:03:57.700584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.752511 (+ 51927us) service_pool.cc:224] Handling call
0709 13:03:57.796662 (+ 44151us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.878881 (+ 82219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.878930 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.879671 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.616510 (+736839us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:58.616594 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.616690 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.528921 (+912231us) raft_consensus.cc:1685] finished
0709 13:03:59.528940 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.529129 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10966,"raft.queue_time_us":86036,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":13490,"thread_start_us":736234,"threads_started":1,"wal-append.queue_time_us":849798}
W20260709 13:03:59.584438  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 347474us
I20260709 13:03:59.521782  8254 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:03:59.657560  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 337214us
W20260709 13:03:59.548650  8118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.013s	sys 0.038s
I20260709 13:03:59.653123  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 923) took 1617 ms. Trace:
I20260709 13:03:59.529999  8164 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:59.671715  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.442876  7909 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:59.675184  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:03:59.676249  2380 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:03:59.684870  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3143) took 1102 ms. Trace:
I20260709 13:03:59.685379  1720 rpcz_store.cc:276] 0709 13:03:58.582315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.584813 (+  2498us) service_pool.cc:224] Handling call
0709 13:03:58.585225 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:58.613996 (+ 28771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.614041 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:58.632308 (+ 18267us) log.cc:844] Serialized 52 byte log entry
0709 13:03:59.415572 (+783264us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.415610 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.415689 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.684567 (+268878us) raft_consensus.cc:1685] finished
0709 13:03:59.684589 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.684791 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28543,"raft.queue_time_us":122,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":690772,"threads_started":1,"wal-append.queue_time_us":1006281}
I20260709 13:03:59.475399  8199 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.535028  8110 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.712818  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:59.663094  8254 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:59.717895  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 402823us
I20260709 13:03:59.714037  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2794) took 2720 ms. Trace:
I20260709 13:03:59.526122  7902 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.696151  2380 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:03:59.562687  8249 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.658391  8009 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.702353  7450 raft_consensus.cc:697] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:03:59.764528  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:03:59.807693  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:03:59.688414  8259 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.659629  1723 rpcz_store.cc:276] 0709 13:03:58.035618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.035778 (+   160us) service_pool.cc:224] Handling call
0709 13:03:58.036109 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.036320 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:59.277046 (+1240726us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.652078 (+375032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.652441 (+   363us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.652519 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.652640 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.653009 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31512,"prepare.queue_time_us":688339,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"thread_start_us":1583228,"threads_started":2}
I20260709 13:03:59.761497  1575 rpcz_store.cc:276] 0709 13:03:56.995127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.384773 (+1389646us) service_pool.cc:224] Handling call
0709 13:03:58.385028 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.385228 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:59.276551 (+891323us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.712426 (+435875us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.713585 (+  1159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.713649 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:59.713712 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.713797 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.713969 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51849,"prepare.queue_time_us":806040,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":1309168,"threads_started":2}
I20260709 13:03:59.814126  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:03:59.797912  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2792) took 2928 ms. Trace:
I20260709 13:03:59.815476  1576 rpcz_store.cc:276] 0709 13:03:56.869354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.360524 (+1491170us) service_pool.cc:224] Handling call
0709 13:03:58.360907 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.361179 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:59.072644 (+711465us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.792259 (+719615us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.792386 (+   127us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:59.797107 (+  4721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.797375 (+   268us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.797793 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47643,"prepare.queue_time_us":810259,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":140593,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":34278,"thread_start_us":1410386,"threads_started":3}
I20260709 13:03:59.784037  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2817) took 1460 ms. Trace:
I20260709 13:03:59.815052  1645 raft_consensus.cc:2376] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16. Re-sending same reply.
I20260709 13:03:59.821301  1568 rpcz_store.cc:276] 0709 13:03:58.323276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.322189 (+998913us) service_pool.cc:224] Handling call
0709 13:03:59.322546 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.322999 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:59.384143 (+ 61144us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.783510 (+399367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.783602 (+    92us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.783658 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.783758 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.783943 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55422,"prepare.queue_time_us":73752,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":2204,"thread_start_us":398724,"threads_started":1}
I20260709 13:03:59.827919  1568 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:59.832398  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2392) took 1292 ms. Trace:
I20260709 13:03:59.833148  1651 rpcz_store.cc:276] 0709 13:03:58.539477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.539581 (+   104us) service_pool.cc:224] Handling call
0709 13:03:59.831862 (+1292281us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400411c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:03:59.832240 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844119296}
I20260709 13:03:59.690837  8164 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.115s
W20260709 13:03:59.830999  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 382901us
I20260709 13:03:59.853160  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:59.853420  8066 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.772750  8225 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:03:59.881228  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2399) took 1303 ms. Trace:
W20260709 13:03:59.881639  8267 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:59.881319  8225 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.881688  1646 rpcz_store.cc:276] 0709 13:03:58.577402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.582827 (+  5425us) service_pool.cc:224] Handling call
0709 13:03:58.583151 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:58.974763 (+391612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.974795 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.068942 (+ 94147us) log.cc:844] Serialized 26 byte log entry
0709 13:03:59.462049 (+393107us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:59.462084 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.462152 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.880924 (+418772us) raft_consensus.cc:1685] finished
0709 13:03:59.880960 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.881140 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41699,"raft.queue_time_us":126880,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":742135,"threads_started":2,"wal-append.queue_time_us":738003}
I20260709 13:03:59.885078  8249 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:59.855208  1721 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:59.886538  8249 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:59.887056  8249 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.887493  8249 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.865391  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:03:59.889256  1576 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.894394  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3137) took 1434 ms. Trace:
I20260709 13:03:59.789929  7850 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:59.773201  8256 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.896873  1721 rpcz_store.cc:276] 0709 13:03:58.459915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.584896 (+124981us) service_pool.cc:224] Handling call
0709 13:03:58.639754 (+ 54858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.886067 (+1246313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.893991 (+  7924us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400223154. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:03:59.894300 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2791829248}
I20260709 13:03:59.870852  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.819123  8199 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:59.878938  8260 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:59.880060  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:59.879972  8066 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:03:59.789327  8204 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:03:59.847301  7902 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:59.904285  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.905045  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2788) took 3096 ms. Trace:
I20260709 13:03:59.907232  1576 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:03:59.911263  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:03:59.911914  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.000s	sys 0.026s
I20260709 13:03:59.858758  8170 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:03:59.913834  2371 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:59.915203  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 923) took 1236 ms. Trace:
I20260709 13:03:59.918648  1573 rpcz_store.cc:276] 0709 13:03:56.808614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.238196 (+1429582us) service_pool.cc:224] Handling call
0709 13:03:58.238454 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.238676 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:58.628566 (+389890us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.853892 (+1225326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.904641 (+ 50749us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:59.904773 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.904852 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.904986 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":826663,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":212223,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":12057,"thread_start_us":1572831,"threads_started":3,"wal-append.queue_time_us":1043973}
W20260709 13:03:59.920737  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.000s	sys 0.027s
W20260709 13:03:59.921057  1570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.018s	sys 0.010s
W20260709 13:03:59.926577  1570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.019s	sys 0.010s
I20260709 13:03:59.926940  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2791) took 3063 ms. Trace:
I20260709 13:03:59.927294  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2810) took 1927 ms. Trace:
I20260709 13:03:59.927889  1690 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:03:59.929075  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:03:59.929420  1570 rpcz_store.cc:276] 0709 13:03:57.999678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.305740 (+1306062us) service_pool.cc:224] Handling call
0709 13:03:59.320893 (+ 15153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.926990 (+606097us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:59.927019 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.927057 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:59.927088 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.927146 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.927244 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135488,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":40573,"thread_start_us":545023,"threads_started":1}
I20260709 13:03:59.927353  1569 rpcz_store.cc:276] 0709 13:03:56.863768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.246181 (+1382413us) service_pool.cc:224] Handling call
0709 13:03:58.322842 (+ 76661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.323057 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:58.729657 (+406600us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.926609 (+1196952us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.926661 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:59.926705 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.926773 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.926890 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":835481,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":1557049,"threads_started":2}
W20260709 13:03:59.911417  8226 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.452s	user 0.015s	sys 0.000s
W20260709 13:03:59.940049  1574 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.013s	sys 0.017s
W20260709 13:03:59.943996  1574 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.013s	sys 0.017s
I20260709 13:03:59.885145  1472 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:03:59.924176  2375 rpcz_store.cc:276] 0709 13:03:58.678854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.679002 (+   148us) service_pool.cc:224] Handling call
0709 13:03:58.707110 (+ 28108us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:58.727608 (+ 20498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.727654 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:59.104698 (+377044us) log.cc:844] Serialized 78 byte log entry
0709 13:03:59.364600 (+259902us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:59.852673 (+488073us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.853361 (+   688us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.853580 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.853671 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.913370 (+ 59699us) raft_consensus.cc:1685] finished
0709 13:03:59.913398 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.915115 (+  1717us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19926,"raft.queue_time_us":42520,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":83312,"thread_start_us":692851,"threads_started":2,"wal-append.queue_time_us":522754}
I20260709 13:03:59.946745  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2811) took 1946 ms. Trace:
I20260709 13:03:59.947286  1574 rpcz_store.cc:276] 0709 13:03:57.999922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.315795 (+1315873us) service_pool.cc:224] Handling call
0709 13:03:59.316114 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.316425 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:59.357690 (+ 41265us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.946255 (+588565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.946343 (+    88us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:59.946417 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.946507 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.946672 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85678,"prepare.queue_time_us":49926,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":587155,"threads_started":1}
W20260709 13:03:59.951340  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.641s (SENT) [suppressed 36 similar messages]
W20260709 13:03:59.960022  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.712s (SENT)
I20260709 13:03:59.965592  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:03:59.936578  7882 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.164s	user 0.012s	sys 0.002s
I20260709 13:03:59.831661  8259 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:59.913455  8256 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:00.094393  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.388326  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.006s	sys 0.031s
W20260709 13:04:00.418824  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 458710us
W20260709 13:04:00.419847  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.007s	sys 0.033s
W20260709 13:04:00.295904  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.014s	sys 0.013s
I20260709 13:03:59.974818  1571 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2795) took 2978 ms. Trace:
I20260709 13:04:00.061542  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:00.381721  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:00.031148  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 997) took 1411 ms. Trace:
W20260709 13:04:00.421136  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.028s	sys 0.014s
W20260709 13:04:00.422636  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 591009us
W20260709 13:04:00.424012  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 1.00467e+06us
I20260709 13:03:59.927464  8106 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.984907  2342 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.172s (SENT)
I20260709 13:04:00.129621  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:00.462327  8225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.016s	sys 0.039s
I20260709 13:04:00.422134  1571 rpcz_store.cc:276] 0709 13:03:56.996383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.440701 (+1444318us) service_pool.cc:224] Handling call
0709 13:03:58.512532 (+ 71831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.513262 (+   730us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:59.107435 (+594173us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.974267 (+866832us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:59.974356 (+    89us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.974440 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.974541 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.974729 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38684,"prepare.queue_time_us":678767,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":109,"raft.queue_time_us":261243,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":12850,"thread_start_us":1307628,"threads_started":3,"wal-append.queue_time_us":759762}
I20260709 13:04:00.422781  1648 raft_consensus.cc:2376] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20. Re-sending same reply.
I20260709 13:04:00.423269  1642 rpcz_store.cc:276] 0709 13:03:58.619618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.067670 (+448052us) service_pool.cc:224] Handling call
0709 13:03:59.067970 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.068273 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:59.528856 (+460583us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.030689 (+501833us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.030762 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:00.030824 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.030915 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.031076 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30932,"prepare.queue_time_us":372976,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":45631,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":92732,"thread_start_us":783210,"threads_started":2}
W20260709 13:04:00.432623  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 504541us
I20260709 13:04:00.427292  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:00.058537  8268 log.cc:927] Time spent T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.132s	user 0.001s	sys 0.001s
I20260709 13:04:00.247821  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:00.247411  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:03:59.982004  8002 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.448s	user 0.015s	sys 0.000s
I20260709 13:04:00.371582  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.223224  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 702) took 1402 ms. Trace:
I20260709 13:04:00.246865  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:00.473862  1573 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:00.325634  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:00.185717  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.475031  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:00.062832  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.461874  2374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.000s	sys 0.009s
I20260709 13:04:00.301693  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:00.477166  1571 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:04:00.423252  1570 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:00.142510  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 700) took 1403 ms. Trace:
W20260709 13:04:00.463102  8225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.016s	sys 0.039s
I20260709 13:04:00.058892  1472 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 12 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:00.357560  1569 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
W20260709 13:04:00.003351  8267 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:04:00.200935  7866 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:00.082302  7850 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.215932  7914 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:00.006929  8131 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.001s
W20260709 13:04:00.217577  8265 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:04:00.422276  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:00.486896  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 59656.7us
W20260709 13:04:00.513249  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.775s (SENT)
I20260709 13:03:59.941530  8146 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:00.240659  8224 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:00.241433  8009 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:00.444972  8211 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.163s
W20260709 13:04:00.423884  7988 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.008s	sys 0.011s
I20260709 13:04:00.154906  8285 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.492s
I20260709 13:04:00.563458  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:00.590979  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.591351  2374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.000s	sys 0.017s
I20260709 13:04:00.591593  2377 rpcz_store.cc:276] 0709 13:03:58.738967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.328847 (+589880us) service_pool.cc:224] Handling call
0709 13:04:00.123356 (+794509us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400573ad4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:00.142411 (+ 19055us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263011712}
I20260709 13:04:00.593097  1647 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:00.003901  8277 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.594745  1574 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:00.681669  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:00.682858  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.683593  1574 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:03:59.973244  8157 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.214557  8273 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.205315  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 209) took 54106 ms (client timeout 15000 ms). Trace:
I20260709 13:04:00.208880  8280 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:00.489565  8106 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:00.008206  7450 consensus_queue.cc:237] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:03:59.995056  8178 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:59.944252  8250 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:00.030620  8274 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.971611  8226 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.234498  8223 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:00.482317  7971 consensus_meta.cc:287] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.127s	user 0.016s	sys 0.003s
W20260709 13:04:00.470533  8231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.020s	sys 0.035s
I20260709 13:04:00.607956  1716 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:00.219097  8066 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:00.488080  8249 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.600s	user 0.005s	sys 0.009s
I20260709 13:04:00.700671  8193 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.408179  8252 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.864s	user 0.013s	sys 0.004s
I20260709 13:04:00.007355  8195 consensus_queue.cc:1059] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.565s
I20260709 13:04:00.238188  8068 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:00.615728  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:00.449088  7882 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.426287  8281 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:00.781391  8253 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:00.034708  8170 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:00.614053  1472 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 13 to 14, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 14 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.639179  1570 raft_consensus.cc:2410] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:04:00.607883  8229 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:00.684741  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.847713  1574 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:00.603120  8290 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:04:00.687744  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 124341us
W20260709 13:04:00.597667  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 612480us
I20260709 13:04:00.692121  8273 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.604574  7914 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:00.674288  8112 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.156s	user 0.013s	sys 0.004s
I20260709 13:04:00.589848  2371 rpcz_store.cc:276] 0709 13:03:58.820765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.331105 (+510340us) service_pool.cc:224] Handling call
0709 13:03:59.913593 (+582488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.924033 (+ 10440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.222907 (+298874us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b540064f350. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:00.223129 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1338123520}
W20260709 13:04:00.728318  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.724784  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.955610  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:00.955942  8303 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:04:00.956610  1571 raft_consensus.cc:2368] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:04:00.957944  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.959167  1571 raft_consensus.cc:2410] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:00.959923  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 924) took 1290 ms. Trace:
I20260709 13:04:00.627998  7866 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:00.965483  8106 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:00.966776  8296 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
I20260709 13:04:00.967116  8259 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:00.748003  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.751521  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.692531  8280 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.969090  8280 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:00.965056  2374 rpcz_store.cc:276] 0709 13:03:59.669943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.670126 (+   183us) service_pool.cc:224] Handling call
0709 13:03:59.711080 (+ 40954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.711757 (+   677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.711804 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.718252 (+  6448us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.675822 (+957570us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:00.675866 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.675937 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.959713 (+283776us) raft_consensus.cc:1685] finished
0709 13:04:00.959736 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.959868 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":42297,"thread_start_us":856243,"threads_started":1,"wal-append.queue_time_us":1077674}
I20260709 13:04:00.866734  8068 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:00.970642  8068 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.606729  8002 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.974663  8002 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:00.216744  8110 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:00.974918  8065 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:00.975613  1643 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 21: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:00.944820  8211 raft_consensus.cc:3055] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:00.976908  8211 raft_consensus.cc:740] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:00.626968  7850 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:00.977952  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2405) took 2337 ms (client timeout 2102 ms). Trace:
W20260709 13:04:00.978374  1643 rpcz_store.cc:269] 0709 13:03:58.640630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.653093 (+ 12463us) service_pool.cc:224] Handling call
0709 13:04:00.977672 (+2324579us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5400306e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:00.977827 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5167677696}
W20260709 13:04:00.765317  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 154387us
I20260709 13:04:00.979187  7882 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:00.764143  8302 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.789160  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 366776us
W20260709 13:04:00.763630  8231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.021s	sys 0.036s
I20260709 13:04:00.785542  2379 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:00.647169  8146 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:00.812680  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 196953us
I20260709 13:04:00.748708  8178 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:00.614137  8308 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.190s
I20260709 13:04:00.616818  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:00.831046  1647 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:00.693012  8277 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:00.808593  8252 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:04:00.860918  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.067962  1574 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 13:04:00.720525  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 218995us
W20260709 13:04:01.069231  1688 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.869s (SENT)
I20260709 13:04:00.864442  1472 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 13 to 14, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 14 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.866139  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3144) took 1454 ms. Trace:
I20260709 13:04:01.069754  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.070120  1716 rpcz_store.cc:276] 0709 13:03:59.411278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.411484 (+   206us) service_pool.cc:224] Handling call
0709 13:03:59.459673 (+ 48189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:00.763770 (+1304097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.865785 (+102015us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400222c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:00.866034 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2639036672}
I20260709 13:04:00.912958  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:00.802696  8193 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.839259  8169 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:00.784803  8298 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.166s
I20260709 13:04:00.946096   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:00.846849  8170 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.781595  7971 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.786314  8249 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.867645  1646 raft_consensus.cc:1240] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:00.960955  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:00.719600  8248 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:00.966454  7866 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:00.718784  7988 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.009s	sys 0.015s
I20260709 13:04:00.977402  8226 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:00.629098  8131 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.093289  8278 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:01.128744  8248 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:01.131372  8131 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:00.980183  7850 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:00.988097  8302 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.756891  8274 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:00.994748  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:00.997153  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:01.026280  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.990890  8300 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.505s	user 0.001s	sys 0.001s
I20260709 13:04:00.975665  8065 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:01.041975  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:01.050475  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 536967us
W20260709 13:04:01.152114  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.812s (SENT)
I20260709 13:04:01.152077  1643 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:01.153038  2344 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:00.999444  8294 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.382s	user 0.000s	sys 0.002s
W20260709 13:04:01.037405  8279 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.353s	user 0.001s	sys 0.001s
I20260709 13:04:01.078779  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:01.089164  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 226421us
W20260709 13:04:01.091022  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 137425us
W20260709 13:04:01.118065  1688 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.134s (SENT)
I20260709 13:04:01.093024  1472 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:01.104458  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.001127  7882 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:04:01.119581  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.121459  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.198819  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.199699  1642 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:01.206995  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.207909  1642 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:01.141171  8297 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.070894  8151 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:01.082863  8306 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:04:00.981341  1648 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:01.216782  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:01.087863  8199 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.175s	user 0.011s	sys 0.004s
W20260709 13:04:01.136049  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.137216  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.090683  8259 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.125355  8254 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.350s	user 0.009s	sys 0.010s
I20260709 13:04:01.094384  8170 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.088732  8169 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:01.150290  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 153137us
W20260709 13:04:01.148512  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 153792us
W20260709 13:04:01.157891  2344 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.979s (SENT)
I20260709 13:04:01.140322  8195 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:04:01.147586  8204 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:04:01.125833  8211 consensus_queue.cc:260] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.132272  8002 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:04:01.236070  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2830) took 1384 ms. Trace:
I20260709 13:04:01.245954  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:01.231961  1472 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 12 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 13 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.229686  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2821) took 1849 ms. Trace:
I20260709 13:04:01.246024  2344 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:00.996049  8112 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.216075  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.220894  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:01.162592  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.222014  1574 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
W20260709 13:04:01.257717  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139632us
I20260709 13:04:01.258576  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:01.258931  1688 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.155s (SENT)
I20260709 13:04:01.259789  1574 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:01.261035  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.262540  1574 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:01.247902  1567 rpcz_store.cc:276] 0709 13:03:59.380263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.546119 (+165856us) service_pool.cc:224] Handling call
0709 13:03:59.546590 (+   471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:59.547015 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.547047 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:59.547792 (+   745us) log.cc:844] Serialized 52 byte log entry
0709 13:03:59.924487 (+376695us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.924529 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.924593 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.229480 (+1304887us) raft_consensus.cc:1685] finished
0709 13:04:01.229498 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.229629 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145361,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6571,"thread_start_us":376234,"threads_started":1,"wal-append.queue_time_us":901218}
I20260709 13:04:01.246043  1568 rpcz_store.cc:276] 0709 13:03:59.852161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.874350 (+ 22189us) service_pool.cc:224] Handling call
0709 13:03:59.874841 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.875114 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.875149 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.912687 (+ 37538us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.430659 (+517972us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:00.430699 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.430756 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.235663 (+804907us) raft_consensus.cc:1685] finished
0709 13:04:01.235699 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.235966 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":432953,"threads_started":1,"wal-append.queue_time_us":843690}
I20260709 13:04:01.259649  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.158703  8285 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.267714  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:01.227283  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90153.7us
I20260709 13:04:01.268064  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.269176  1642 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:01.268937  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:01.270243  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 112318us
I20260709 13:04:01.271318  1720 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:04:01.271777  1720 pending_rounds.cc:85] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 7
I20260709 13:04:01.248878  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.272276  1720 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:04:01.272781  1720 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:01.272814  1571 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:01.273244  1720 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:04:01.273761  1720 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:01.274649  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.275681  1718 raft_consensus.cc:2393] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:01.276895  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:01.262430  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:01.166080  7902 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.235s	user 0.011s	sys 0.006s
I20260709 13:04:01.151000  8256 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:01.275481  1472 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 9 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:01.289830  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.196741  8226 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.219997  8318 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:01.301433  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.278947  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.234289  8223 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.241392  8193 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:01.230638  8065 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.268028  8110 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.154673  7850 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:01.287523  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2825) took 1898 ms. Trace:
I20260709 13:04:01.355369  1575 rpcz_store.cc:276] 0709 13:03:59.389604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.810702 (+421098us) service_pool.cc:224] Handling call
0709 13:03:59.811360 (+   658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.811757 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:59.811780 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.812268 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.179893 (+367625us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.179961 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.430744 (+250783us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.287291 (+856547us) raft_consensus.cc:1685] finished
0709 13:04:01.287311 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.287461 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101510,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":16921,"thread_start_us":363774,"threads_started":1,"wal-append.queue_time_us":593094}
I20260709 13:04:01.288321  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.358374  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.359676  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.290782  1717 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:01.360999  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.362116  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.265312  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.289067  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:01.289019  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.363436  1643 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.364563  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:01.232782  8309 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:04:01.230238  8249 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:01.242026  8273 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:01.302690  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:01.370034  1614 connection.cc:256] RPC call timeout handler was delayed by -1.176692050999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
I20260709 13:04:01.307790  8199 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:01.280220  1571 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.304896  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.377197  1574 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:04:01.378314  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.232038  7971 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:01.285185  7988 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:01.306600  8310 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:04:01.236588  8146 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:01.255434  8169 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.308058  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.308211  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.404156  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.404498  1570 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:01.404958  1716 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:01.309252  1576 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:01.309063  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.234613  8068 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:01.310041  1472 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 8 to 13, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 13 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.409626  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:01.409888  1717 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.410779  1647 raft_consensus.cc:2368] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21.
W20260709 13:04:01.312500  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43660.5us
I20260709 13:04:01.411305  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:01.412037  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.240887  8316 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:01.278023  1723 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:01.291941  8229 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.312150  8318 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.316941  8313 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:01.317418  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:01.318758  2379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 919) took 2687 ms (client timeout 1854 ms). Trace:
I20260709 13:04:01.230957  8259 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.320760  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.322178  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2400) took 2739 ms. Trace:
I20260709 13:04:01.154522  8178 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.326457  8274 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:01.322204  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:01.511446  1567 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.328420  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:01.329643  7909 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.319878  8112 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:01.269999  8254 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.277668  7450 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:01.233987  8312 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:04:01.289047  8253 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:01.462896  8313 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.446310  8314 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:04:01.303725  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.371012  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:01.371613  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.537629  1646 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:01.538306  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:01.538713  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.378863  1574 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:04:01.389761  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:01.228263  8297 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.306625  8277 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:01.385186  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:01.397627  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:01.387742  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.542744  2376 raft_consensus.cc:2410] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
W20260709 13:04:01.281570  1688 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 2.225s (SENT)
I20260709 13:04:01.452876  7971 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:04:01.543982  1644 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:01.388808  7850 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:01.375640  7882 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:01.333202  8223 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.459050  1643 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:01.585418  1717 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:01.585517  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.324250  8280 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:01.463369  2379 rpcz_store.cc:269] 0709 13:03:58.631035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.631572 (+   537us) service_pool.cc:224] Handling call
0709 13:04:01.318470 (+2686898us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54004ae054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:01.318630 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4771073920}
I20260709 13:04:01.589599  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.464850  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:01.590373  1643 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:01.481843  1570 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:04:01.592456  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3151) took 1114 ms. Trace:
I20260709 13:04:01.592967  1570 rpcz_store.cc:276] 0709 13:04:00.477925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.463958 (+986033us) service_pool.cc:224] Handling call
0709 13:04:01.481297 (+ 17339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.592013 (+110716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.592367 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.535271  8313 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.495316  1571 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
W20260709 13:04:01.514153  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 185827us
W20260709 13:04:01.516860  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 203249us
I20260709 13:04:01.467541  1649 rpcz_store.cc:276] 0709 13:03:58.582303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.582737 (+   434us) service_pool.cc:224] Handling call
0709 13:03:58.602708 (+ 19971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.299573 (+696865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.299602 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.321210 (+ 21608us) log.cc:844] Serialized 26 byte log entry
0709 13:03:59.759203 (+437993us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:59.811694 (+ 52491us) log.cc:844] Serialized 14 byte log entry
0709 13:03:59.811969 (+   275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:59.812081 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.321880 (+1509799us) raft_consensus.cc:1685] finished
0709 13:04:01.321905 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.322079 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22579,"prepare.queue_time_us":51866,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":146034,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":14051,"thread_start_us":1057653,"threads_started":2,"wal-append.queue_time_us":1124541}
I20260709 13:04:01.448881  8065 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.520180  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:01.524263  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:01.556680  7971 consensus_queue.cc:1243] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:01.463074  8221 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.368039  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:01.530267  1568 raft_consensus.cc:1240] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:01.608716  1645 raft_consensus.cc:2410] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:01.611320  1647 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:01.131743  8248 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.534188  1569 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:01.372934  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.541930  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:01.543198  1721 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.543494  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:01.415714  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134558us
I20260709 13:04:01.614665  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.612577  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.615603  1718 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
I20260709 13:04:01.542985  2374 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:01.616230  1643 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:01.449769  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.547029  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:01.555241  8297 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:01.556876  1574 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:01.624454  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.525101  7909 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.559883  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:04:01.625048  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.561164  8304 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.394s	user 0.000s	sys 0.002s
W20260709 13:04:01.562873  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.563681  8321 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:04:01.565222  8302 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:01.566850  8230 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 2.010s	user 0.013s	sys 0.007s
I20260709 13:04:01.530076  8253 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:01.455540  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65541.8us
W20260709 13:04:01.709380  1614 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Timed out: RequestConsensusVote RPC to 127.0.107.196:45309 timed out after 1.854s (SENT)
I20260709 13:04:01.381109  8281 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:01.588493  1651 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.002s
I20260709 13:04:01.556633  8329 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:01.591190  1575 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:01.463181  1719 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:01.483098  1650 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:01.560228  8259 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:01.602838  1716 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.528052  1472 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:01.605258  1570 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:01.612494  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3152) took 1125 ms. Trace:
I20260709 13:04:01.725281  1569 rpcz_store.cc:276] 0709 13:04:00.486832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.508563 (+1021731us) service_pool.cc:224] Handling call
0709 13:04:01.533957 (+ 25394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.612160 (+ 78203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.612428 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:01.524374  7450 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.616305  2342 leader_election.cc:400] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21.
I20260709 13:04:01.732801  2342 leader_election.cc:403] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:04:01.617641  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:01.570593  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.553s (SENT)
W20260709 13:04:01.734493  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 163867us
W20260709 13:04:01.737392  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.673s (SENT)
I20260709 13:04:01.737906  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:01.617344  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 167671us
I20260709 13:04:01.725066  1472 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:01.739319  1725 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:01.739689  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.569001  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 927) took 1786 ms. Trace:
I20260709 13:04:01.740624  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.625526  2376 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.626382  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.627959  1642 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:01.631561  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3963795480000272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:04:01.637741  2372 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
W20260709 13:04:01.743036  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3855795480000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 13:04:01.662489  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.743489  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3756795479999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:04:01.743772  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.750s (SENT)
I20260709 13:04:01.682749  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:01.691187  1567 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:04:01.696882  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.745080  1647 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:01.603863  8199 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:01.748634  8199 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:01.706334  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 424565us
I20260709 13:04:01.753017  8288 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.753538  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:01.586093  7882 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.716315  1651 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:01.714035  2378 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 13.
I20260709 13:04:01.623445  7971 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:01.717871  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.744443  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:01.743871  1539 connection.cc:256] RPC call timeout handler was delayed by -2.35677954800002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:04:01.779877  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3486795480000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:04:01.780196  1539 connection.cc:256] RPC call timeout handler was delayed by -2.290179548000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
I20260709 13:04:01.746256  1570 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:04:01.780459  1539 connection.cc:256] RPC call timeout handler was delayed by -2.2433795480000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:04:01.780400  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 64510.7us
W20260709 13:04:01.781390  1539 connection.cc:256] RPC call timeout handler was delayed by -2.1884795480000321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260709 13:04:01.781819  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9859795480000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 13:04:01.782104  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9454795480000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
W20260709 13:04:01.782383  1539 connection.cc:256] RPC call timeout handler was delayed by -1.9157795480000459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:04:01.740486  2380 rpcz_store.cc:276] 0709 13:03:59.782634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.785356 (+  2722us) service_pool.cc:224] Handling call
0709 13:03:59.785913 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.789517 (+  3604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.789583 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.791779 (+  2196us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.225690 (+433911us) spinlock_profiling.cc:243] Waited 924 us on lock 0x7b180047db20. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e0d1 00007fc833d29a0f
0709 13:04:00.646937 (+421247us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:00.646990 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.647061 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.544962 (+897901us) raft_consensus.cc:1685] finished
0709 13:04:01.544991 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.568888 (+ 23897us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2964,"raft.queue_time_us":61352,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":10150,"spinlock_wait_cycles":2125696,"thread_start_us":420958,"threads_started":1,"wal-append.queue_time_us":1272415}
I20260709 13:04:01.747380  7866 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.777820  1718 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 13:04:01.786091  1721 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
W20260709 13:04:01.786496  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 170262us
I20260709 13:04:01.784437  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:01.791472  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.783847  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.702147  8112 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.779278  1725 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1acf73e529394004842219637cff7cfc"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:01.802198  1649 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:01.626163  8146 consensus_queue.cc:1059] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 13:04:01.626050  8170 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.805796  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:01.707273  8285 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.765537  1716 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 13:04:01.812255  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72574.9us
I20260709 13:04:01.743978  7450 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:01.823493  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79692.6us
W20260709 13:04:01.852761  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.828s (SENT)
I20260709 13:04:01.853363  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.722836  8329 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:01.712312  8230 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.730402  8281 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:01.842782  1616 leader_election.cc:400] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:04:01.878940  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.849802  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.831938  1725 raft_consensus.cc:606] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:01.882387  1725 raft_consensus.cc:614] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:01.859490  1716 raft_consensus.cc:1240] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:01.804944  8297 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:01.838264  8304 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:04:01.832381  1472 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:01.820942  8249 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.753605  8274 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.862479  8273 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.836640  8068 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.902683  1573 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:01.902501  1472 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:04:01.902781  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 119002us
I20260709 13:04:01.838541  7971 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.887934  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:01.797771  8302 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:04:01.803447  8193 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:04:01.843178  8112 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.846302  8231 consensus_queue.cc:1059] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:01.877177  1719 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:04:01.895898  8314 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.890398  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.895671  8259 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:00.691999  1466 rpcz_store.cc:269] 0709 13:03:05.924219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.929696 (+  5477us) service_pool.cc:224] Handling call
0709 13:04:00.030332 (+54100636us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:52.392965 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:52.415681 (+ 22716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:52.415730 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:03:52.415764 (+    34us) tablet.cc:662] Decoding operations
0709 13:03:52.417244 (+  1480us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:52.417347 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:52.417379 (+    32us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:03:52.418812 (+  1433us) tablet.cc:701] Row locks acquired
0709 13:03:52.418863 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:03:52.419067 (+   204us) write_op.cc:270] Start()
0709 13:03:52.419231 (+   164us) write_op.cc:276] Timestamp: P: 1783602232418981 usec, L: 0
0709 13:03:52.419253 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:52.470857 (+ 51604us) log.cc:844] Serialized 5936 byte log entry
0709 13:03:53.216742 (+745885us) op_driver.cc:464] REPLICATION: finished
0709 13:03:53.493848 (+277106us) write_op.cc:301] APPLY: starting
0709 13:03:53.493966 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:53.752853 (+258887us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:53.807371 (+ 54518us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:54.910399 (+1103028us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:54.910881 (+   482us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 13:03:54.910929 (+    48us) write_op.cc:312] APPLY: finished
0709 13:03:54.911460 (+   531us) log.cc:844] Serialized 139 byte log entry
0709 13:03:55.027067 (+115607us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:55.039905 (+ 12838us) write_op.cc:454] Released schema lock
0709 13:03:55.053899 (+ 13994us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:55.105936 (+ 52037us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:55.105989 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":371812,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":269023,"apply.run_cpu_time_us":59550,"apply.run_wall_time_us":1612001,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66917,"num_ops":15,"prepare.queue_time_us":110685,"prepare.run_cpu_time_us":25902,"prepare.run_wall_time_us":164309,"raft.queue_time_us":192071,"raft.run_cpu_time_us":72957,"raft.run_wall_time_us":158497,"replication_time_us":779613,"spinlock_wait_cycles":1687296,"thread_start_us":417645,"threads_started":5,"wal-append.queue_time_us":279712,"wal-append.run_cpu_time_us":35694,"wal-append.run_wall_time_us":2585865}]]}
I20260709 13:04:01.863756  8285 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.815135  7850 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:01.930104  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77370.2us
I20260709 13:04:01.899408  8336 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:01.941373  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51069.9us
I20260709 13:04:01.942811  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:01.911484  8254 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:01.954218  8322 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 13:04:01.835577  8253 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:01.921947  8297 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.958662  1575 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:01.970031  1569 raft_consensus.cc:1240] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:01.972491  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:01.973565  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 64080.4us
I20260709 13:04:01.966951  2376 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:01.992105  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:02.039456  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151590us
I20260709 13:04:01.963583  8231 raft_consensus.cc:3055] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:01.958689  8146 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.041095  1472 catalog_manager.cc:5697] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 12 to 13, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 13 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.965505  8274 raft_consensus.cc:3055] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:02.064821  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72788.8us
I20260709 13:04:01.967916  8319 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.164s
I20260709 13:04:01.961284  7971 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.929113  8131 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:01.968887  8317 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:02.088167  8335 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:04:02.069067  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2831) took 2215 ms. Trace:
I20260709 13:04:02.088742  8317 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.089047  1576 rpcz_store.cc:276] 0709 13:03:59.852662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.919415 (+ 66753us) service_pool.cc:224] Handling call
0709 13:03:59.919740 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.405561 (+1485821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.067888 (+662327us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400214dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.068227 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3193510272}
I20260709 13:04:02.088256  8131 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.070465  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:01.991552  8254 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.982901  8325 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21.
I20260709 13:04:02.084522  1472 catalog_manager.cc:5697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 12 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 13 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.069597  8336 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.051532  8331 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.102166  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:02.072109  8231 raft_consensus.cc:740] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:02.108579  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:02.077306  8204 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:02.106505  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 279474us
I20260709 13:04:02.080016  8274 raft_consensus.cc:740] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:01.931159  8329 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.121862  1643 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 15. Current term is 16. Ops: []
W20260709 13:04:02.122373  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.641s (SENT)
W20260709 13:04:02.123035  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 147101us
I20260709 13:04:02.129124  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:02.107031  8112 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.104079  7882 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.137162  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:02.136708  1614 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:02.105108  8280 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:02.142225  8281 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.117309  8318 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.127251  8346 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:02.146910  8353 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:02.148628  8350 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:02.150290  8339 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:04:02.150790  8204 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.151329  8329 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.157807  8285 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:02.161649  8342 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:04:02.150004  8319 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:02.165518  8319 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:02.166122  8319 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:02.169662  8338 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:04:02.131520  8170 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:02.171916  8112 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.173198  8231 consensus_queue.cc:260] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.174890  8231 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:02.132179  8230 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.161052  8274 consensus_queue.cc:260] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.178953  8274 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.182049  8355 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:02.183104  2345 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.184314  8341 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.238s	user 0.001s	sys 0.001s
I20260709 13:04:02.191253  8131 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.167440  8319 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.193372  8319 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:02.193764  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:02.194449  1719 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:04:02.194832  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:02.183157  8355 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:02.198784  8355 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.198956  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:02.202131  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 999) took 1475 ms. Trace:
W20260709 13:04:02.195005  8313 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.003s
I20260709 13:04:02.202605  1644 rpcz_store.cc:276] 0709 13:04:00.727224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.727438 (+   214us) service_pool.cc:224] Handling call
0709 13:04:00.727878 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.544494 (+816616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.201880 (+657386us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54001bc1d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.202072 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864642432}
I20260709 13:04:02.203064  8313 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.135363  7902 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:02.206866  7902 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:02.207492  7902 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.209453  7866 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.206089  8313 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.209479  8336 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:02.203692  8169 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.753s	user 0.013s	sys 0.000s
W20260709 13:04:02.208173  8292 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.257s	user 0.025s	sys 0.040s
I20260709 13:04:02.220463  8336 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.224562  7450 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.229229  2375 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:02.228096  8336 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.220633  7866 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.241904  7914 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.243343  7914 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.243438  8110 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:02.252921  8110 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:02.256471  8110 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.223872  8169 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.264014  8290 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.266601  8223 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:02.225679  8249 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:02.274886  8195 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:02.278746  8290 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:02.273805  8110 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.279592  8195 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:02.276089  8336 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:02.373381  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 208) took 18335 ms (client timeout 15000 ms). Trace:
W20260709 13:04:02.374868  1466 rpcz_store.cc:269] 0709 13:03:44.037849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.371632 (+18333783us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:02.372663 (+  1031us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:02.371131  1472 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 10 to 11, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 11 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:02.379519  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 222) took 15085 ms (client timeout 15000 ms). Trace:
W20260709 13:04:02.380223  1466 rpcz_store.cc:269] 0709 13:03:47.294316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.379104 (+15084788us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:02.379369 (+   265us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:04:02.205268  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 68716.3us
W20260709 13:04:02.209669  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72564.4us
I20260709 13:04:02.390491  1567 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:02.394057  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2845) took 1068 ms. Trace:
I20260709 13:04:02.420095  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:02.388186  1472 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:02.400477  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.233028  7450 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:02.408298  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:02.444205  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:02.406423  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:02.408648  1721 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:02.400583  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:02.395162  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2837) took 1165 ms. Trace:
I20260709 13:04:02.435585  8195 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.446319  1574 rpcz_store.cc:276] 0709 13:04:01.229153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.628186 (+399033us) service_pool.cc:224] Handling call
0709 13:04:01.628814 (+   628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.718776 (+ 89962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.718821 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.736754 (+ 17933us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.131638 (+394884us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:02.131697 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.131790 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.394444 (+262654us) raft_consensus.cc:1685] finished
0709 13:04:02.394529 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.394947 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89304,"raft.queue_time_us":94138,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":23032,"thread_start_us":394208,"threads_started":1,"wal-append.queue_time_us":503734}
I20260709 13:04:02.450181  8355 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:02.452411  8355 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.453148  8355 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:02.459256  8169 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:02.420231  1570 rpcz_store.cc:276] 0709 13:04:01.325352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.781394 (+456042us) service_pool.cc:224] Handling call
0709 13:04:01.795160 (+ 13766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.795448 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:01.795477 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:01.796807 (+  1330us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.066276 (+269469us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:02.066324 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.066450 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.392996 (+326546us) raft_consensus.cc:1685] finished
0709 13:04:02.393087 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.393602 (+   515us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":269045,"threads_started":1,"wal-append.queue_time_us":351608}
I20260709 13:04:02.465169  8195 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.472415  8336 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:02.474630  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.405381  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 258936us
I20260709 13:04:02.478459  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:02.414410  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 312315us
I20260709 13:04:02.487485  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3162) took 1131 ms. Trace:
I20260709 13:04:02.498915  8231 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.501211  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:02.503039  1722 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.496959  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:02.507177  8345 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.488183  1717 rpcz_store.cc:276] 0709 13:04:01.355682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.361924 (+  6242us) service_pool.cc:224] Handling call
0709 13:04:02.483047 (+1121123us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54003dc7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.487215 (+  4168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108394240}
I20260709 13:04:02.508323  8345 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:02.519510  8299 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 1.451s	user 0.008s	sys 0.009s
I20260709 13:04:02.524179  2371 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:02.537328  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:02.526168  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2846) took 1200 ms. Trace:
I20260709 13:04:02.542716  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:02.546195  1722 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:02.549692  2375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41530 (request call id 708) took 1186 ms. Trace:
I20260709 13:04:02.550314  2375 rpcz_store.cc:276] 0709 13:04:01.362765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.365495 (+  2730us) service_pool.cc:224] Handling call
0709 13:04:01.369647 (+  4152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.229688 (+860041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.548778 (+319090us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540057e4d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.549423 (+   645us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1975263488}
I20260709 13:04:02.524281  8345 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:02.553380  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49780.8us
I20260709 13:04:02.554265  1644 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.554874  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:02.563206  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2851) took 1232 ms. Trace:
I20260709 13:04:02.570015  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3145) took 2095 ms. Trace:
I20260709 13:04:02.570616  8329 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.571161  8329 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.573230  8329 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:02.573585  1645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2428) took 2326 ms (client timeout 2210 ms). Trace:
W20260709 13:04:02.574385  1645 rpcz_store.cc:269] 0709 13:04:00.246381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.247585 (+  1204us) service_pool.cc:224] Handling call
0709 13:04:02.565943 (+2318358us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003fc854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.573191 (+  7248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579920640}
I20260709 13:04:02.574381  8169 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:02.576206  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41530 (request call id 709) took 1213 ms. Trace:
I20260709 13:04:02.576870  2378 rpcz_store.cc:276] 0709 13:04:01.363051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.365578 (+  2527us) service_pool.cc:224] Handling call
0709 13:04:02.557282 (+1191704us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400730354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.575951 (+ 18669us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392648960}
W20260709 13:04:02.579355  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.573479  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:02.580668  7909 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:02.583240  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2427) took 2337 ms (client timeout 1736 ms). Trace:
I20260709 13:04:02.583866  1646 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:02.586627  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2435) took 1541 ms. Trace:
I20260709 13:04:02.570770  1573 rpcz_store.cc:276] 0709 13:04:00.474831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.968572 (+493741us) service_pool.cc:224] Handling call
0709 13:04:00.982908 (+ 14336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.909711 (+926803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.569252 (+659541us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54001257d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.569730 (+   478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108179200}
I20260709 13:04:02.540766  1567 rpcz_store.cc:276] 0709 13:04:01.325627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.789447 (+463820us) service_pool.cc:224] Handling call
0709 13:04:01.804596 (+ 15149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.418741 (+614145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.504264 (+ 85523us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b540028afd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.526064 (+ 21800us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1344566528}
I20260709 13:04:02.563902  1575 rpcz_store.cc:276] 0709 13:04:01.330240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.844255 (+514015us) service_pool.cc:224] Handling call
0709 13:04:01.876706 (+ 32451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.959281 (+ 82575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.562293 (+603012us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b54000276d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.562948 (+   655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143134976}
I20260709 13:04:02.587201  1646 rpcz_store.cc:276] 0709 13:04:01.044795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.624333 (+579538us) service_pool.cc:224] Handling call
0709 13:04:02.582941 (+958608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.584578 (+  1637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.586178 (+  1600us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54000f55d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.586410 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2015453952}
W20260709 13:04:02.583987  1650 rpcz_store.cc:269] 0709 13:04:00.246021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.247104 (+  1083us) service_pool.cc:224] Handling call
0709 13:04:02.582613 (+2335509us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54000f55d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.582885 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854861184}
I20260709 13:04:02.598477  8193 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:02.599239  8193 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:02.601868  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3152) took 1294 ms. Trace:
I20260709 13:04:02.495370  1472 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 10 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 14 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.602488  1721 rpcz_store.cc:276] 0709 13:04:01.307510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.808122 (+500612us) service_pool.cc:224] Handling call
0709 13:04:01.808533 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.504879 (+696346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.601333 (+ 96454us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540010d6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.601755 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376934144}
W20260709 13:04:02.620790  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.635310  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 92861.1us
I20260709 13:04:02.624306  1472 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 10 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 13 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:02.635380  1644 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
W20260709 13:04:02.646312  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 168256us
I20260709 13:04:02.659715  8193 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.662432  8193 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:02.666208  1651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2426) took 2420 ms (client timeout 1938 ms). Trace:
W20260709 13:04:02.667297  1651 rpcz_store.cc:269] 0709 13:04:00.245497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.246524 (+  1027us) service_pool.cc:224] Handling call
0709 13:04:02.665384 (+2418860us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400307854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.665824 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136126720}
I20260709 13:04:02.672281  8290 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.675149  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2849) took 1348 ms. Trace:
I20260709 13:04:02.676926  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2611) took 1399 ms. Trace:
I20260709 13:04:02.677520  1643 rpcz_store.cc:276] 0709 13:04:01.264704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.682570 (+417866us) service_pool.cc:224] Handling call
0709 13:04:01.702660 (+ 20090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.125546 (+422886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.663666 (+538120us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540005d7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.664158 (+   492us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949081472}
I20260709 13:04:02.678274  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3156) took 1361 ms. Trace:
I20260709 13:04:02.678849  1718 rpcz_store.cc:276] 0709 13:04:01.317021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.836263 (+519242us) service_pool.cc:224] Handling call
0709 13:04:02.676657 (+840394us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b580016fe98. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:04:02.677343 (+   686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.677775 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.677804 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.677864 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:02.677909 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.677967 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.678098 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42395008}
W20260709 13:04:02.675922  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.681978  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.683898  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:02.688097  8231 raft_consensus.cc:2764] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:04:02.676151  1569 rpcz_store.cc:276] 0709 13:04:01.326534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.794237 (+467703us) service_pool.cc:224] Handling call
0709 13:04:01.806833 (+ 12596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.970581 (+163748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.674236 (+703655us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400364cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.674823 (+   587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374529792}
I20260709 13:04:02.691256  1690 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:02.697880  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2822) took 3309 ms. Trace:
I20260709 13:04:02.701679  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1000) took 1861 ms. Trace:
I20260709 13:04:02.702234  1648 rpcz_store.cc:276] 0709 13:04:00.727605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.729848 (+  2243us) service_pool.cc:224] Handling call
0709 13:04:00.981133 (+251285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.223492 (+242359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.497812 (+1274320us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400306e50. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.589401 (+ 91589us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568237952}
I20260709 13:04:02.706564  8226 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:02.714069  8211 raft_consensus.cc:3060] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.717108  8226 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:02.717783  8253 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:02.719735  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 164969us
I20260709 13:04:02.675879  8290 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:02.726255  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46261.3us
I20260709 13:04:02.698647  1572 rpcz_store.cc:276] 0709 13:03:59.388460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.559280 (+170820us) service_pool.cc:224] Handling call
0709 13:03:59.560212 (+   932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.560592 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.560630 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.620150 (+ 59520us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.088870 (+468720us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:00.088935 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.089034 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.517046 (+2428012us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:04:02.697138 (+180092us) raft_consensus.cc:1685] finished
0709 13:04:02.697165 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.697570 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":212100,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":18577,"spinlock_wait_cycles":23626496,"thread_start_us":378396,"threads_started":1,"wal-append.queue_time_us":1239397}
I20260709 13:04:02.745576  8066 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:02.746302  8066 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.746815  8066 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.748405  8002 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 13:04:02.753203  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 758) took 1893 ms. Trace:
I20260709 13:04:02.753784  7909 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:02.754297  7909 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:02.755143  1573 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:04:02.755204  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2619) took 1461 ms (client timeout 1555 ms). Trace:
W20260709 13:04:02.755882  1642 rpcz_store.cc:269] 0709 13:04:01.293290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.303229 (+  9939us) service_pool.cc:224] Handling call
0709 13:04:02.754675 (+1451446us) spinlock_profiling.cc:243] Waited 86.3 ms on lock 0x7b54000b9854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.754993 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198580096}
I20260709 13:04:02.733366  8169 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:02.779680  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:02.777864  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:02.779739  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:02.755605  7909 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.786532  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:02.779302  8345 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:02.802362  1724 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:02.810318  8357 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.821249  7850 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.820915  8357 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.822405  7850 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:02.829493  8224 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:02.829656  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:02.830410  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:02.828650  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65886.6us
I20260709 13:04:02.753786  2377 rpcz_store.cc:276] 0709 13:04:00.859606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.860109 (+   503us) service_pool.cc:224] Handling call
0709 13:04:00.860889 (+   780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.227877 (+366988us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:01.448105 (+220228us) log.cc:844] Serialized 14 byte log entry
0709 13:04:02.555481 (+1107376us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b1800361c00. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a
0709 13:04:02.555940 (+   459us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.571361 (+ 15421us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.603729 (+ 32368us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:02.603799 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.718037 (+114238us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b1800361c00. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce3dbe 00007fc833cee930 00007fc833cee8d8
0709 13:04:02.749129 (+ 31092us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b1800361c00. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c4f2a6 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419
0709 13:04:02.750691 (+  1562us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54001568d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.750778 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:02.751374 (+   596us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b54001570a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833ce3dbe 00007fc833cee930 00007fc833cee8d8 00007fc833cee858 00007fc833cee813 00007fc833ceda9c 00007fc839bd9626 00007fc839bd3dd8 00007fc833d2a7cb 00007fc833d2d2c0
0709 13:04:02.752628 (+  1254us) raft_consensus.cc:1685] finished
0709 13:04:02.752657 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.752958 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":370256,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":37731,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":258,"spinlock_wait_cycles":970628352,"thread_start_us":208121,"threads_started":2,"wal-append.queue_time_us":369473}
I20260709 13:04:02.842159  1567 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.806902  8345 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:02.827673  1472 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 6 to 11, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 11 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:02.848773  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 83089.7us
I20260709 13:04:02.829520  8317 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:02.856177  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74472.4us
W20260709 13:04:02.865629  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100816us
I20260709 13:04:02.841099  8273 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:02.864825  8290 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:04:02.877408  8297 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:04:02.878631  8325 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:02.884279  1724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2702) took 1620 ms (client timeout 1999 ms). Trace:
I20260709 13:04:02.884743  7914 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:02.885752  8178 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:02.893646  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:02.895373  1724 rpcz_store.cc:269] 0709 13:04:01.263512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.264451 (+   939us) service_pool.cc:224] Handling call
0709 13:04:02.872469 (+1608018us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400224554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:02.884146 (+ 11677us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3225172736}
I20260709 13:04:02.857916  8224 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:02.888191  1472 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 15 to 16, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:02.896261  8169 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:02.910751  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.906978  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:02.901559  8178 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.911182  7909 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.909438  8325 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:02.931535  1572 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.923755  8226 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:02.938670  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.924396  8178 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.948755  8357 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:02.951646  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 92879us
I20260709 13:04:02.929709  8273 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.962945  8273 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:02.965389  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:02.972522  8274 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.732s	user 0.027s	sys 0.001s
I20260709 13:04:02.977327  8325 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:02.980813  8224 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:02.998095  8110 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.018s
I20260709 13:04:02.984889  8169 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:03.027071  7866 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.781s	user 0.019s	sys 0.000s
I20260709 13:04:02.972673  8226 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:03.050618  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 110304us
W20260709 13:04:03.041934  8195 consensus_meta.cc:287] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.014s
I20260709 13:04:03.033519  7850 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:03.069871  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.066000  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 189570us
I20260709 13:04:03.004747  8345 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:03.086501  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 254587us
I20260709 13:04:03.059744  8169 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:04:03.100360  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 146552us
I20260709 13:04:03.101042  8317 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:03.111145  7902 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.903s	user 0.022s	sys 0.003s
W20260709 13:04:03.111931  8355 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.658s	user 0.016s	sys 0.002s
I20260709 13:04:03.115331  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.052232  7866 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.116462  8223 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:04:03.117225  2372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 935) took 1081 ms. Trace:
I20260709 13:04:03.117743  1720 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:04:03.122972  8329 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.007s
W20260709 13:04:03.125137  8319 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.931s	user 0.021s	sys 0.011s
I20260709 13:04:03.125001  1568 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
W20260709 13:04:03.125748  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 121278us
W20260709 13:04:03.128960  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.116521  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:03.151369  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51272.9us
W20260709 13:04:03.151791  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.132041  8226 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.154186  2372 rpcz_store.cc:276] 0709 13:04:01.936283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.936608 (+   325us) service_pool.cc:224] Handling call
0709 13:04:01.937184 (+   576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.961102 (+ 23918us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:02.038531 (+ 77429us) log.cc:844] Serialized 14 byte log entry
0709 13:04:02.752825 (+714294us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b1800368fe0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c63e07 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c80c02 00007fc83390820b 00007fc83390dd23 00007fc83390e23a 00007fc833d78a2a
0709 13:04:02.954896 (+202071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.955071 (+   175us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:02.955263 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.995688 (+ 40425us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54005e0e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:02.995764 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.016839 (+ 21075us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38768,"prepare.run_cpu_time_us":160,"prepare.run_wall_time_us":161,"raft.queue_time_us":8082,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":4521,"spinlock_wait_cycles":63445504,"thread_start_us":165277,"threads_started":1,"wal-append.queue_time_us":973426}
I20260709 13:04:03.187393  1718 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 21.
I20260709 13:04:03.203277  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:03.195451  8362 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.002s
I20260709 13:04:03.157888  8355 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.172194  8329 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.107372  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:04:03.224812  1721 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 12.
I20260709 13:04:03.229347  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.158224  7902 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.178730  8317 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.237121  7866 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:03.240499  1642 raft_consensus.cc:606] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:03.241508  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:03.244863  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 148108us
I20260709 13:04:03.256726  1642 raft_consensus.cc:614] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:03.246316  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:03.243992  8193 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.581s	user 0.015s	sys 0.003s
W20260709 13:04:03.282109  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.285598  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95412.3us
W20260709 13:04:03.289484  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151086us
W20260709 13:04:03.289819  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.304736  1568 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:03.312104  1569 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:03.244740  8319 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 13:04:03.318534  1574 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.315080  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:03.305429  8110 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.356904  8110 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.357985  1720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:03.358749  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:03.358949  1720 raft_consensus.cc:606] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:03.360041  1720 raft_consensus.cc:614] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:03.332208  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:03.332551  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.333971  1718 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:03.293780  8169 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.337242  8355 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.340067  8345 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.341377  8329 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.348546  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:03.394979  1721 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.321293  8298 consensus_queue.cc:1059] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 13:04:03.354800  7902 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.383366  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.384991  1567 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:03.364316  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:03.389936  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 258444us
I20260709 13:04:03.396068  1568 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:04:03.298822  8278 consensus_queue.cc:1059] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.206s
I20260709 13:04:03.360857  8350 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.360632  8193 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.360226  8110 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:03.361675  8317 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:03.400839  8211 consensus_meta.cc:287] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.010s
I20260709 13:04:03.413226  8367 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:03.418195  8367 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.419116  8367 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.419391  8278 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:03.420064  8278 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:03.421072  1647 raft_consensus.cc:1275] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:04:03.413735  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55298.1us
I20260709 13:04:03.423053  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:03.424649  8357 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:03.414963  8376 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.163s	user 0.000s	sys 0.002s
W20260709 13:04:03.415544  8379 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:04:03.403100  8253 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.685s	user 0.018s	sys 0.002s
I20260709 13:04:03.429546  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:03.408242  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 229036us
W20260709 13:04:03.410787  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 78653.6us
I20260709 13:04:03.415859  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.440618  8226 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:03.442055  8226 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.441179  8382 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.440676  1570 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.446271  1572 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:03.409314  8350 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.456959  8350 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.458747  8252 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.384217  7450 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 13:04:03.464543  8066 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.717s	user 0.004s	sys 0.015s
I20260709 13:04:03.465169  8066 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.466528  1724 raft_consensus.cc:1240] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:03.428812  7909 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:03.483963  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.485009  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.467955  8066 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:03.495442  7866 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:03.497102  1574 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:03.508634  1721 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
W20260709 13:04:03.524323  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.478564  8382 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.428103  8178 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.496256  7866 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:03.482321  7909 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.535939  8386 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:03.536446  1642 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:03.538498  8370 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:03.539055  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.541899  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:03.549322  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.570958  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:03.572615  8193 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.572889  7866 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.573473  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:03.574625  7866 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.510919  8387 raft_consensus.cc:3055] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:03.578157  8110 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.578284  8387 raft_consensus.cc:740] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:03.582166  8178 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:03.579840  8387 consensus_queue.cc:260] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.584000  8387 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:03.591099  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60354.3us
I20260709 13:04:03.592306  8252 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.592844  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:03.593958  8317 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.593218  8252 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:03.595304  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.657s (SENT)
I20260709 13:04:03.596293  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:03.597580  8252 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:03.600831  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 99200us
I20260709 13:04:03.536170  8226 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.515188  8336 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:04:03.525667  8211 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:04:03.605989  8370 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.610272  8297 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.615960  1570 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
W20260709 13:04:03.631999  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87070.4us
W20260709 13:04:03.651434  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 242144us
I20260709 13:04:03.563520  7909 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:03.610855  8297 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:03.658952  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46200.2us
W20260709 13:04:03.661732  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59923.6us
W20260709 13:04:03.668462  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.682005  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47700.2us
I20260709 13:04:03.692075  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:03.689941  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:03.687104  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:03.663542  8290 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:03.663348  8297 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:03.703145  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 91988.5us
I20260709 13:04:03.672777  8371 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:03.686606  7450 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.715554  8393 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:03.720727  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.722110  1570 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:04:03.724725  8226 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:03.726819  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57541us
I20260709 13:04:03.739037  8371 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.742107  8371 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.736842  1690 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:03.749536  8384 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.358s	user 0.004s	sys 0.002s
I20260709 13:04:03.752614  8290 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.756515  8211 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:03.767800  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 74133.7us
I20260709 13:04:03.746505  8253 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:03.770651  7450 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.771406  7450 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.776927  8396 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:03.778090  8396 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:03.725715  8226 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:03.788820  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53317.6us
I20260709 13:04:03.792603  8370 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:03.689235  8325 consensus_queue.cc:1059] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:04:03.791932  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:03.756232  8290 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:03.798877  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1005) took 2170 ms. Trace:
I20260709 13:04:03.803186  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.803849  1570 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:03.806010  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 122503us
I20260709 13:04:03.807580  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.809325  1718 raft_consensus.cc:606] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:03.809825  1718 raft_consensus.cc:614] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:03.761960  7902 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.807018  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:03.801565  1647 rpcz_store.cc:276] 0709 13:04:01.628473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.778890 (+150417us) service_pool.cc:224] Handling call
0709 13:04:01.822317 (+ 43427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.421689 (+1599372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.793808 (+372119us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540038b8d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:03.798782 (+  4974us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3673457024}
W20260709 13:04:03.813931  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 65986.1us
I20260709 13:04:03.819801  8329 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.821115  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.822172  8252 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.822911  8252 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.824519  8066 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.824928  8211 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.825603  8211 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.825695  7902 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.832429  7450 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.833060  7902 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.833767  8226 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.826027  8195 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:03.834930  8195 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:03.835899  8195 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.837668  8195 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.812248  8382 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:03.841552  8315 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:03.720521  8389 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.144s
I20260709 13:04:03.799882  8253 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:03.842258  8315 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.842350  1569 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:03.820713  8329 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.855355  1690 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:03.857647  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.858222  8253 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.859745  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.860936  1721 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:04:03.861544  8389 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.865880  8110 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.865891  8336 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:03.867343  8336 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:03.868634  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 76816.3us
I20260709 13:04:03.868475  8336 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.868005  8329 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.870353  8336 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:03.794476  8367 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.875923  8226 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:03.878355  8277 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:03.882035  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.882256  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.882872  1574 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.882833  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.883796  1570 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:03.884824  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.885813  1567 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 19: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:03.886757  8278 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:03.888116  8392 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:04:03.889577  7850 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:03.890190  7850 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:03.862358  8389 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.894392  8278 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.895062  8389 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:03.895923  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.896890  1569 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:03.897862  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:03.898077  1574 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:03.899857  8367 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.900691  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.907918  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:03.891582  7850 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.909477  8252 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.910310  8252 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:03.910944  8252 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:03.911789  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.912729  7850 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:03.902910  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52235us
W20260709 13:04:03.917317  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 108072us
I20260709 13:04:03.920128  1721 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:03.918514  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:03.922605  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:03.917593  8302 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:03.865036  8315 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.923615  8302 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:03.924147  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:03.925163  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:03.925935  8302 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.926887  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.927582  1574 raft_consensus.cc:2376] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19. Re-sending same reply.
I20260709 13:04:03.923454  1724 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:03.929304  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.930272  1722 raft_consensus.cc:2410] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:04:03.890218  8397 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
I20260709 13:04:03.779839  8396 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.923645  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.932667  1721 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:03.935344  8387 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.936852  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.905241  1718 raft_consensus.cc:606] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:03.938200  1570 raft_consensus.cc:2410] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:04:03.938264  1718 raft_consensus.cc:614] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:03.938565  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.939273  1569 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:03.846879  8290 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:03.943984  8290 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.945137  8401 raft_consensus.cc:3055] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:03.946591  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.947198  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.948006  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.948863  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.949142  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.948302  1573 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:03.949364  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.948024  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.951344  1571 raft_consensus.cc:2376] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 17. Re-sending same reply.
I20260709 13:04:03.951272  1568 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:03.952247  8211 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:03.950130  1567 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:03.955541  8193 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.946259  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.949991  1572 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:03.958106  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:03.958225  1574 raft_consensus.cc:2376] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 17. Re-sending same reply.
I20260709 13:04:03.960623  8178 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.961175  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:03.962220  1642 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
W20260709 13:04:03.962332  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88894.1us
I20260709 13:04:03.961274  8178 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:03.952399  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.970268  8178 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:03.973806  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:03.977285  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:03.978243  1576 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:04:03.958434  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:03.984599  1718 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:03.976820  1721 raft_consensus.cc:2410] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:04:03.987335  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 310472us
I20260709 13:04:03.988952  1724 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:04.002053  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:03.902642  7909 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:03.945940  8401 raft_consensus.cc:740] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:04.017266  1575 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:04:04.022668  1576 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:04.024581  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:04.027192  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.029007  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.029750  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.031706  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:04.032663  1717 raft_consensus.cc:606] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:04.033890  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.034250  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:04.035054  1572 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.865247  8350 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.036633  8400 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:04.037305  8400 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.037683  8400 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.025192  1719 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:04.039849  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.045392  1569 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:04.046684   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:04.055630  1575 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:04.055639  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.059132  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.050040  1725 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:04.059254  1721 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:04:04.040923  1571 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:04:04.049254  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.063653  1574 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.065896  1572 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:04:04.074553  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.173s (SENT)
I20260709 13:04:04.075179  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:04.028968  8329 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.081168  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:04.036646  7902 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.082921  1576 raft_consensus.cc:2376] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 14. Re-sending same reply.
I20260709 13:04:03.971571  8252 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.064785  8178 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.046756  8350 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:04.045104  8193 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:04.085909  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:04.087052  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.094316  7902 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.095062  8316 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.092113  8382 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.101109  1575 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:04.054633  7450 consensus_queue.cc:1048] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:04:04.104697  1571 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.105579  1576 raft_consensus.cc:2410] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:04:04.107120  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58058.4us
I20260709 13:04:04.131286  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.133010   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:04.132608  8382 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.090205  8325 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.091028  8290 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:04.148613  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.151841  1575 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
W20260709 13:04:04.148236  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 64707.3us
I20260709 13:04:04.150449  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:04.154349  1570 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:04.160694  1574 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:04.172833  1647 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:04:04.174419  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.175662  1575 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
W20260709 13:04:04.175806  8405 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.088s	user 0.000s	sys 0.002s
I20260709 13:04:04.180861  8410 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:04.181695  8410 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.184547  8253 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.185475  1690 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:04.186174  8382 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.187546  7902 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.189035  8396 raft_consensus.cc:2804] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:04.189810  8396 raft_consensus.cc:697] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:04.189420  8400 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.190936  8396 consensus_queue.cc:237] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.191859  8319 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:04.150039  8345 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:04.193781  8345 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.180886  8325 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.198973  8325 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.122778  8387 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.185297  8253 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.179847  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.217085  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:04.218590  1721 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:04.180878  1719 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:04.206149  1576 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:04.215848  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:04.233889  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.234511  8253 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.234877  8400 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.235287  1570 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 13: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:04.235859  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.238972  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:04.241587  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.244617  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:04.252873  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.254887  7902 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:04.256477  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:04.258539  8325 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.258972  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.265342  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:04.277752  1568 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:04:04.197114  7909 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.141254  8178 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.142875  8329 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.201748  8345 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:04.290300  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.292718  1721 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 22.
I20260709 13:04:04.297784  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.214182  8336 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:04.301826  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:04.304209  2372 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:04.266309  8297 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.003s
I20260709 13:04:04.296759  8382 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:04.306492  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.185477  8302 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:04:04.140491  8316 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.306141  1614 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:04.305467  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:04.319720  8386 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:04.320261  8386 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.320783  8386 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:04.323691  8336 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.324610  1644 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:04.326749  8336 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.329926  8316 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.330652  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.331513  1723 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:04:04.297678  8410 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:04.332842  7914 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.333137  8382 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.333511  7914 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.335148  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.335795  8382 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.313154  8345 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.306237  8390 log.cc:927] Time spent T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:04:04.344336  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.337236  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:04.346537  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.215799  8193 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.343575  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.346638  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.336112  1570 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 19: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:04.348424  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.306018  8224 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:04.345207  2372 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:04.356487  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.357162  8329 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.357681  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:04.357885  8329 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:04.358593  1642 raft_consensus.cc:2410] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:04:04.359773  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:04.360636  2378 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:04:04.366686  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.368394  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.369220  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.369233  1570 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:04.370147  1574 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:04.370829  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.371068  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.371847  1570 raft_consensus.cc:2410] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:04.371758  1574 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:04.285364  8253 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:04:04.373389  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.375597  1574 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:04.377223  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.380061  8178 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:04.380882  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.382787  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:04.383927  1575 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:04.384708  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.385313  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:04.385666  8336 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:04.388765  8336 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.389261  8336 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.389539  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.390466  1648 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:04.379643  8193 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:04.211284  8401 consensus_queue.cc:260] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.384781  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.315291  8297 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.394413  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:04.395382  8195 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:04.395471  8382 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:04.398111  8382 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.398504  8382 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.398442  2378 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:04:04.353638  8316 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:04.401427  8316 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:04.402869  8316 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.405112  8315 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.407954  8315 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.387924  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.386251  2371 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:04.387534  2376 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:04.392182  1720 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:04:04.410697  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.418646  1576 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:04.420037  1576 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:04.420529  1576 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:04.421623  1576 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.424795  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.426728  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.431037  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.439533  1576 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.336133  7914 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:04.432264  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:04.448318  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.449226  1721 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:04.306237  8211 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 13:04:04.296994  8387 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.479928  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:04.466040  1571 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:04.475831  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.482508  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.483377  1718 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:04.396229  8297 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.462716  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:04.343272  8345 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.306416  7902 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.483294  1642 raft_consensus.cc:2376] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18. Re-sending same reply.
I20260709 13:04:04.496801  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:04.496608  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.432088  8399 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:04.502168  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.506995  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.510437  1575 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:04:04.520529  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:04.476044  8290 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:04.426853  8382 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.522034  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:04.518852  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:04.527751  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.529757  1576 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:04:04.530275  1576 pending_rounds.cc:85] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 12
I20260709 13:04:04.523104  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:04.533107  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.528532  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:04.541850  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:04.514482  8195 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:04.549479  1644 raft_consensus.cc:2393] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:04.562094  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 151735us
I20260709 13:04:04.556572  1642 raft_consensus.cc:2376] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20. Re-sending same reply.
I20260709 13:04:04.555101  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.556493  1571 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 13:04:04.530577  1576 pending_rounds.cc:107] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 16.13
W20260709 13:04:04.566824  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43922.6us
I20260709 13:04:04.578159  1576 raft_consensus.cc:2889] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:04.546272  8317 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:04.580549  8317 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.581051  8317 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:04.516829  8399 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.584687  8288 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:04.585291  8288 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.585652  8387 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:04.580595  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.586838  1648 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 21: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:04.585688  8288 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.588115  8382 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.533211  8298 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:04.528971  8297 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:04.617519  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:04:04.548633  7850 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.567s	user 0.015s	sys 0.000s
I20260709 13:04:04.623149  1690 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:04.566185  8195 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.638819  1572 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:04.647986  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:04.548117  8110 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:04:04.661371  2380 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:04:04.628616  8298 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.541079  8290 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.657384  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:04.664687  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.636422  8387 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.645174  8195 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:04.658954  7850 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:04.663188  8278 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:04.708735  8278 raft_consensus.cc:697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:04.710173  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.710098  8278 consensus_queue.cc:237] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.668560  8298 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:04.719255  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:04.711635  8414 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
I20260709 13:04:04.711241  1720 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:04.735569  2376 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:04.674716  7914 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:04.662367  8297 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.778306  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:04.781903  1720 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.772631  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:04.784479  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 76575.4us
I20260709 13:04:04.789496  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:04.790378  1718 raft_consensus.cc:3060] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.760892  8416 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.793836  1644 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:04.724037  8211 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:04:04.821164  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.825511  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:04.832458  1723 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 21. Current term is 22. Ops: [19.14-21.15]
W20260709 13:04:04.833798  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.898s (SENT)
I20260709 13:04:04.841233  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:04.856541  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.857817  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:04.859216  1648 raft_consensus.cc:2410] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
W20260709 13:04:04.859470  8417 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
W20260709 13:04:04.861018  8413 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:04:04.861192  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:04.863937  8421 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:04:04.852059  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:04.865772  8426 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:04.866292  8426 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.866721  8426 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:04.837966  8329 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:04.842612  8345 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.779084  8387 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:04.831794  8389 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:04.907385  8315 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:04:04.832195  8416 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:04.946417  1650 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:04.914103  8290 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:04.916976  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:04.878280  8329 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.912689  7866 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:04.926700  1642 raft_consensus.cc:1240] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:04:04.944409  1718 raft_consensus.cc:2468] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:04.954469  1648 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:04.958702  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 165209us
I20260709 13:04:04.883455  7902 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:04.950924  8420 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.268s	user 0.000s	sys 0.003s
W20260709 13:04:04.980710  8428 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:04:04.984858  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.179s (SENT) [suppressed 32 similar messages]
I20260709 13:04:04.965405  8329 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:04.995880  1649 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:04:04.980715  8418 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.448s	user 0.003s	sys 0.001s
W20260709 13:04:05.021428  8429 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:04:05.021070  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:04.997978  7866 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:05.030138  7866 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:05.031191  7866 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.034885  8416 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:05.038965  8387 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:05.040860  8297 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:05.042845  8386 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.005s
I20260709 13:04:05.043529  8386 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.046237  8386 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:05.067760  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:05.068981  2377 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:05.012857  8419 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 13:04:05.098937  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:05.099967  2376 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:05.106240  8224 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:05.106873  8224 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:05.115226  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:05.117699  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:05.119566  1642 raft_consensus.cc:3060] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.116772  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:05.069036  7866 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.133337  8410 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:05.160924  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:05.165380  8345 raft_consensus.cc:2764] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:05.168169  8252 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.168990  8252 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.171768  8252 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:05.168419  8401 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:05.173444  1717 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.171866  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.176261  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:05.177266  1718 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:05.186053  8399 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:05.201134  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.203634  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.204890  8224 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:05.206665  8224 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:05.213581  8382 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.214574  8382 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.218858  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.219986  1717 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:04:05.224509  8389 consensus_queue.cc:1175] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3b57321031416cba9db75450ef302a" responder_term: 22 status { last_received { term: 19 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 21. Current term is 22. Ops: [21.15-21.15]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:04:05.241714  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95604us
I20260709 13:04:05.245436  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:05.253962  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.750s (SENT)
W20260709 13:04:05.257496  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.258163  2377 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:05.275830  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:05.258767  1720 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
W20260709 13:04:05.149816  8336 consensus_meta.cc:287] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.760s	user 0.014s	sys 0.003s
I20260709 13:04:05.310777  1648 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:05.314005  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.317535  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:05.321696  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 413218us
I20260709 13:04:05.299038  2342 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:05.331816  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.332688  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.333202  1722 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
W20260709 13:04:05.338693  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.364923  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.365448  1573 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:04:05.267678  8410 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:05.363823  2379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:05.371605  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.375538  1569 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:04:05.376073  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.031s	sys 0.051s
I20260709 13:04:05.377444  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:05.377818  8193 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.379158  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:05.335472  8387 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:05.374819  2379 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:05.382210  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:05.426455  1572 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.383327  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:05.428947  1648 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:05.379999  1567 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:04:05.365231  8399 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:05.411800  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5151149079999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 13:04:05.438462  1539 connection.cc:256] RPC call timeout handler was delayed by -1.425114907999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
I20260709 13:04:05.380091  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:05.422773  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.439730  1718 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:05.433647  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:05.438927  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3666149079999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
W20260709 13:04:05.385365  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.031s	sys 0.051s
I20260709 13:04:05.387840  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:05.441145  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2559149079999656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 13:04:05.441656  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2469149079999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:04:05.442088  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2469149079999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:04:05.442727  1539 connection.cc:256] RPC call timeout handler was delayed by -1.115514907999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
I20260709 13:04:05.442988  1724 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:05.441146  1717 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:04:05.380321  8370 raft_consensus.cc:2804] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:05.300529  8336 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:05.456148  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68440.2us
I20260709 13:04:05.465314  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:05.434238  8178 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:05.456029  8387 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:05.479209  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3198) took 1016 ms. Trace:
I20260709 13:04:05.479708  8382 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:05.480827  8435 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:04:05.482085  8434 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:04:05.487459  8410 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:05.490705  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2886) took 1220 ms. Trace:
I20260709 13:04:05.492998  8370 raft_consensus.cc:697] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 18 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:05.494899  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:05.545763  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50953.6us
I20260709 13:04:05.424748  8193 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.498521  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:05.479210  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:05.468204  8315 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:05.503189  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:05.599992  1724 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:05.506208  1648 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:05.511696  1647 raft_consensus.cc:2368] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:04:05.503895  7914 raft_consensus.cc:993] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
W20260709 13:04:05.506968  8288 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.919s	user 0.015s	sys 0.002s
I20260709 13:04:05.504465  1725 rpcz_store.cc:276] 0709 13:04:04.462636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.462785 (+   149us) service_pool.cc:224] Handling call
0709 13:04:04.488235 (+ 25450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.627702 (+139467us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:04.729584 (+101882us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.947734 (+218150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.986103 (+ 38369us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.986364 (+   261us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:04.986417 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.986540 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.478915 (+492375us) raft_consensus.cc:1685] finished
0709 13:04:05.478950 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.479120 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43933,"prepare.queue_time_us":43012,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":81856,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":16139,"thread_start_us":262192,"threads_started":2,"wal-append.queue_time_us":583885}
I20260709 13:04:05.514240   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:05.530365  1572 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:04:05.541432  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.012s
W20260709 13:04:05.549274  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 50143.3us
I20260709 13:04:05.623744  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:05.597813  1617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.009s	sys 0.053s
I20260709 13:04:05.498754  8178 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:05.503294  8317 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.922s	user 0.015s	sys 0.000s
I20260709 13:04:05.511719  8399 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:05.612154  8316 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.029s	sys 0.033s
W20260709 13:04:05.623127  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.001s	sys 0.012s
W20260709 13:04:05.592767  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 94430.5us
I20260709 13:04:05.611032  8319 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:05.534533  1568 rpcz_store.cc:276] 0709 13:04:04.270613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.500910 (+230297us) service_pool.cc:224] Handling call
0709 13:04:04.521181 (+ 20271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.547459 (+ 26278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.547495 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.612055 (+ 64560us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.992006 (+379951us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:04.992081 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.992215 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.490396 (+498181us) raft_consensus.cc:1685] finished
0709 13:04:05.490421 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.490623 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42375,"raft.queue_time_us":38356,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":29462,"thread_start_us":329816,"threads_started":1,"wal-append.queue_time_us":799201}
I20260709 13:04:05.599301  8252 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:05.644538  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:05.618860  1716 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:05.507341  8110 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:04:05.600637  8416 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:05.621981  8426 consensus_meta.cc:287] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.755s	user 0.015s	sys 0.010s
I20260709 13:04:05.631784  8350 raft_consensus.cc:1081] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:04:05.684913  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2891) took 1262 ms. Trace:
I20260709 13:04:05.642017  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:05.649159  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.624723  1617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.009s	sys 0.053s
I20260709 13:04:05.610740  8315 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.650882  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2447) took 1123 ms. Trace:
W20260709 13:04:05.649696  8316 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.029s	sys 0.034s
I20260709 13:04:05.680195  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:05.649793  8417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.001s	sys 0.013s
W20260709 13:04:05.686980  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 45107us
W20260709 13:04:05.777812  1650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.008s	sys 0.009s
I20260709 13:04:05.642683  8399 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.688789  1647 rpcz_store.cc:276] 0709 13:04:04.527779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.544038 (+ 16259us) service_pool.cc:224] Handling call
0709 13:04:05.647650 (+1103612us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54001ba3d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:05.650781 (+  3131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170717056}
I20260709 13:04:05.626864  1648 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 21: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:05.685401  1576 rpcz_store.cc:276] 0709 13:04:04.422822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.617372 (+194550us) service_pool.cc:224] Handling call
0709 13:04:04.617840 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.666427 (+ 48587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.666480 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.705893 (+ 39413us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.943847 (+237954us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:04.944058 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.944186 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.684564 (+740378us) raft_consensus.cc:1685] finished
0709 13:04:05.684585 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.684818 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45909,"raft.run_cpu_time_us":2902,"raft.run_wall_time_us":90421,"thread_start_us":154009,"threads_started":1,"wal-append.queue_time_us":593937}
W20260709 13:04:05.693468  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.652274  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.012s	sys 0.008s
W20260709 13:04:05.782608  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.012s	sys 0.008s
I20260709 13:04:05.707584  8382 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:05.684092  8416 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.793325  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.794005  8416 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:05.794464  1647 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:05.795683  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.793316  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.708250  8315 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.798759  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.799485  1648 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 12.
I20260709 13:04:05.642802  8386 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:04:05.656939  8319 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:05.780300  8401 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.604s	user 0.014s	sys 0.012s
I20260709 13:04:05.723639  8193 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:05.724565  8178 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:05.729882  7866 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.005s
I20260709 13:04:05.731463  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.820665  1643 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:04:05.792065  8425 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:05.657670  7850 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.769074  2342 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.707322  8426 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.629199  8437 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.157s
I20260709 13:04:05.644249  8317 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.630882  8288 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:05.684614  8252 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:05.753118  8387 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:05.687330  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.781423  1650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.008s	sys 0.009s
W20260709 13:04:05.790308  2376 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.690s	user 0.020s	sys 0.000s
W20260709 13:04:05.801473  1688 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.519s (SENT)
W20260709 13:04:05.801992  1651 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.028s	sys 0.063s
I20260709 13:04:05.850040  2376 raft_consensus.cc:2410] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
W20260709 13:04:05.808979  1719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.060s	sys 0.053s
W20260709 13:04:05.809319  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 164825us
W20260709 13:04:05.851593  1719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.060s	sys 0.053s
W20260709 13:04:05.809480  1642 consensus_meta.cc:287] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.004s
I20260709 13:04:05.721858  8350 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:05.778522  8417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.001s	sys 0.014s
W20260709 13:04:05.782119  8329 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.003s
W20260709 13:04:05.862865  1645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.012s	sys 0.034s
W20260709 13:04:05.863386  1645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.012s	sys 0.034s
I20260709 13:04:05.864084  1723 raft_consensus.cc:2393] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 13: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:05.819070  2372 raft_consensus.cc:2410] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:04:05.740697  8370 consensus_queue.cc:237] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:05.782707  8399 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.801908  8382 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.829094  1647 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:04:05.831400  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.812604  8319 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.852065  1642 raft_consensus.cc:2468] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:04:05.850240  1651 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.028s	sys 0.063s
I20260709 13:04:05.865296  1643 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
W20260709 13:04:05.869068  1646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.005s	sys 0.017s
W20260709 13:04:05.889533  1646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.005s	sys 0.017s
I20260709 13:04:05.852067  8178 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:05.866619  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.009s	sys 0.009s
I20260709 13:04:05.894452  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:05.900746  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:05.901484  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.009s
I20260709 13:04:05.901722  1647 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:05.854242  8345 raft_consensus.cc:2764] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:04:05.908885  2378 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.026s	sys 0.033s
W20260709 13:04:05.897881  2372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41546 (request call id 761) took 1418 ms (client timeout 1542 ms). Trace:
W20260709 13:04:05.913690  1721 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.013s	sys 0.019s
W20260709 13:04:05.914276  1721 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.013s	sys 0.019s
W20260709 13:04:05.914000  2372 rpcz_store.cc:269] 0709 13:04:04.479255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.487578 (+  8323us) service_pool.cc:224] Handling call
0709 13:04:05.886668 (+1399090us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54005e2754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:05.897732 (+ 11064us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2981819648}
I20260709 13:04:05.915486  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:05.915930  2378 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.027s	sys 0.033s
I20260709 13:04:05.831421  7850 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:05.919205  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 117786us
I20260709 13:04:05.881448  8350 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:05.885118  8382 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:05.921180  2371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.004s	sys 0.028s
I20260709 13:04:05.941411  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.892853  8336 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:05.946139  2371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.230s	user 0.004s	sys 0.028s
I20260709 13:04:05.951640  2371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 765) took 1397 ms. Trace:
W20260709 13:04:05.952286  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81762.4us
W20260709 13:04:05.927016  2380 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.014s	sys 0.039s
I20260709 13:04:05.951774  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:05.945931  8427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.024s	sys 0.033s
I20260709 13:04:05.953682  1642 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:05.913919  8425 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:05.953872  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 699732us
I20260709 13:04:05.880748  8329 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:05.965740  2380 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.016s	sys 0.039s
I20260709 13:04:05.927122  8401 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:05.955204  8427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.024s	sys 0.033s
I20260709 13:04:06.001027  8426 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:06.009200  8288 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:06.011677  8329 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:06.014050  7850 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:06.015451  8224 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.808s	user 0.015s	sys 0.001s
I20260709 13:04:06.016825  8425 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.017737  8425 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:05.989078  8195 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.221s	user 0.019s	sys 0.001s
I20260709 13:04:06.021539  8195 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:06.027428  1571 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.012s	sys 0.055s
W20260709 13:04:06.029886  1571 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.055s
I20260709 13:04:06.029611  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.037217  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.025877  8401 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.067281  8401 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:06.067922  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.021549  8350 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.069703  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:05.952231  2371 rpcz_store.cc:276] 0709 13:04:04.554174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.567358 (+ 13184us) service_pool.cc:224] Handling call
0709 13:04:04.628166 (+ 60808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.661652 (+ 33486us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:04.675991 (+ 14339us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.951180 (+1275189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.951263 (+    83us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:05.951329 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.951411 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.951564 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20600,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"thread_start_us":1235088,"threads_started":1}
I20260709 13:04:06.071463  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2462) took 1118 ms. Trace:
I20260709 13:04:06.074337  1642 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 22: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:06.009284  8317 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:06.071959  1650 rpcz_store.cc:276] 0709 13:04:04.954373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.954559 (+   186us) service_pool.cc:224] Handling call
0709 13:04:04.977453 (+ 22894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.989564 (+ 12111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.989601 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.007921 (+ 18320us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.849616 (+841695us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:05.849662 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.849732 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.071196 (+221464us) raft_consensus.cc:1685] finished
0709 13:04:06.071220 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.071397 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1002,"raft.queue_time_us":10923,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":13383,"thread_start_us":828450,"threads_started":1,"wal-append.queue_time_us":1044408}
I20260709 13:04:06.085832  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.087029  1643 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 12.
W20260709 13:04:06.076167  8298 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.296s	user 0.014s	sys 0.005s
I20260709 13:04:06.095791  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:06.098533  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:06.102936  1650 raft_consensus.cc:2368] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
W20260709 13:04:06.104889  8442 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:04:06.112936  8195 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:06.107507  8298 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.117776  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:06.080114  8315 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.119251  1649 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:06.125747  8252 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:06.126375  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2464) took 1127 ms. Trace:
I20260709 13:04:06.126767  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2456) took 1264 ms. Trace:
I20260709 13:04:06.126845  1645 rpcz_store.cc:276] 0709 13:04:04.999136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.999350 (+   214us) service_pool.cc:224] Handling call
0709 13:04:05.017069 (+ 17719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.068058 (+ 50989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.068092 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.068779 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.864027 (+795248us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:05.864074 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.864156 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.126100 (+261944us) raft_consensus.cc:1685] finished
0709 13:04:06.126127 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.126295 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":794820,"threads_started":1,"wal-append.queue_time_us":1051683}
I20260709 13:04:06.127341  1644 rpcz_store.cc:276] 0709 13:04:04.862651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.877470 (+ 14819us) service_pool.cc:224] Handling call
0709 13:04:04.908306 (+ 30836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.943122 (+ 34816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.943157 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.964644 (+ 21487us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.783153 (+818509us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:05.793244 (+ 10091us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.793611 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.793702 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.126544 (+332842us) raft_consensus.cc:1685] finished
0709 13:04:06.126569 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.126706 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14527,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":35227,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":1896,"thread_start_us":722461,"threads_started":1,"wal-append.queue_time_us":1101362}
I20260709 13:04:06.134780  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.136157  1642 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:06.137522  8425 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:06.138754  8425 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:06.139321  8425 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:06.139726  8425 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:06.139886  8315 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:06.142725  8416 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.142669  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:06.172616  1575 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.003s	sys 0.011s
W20260709 13:04:06.174974  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.173161  1575 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.004s	sys 0.011s
W20260709 13:04:06.200901  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.010s	sys 0.007s
W20260709 13:04:06.154603  8278 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.014s	sys 0.017s
W20260709 13:04:06.207003  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.007s
I20260709 13:04:06.208849  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:06.209972  1725 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:06.207208  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.213140  1644 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:04:06.205562  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:06.172068  8302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.012s	sys 0.032s
I20260709 13:04:06.210230  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:06.218777  1723 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:06.208268  8278 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.014s	sys 0.018s
I20260709 13:04:06.206526  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:06.173157  8298 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:06.233278  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:06.234338  1724 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.238732  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.241290  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2465) took 1226 ms. Trace:
W20260709 13:04:06.218540  8302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.013s	sys 0.033s
I20260709 13:04:06.241887  1649 rpcz_store.cc:276] 0709 13:04:05.015515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.015781 (+   266us) service_pool.cc:224] Handling call
0709 13:04:06.241038 (+1225257us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000f64d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:06.241214 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520873856}
I20260709 13:04:06.235069  8319 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.248801  8319 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:06.249789  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:06.250309  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:06.252609  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3199) took 1789 ms. Trace:
I20260709 13:04:06.253064  1716 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:06.243970  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.255303  1645 raft_consensus.cc:2393] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:06.241438  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.253129  1719 rpcz_store.cc:276] 0709 13:04:04.462958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.463075 (+   117us) service_pool.cc:224] Handling call
0709 13:04:04.463402 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.521429 (+ 58027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.521463 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.141843 (+620380us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b2c000590a8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:04:05.142931 (+  1088us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.852078 (+709147us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:05.852125 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.852232 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.210964 (+358732us) raft_consensus.cc:1685] finished
0709 13:04:06.210996 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.252500 (+ 41504us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58080,"raft.queue_time_us":48998,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8295,"spinlock_wait_cycles":9412352,"thread_start_us":708652,"threads_started":1,"wal-append.queue_time_us":966745}
W20260709 13:04:06.262547  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.008s	sys 0.016s
W20260709 13:04:06.263175  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 583694us
I20260709 13:04:06.275594  1723 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:06.266829  1724 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:06.277696  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.278759  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:06.284199  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3197) took 1823 ms. Trace:
I20260709 13:04:06.287791  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.289659  1643 raft_consensus.cc:2368] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20.
I20260709 13:04:06.290802  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:06.268532  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2459) took 1404 ms. Trace:
I20260709 13:04:06.298633  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.284848  1721 rpcz_store.cc:276] 0709 13:04:04.460775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.461286 (+   511us) service_pool.cc:224] Handling call
0709 13:04:04.461923 (+   637us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:04.528596 (+ 66673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.528634 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:05.114822 (+586188us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b2c000590a8. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc832667a11 00007fc832677372 00007fc833da9a01 00007fc833d6e43a 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2
0709 13:04:05.115518 (+   696us) log.cc:844] Serialized 52 byte log entry
0709 13:04:05.914873 (+799355us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:05.939582 (+ 24709us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.940318 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.940724 (+   406us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.278707 (+337983us) raft_consensus.cc:1685] finished
0709 13:04:06.283826 (+  5119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.284101 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40498,"prepare.queue_time_us":24630,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":43359,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":22934,"spinlock_wait_cycles":12145920,"thread_start_us":798495,"threads_started":1,"wal-append.queue_time_us":1086804}
I20260709 13:04:06.301206  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.303027  1576 raft_consensus.cc:2410] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:04:06.262646  7914 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.018s
I20260709 13:04:06.322371  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:06.310842  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 386969us
I20260709 13:04:06.308487  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2469) took 1028 ms. Trace:
I20260709 13:04:06.323701  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.309970  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.312875  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.280611  8401 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:06.331192  1573 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:06.342738  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.343902  1642 raft_consensus.cc:2368] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:04:06.345317  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:06.346604  1642 raft_consensus.cc:2410] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
W20260709 13:04:06.342912  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 225124us
I20260709 13:04:06.349642  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:06.350725  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:06.320672  7914 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.000s	sys 0.019s
I20260709 13:04:06.340296  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.294926  1646 rpcz_store.cc:276] 0709 13:04:04.863742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.863932 (+   190us) service_pool.cc:224] Handling call
0709 13:04:04.864296 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.864620 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:05.242523 (+377903us) log.cc:844] Serialized 14 byte log entry
0709 13:04:06.263293 (+1020770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.267161 (+  3868us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:06.268136 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.268226 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.268441 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":787166,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":97114,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":7836,"thread_start_us":1317786,"threads_started":3,"wal-append.queue_time_us":856068}
I20260709 13:04:06.268412  8399 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.354728  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.365826  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:06.377504  1643 raft_consensus.cc:2368] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20.
I20260709 13:04:06.370323  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.378831  1569 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:06.384079  1642 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:06.385609  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.385846  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:06.436077  1644 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:06.330113  1567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:04:06.308394  8456 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:06.390588  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:06.444670  1569 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.328496  8315 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:06.396327  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:06.450590  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.450918  1568 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:06.397802  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.452070  1576 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:06.408285  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:06.452941  1721 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:06.412729  8278 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.366114  8401 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.418236  8350 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:06.369107  8316 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:06.324115  1648 rpcz_store.cc:276] 0709 13:04:05.268502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.800265 (+531763us) service_pool.cc:224] Handling call
0709 13:04:05.801045 (+   780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.810780 (+  9735us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:05.832062 (+ 21282us) log.cc:844] Serialized 14 byte log entry
0709 13:04:06.296412 (+464350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.296608 (+   196us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:06.296664 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.296729 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.296830 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3795,"prepare.queue_time_us":18723,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":64,"raft.queue_time_us":4552,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":12815,"thread_start_us":426438,"threads_started":1}
W20260709 13:04:06.433992  1647 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.003s
I20260709 13:04:06.480852  1647 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:06.484114  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:06.484814  1576 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.430722  8467 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.493970  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:06.495352  1723 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 22: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:06.434391  1571 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:06.436961  1649 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:06.447124  8416 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:06.448885  1644 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:06.454797  1643 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 9. (term mismatch)
I20260709 13:04:06.521342  1643 pending_rounds.cc:85] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 8
I20260709 13:04:06.521724  1643 pending_rounds.cc:107] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 19.9
I20260709 13:04:06.522083  1643 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:06.499192  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:06.456717  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:06.458142  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:06.525511  1719 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:06.475412  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:06.527120  1650 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 22: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:06.454370  8456 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:06.519629  1645 raft_consensus.cc:1275] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:06.536779  1646 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:06.477955  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 408216us
I20260709 13:04:06.547250  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:06.548065  2372 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:06.549729  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:06.550118  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:04:06.551777  1651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.025s	sys 0.048s
I20260709 13:04:06.548130  1721 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
W20260709 13:04:06.552450  1651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.026s	sys 0.048s
I20260709 13:04:06.591063  1648 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:04:06.593214  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:06.595129  1568 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:06.600878  1576 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:06.614367  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:06.619757  2371 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 22: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:06.529070  7850 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.528962  8224 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:06.476699  8315 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.549786  8345 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:06.634245  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.483486  8350 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.551951  8389 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:06.480268  8278 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.552125  8456 raft_consensus.cc:3060] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.671624  1719 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:06.596104  8468 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:06.673044  1569 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:04:06.528930  8316 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:06.675997  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 325170us
I20260709 13:04:06.540001  8467 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:06.703917  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 254129us
I20260709 13:04:06.706039  2372 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:06.638458  8224 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.638441  7850 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.753710  1647 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [14.9-16.10]
W20260709 13:04:06.776225  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.536s (SENT)
I20260709 13:04:06.788664  2344 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:06.799142  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:06.800273  2372 raft_consensus.cc:2368] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20.
I20260709 13:04:06.673861  8389 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:06.813090  8224 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:06.814267  7914 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:06.815003  7914 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.815506  7914 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.804903  8370 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:06.832391  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:06.830447  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2458) took 1967 ms. Trace:
I20260709 13:04:06.816298  7850 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:06.810701  1688 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.745s (SENT)
I20260709 13:04:06.850304  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:06.837213  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:06.824633  8459 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:04:06.806595  8461 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:06.884868  8461 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:06.864241  7850 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:06.865391  8316 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:06.889120  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.865453  8466 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 13:04:06.876008  8425 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.736s	user 0.009s	sys 0.004s
I20260709 13:04:06.851528  8370 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:06.868454  8389 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:06.871304  8483 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:04:06.878065  8315 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:06.865804  8463 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.383s	user 0.001s	sys 0.000s
W20260709 13:04:06.866482  8475 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:04:06.900704  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 401179us
W20260709 13:04:06.948529  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.914940  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60388.9us
I20260709 13:04:07.028683  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:06.865677  8480 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:04:07.020520  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 97493.1us
I20260709 13:04:07.034049  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:07.037496  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.833s (SENT)
I20260709 13:04:07.038231  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:06.970069  8382 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.007s	user 0.015s	sys 0.001s
I20260709 13:04:07.036722  2371 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:04:07.053944  1570 raft_consensus.cc:1275] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:04:07.044306  1616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.000s	sys 0.055s
I20260709 13:04:07.049507  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:07.059070  1616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.001s	sys 0.055s
I20260709 13:04:07.061066  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.058736  1690 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:06.888223  8461 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:07.019930  8484 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:04:07.067263  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.098662  8488 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:07.100818  8500 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:04:07.101810  1575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.002s	sys 0.010s
W20260709 13:04:07.127004  1575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.002s	sys 0.010s
W20260709 13:04:07.127524  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.119809  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:07.109567  1567 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:07.128692  1567 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.970467  7850 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:07.117167  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50031.2us
I20260709 13:04:06.967166  8386 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:07.063824  8459 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:04:07.130327  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.127976  1575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2902) took 1765 ms. Trace:
I20260709 13:04:07.064103  8315 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.021201  8425 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.110958  8465 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.143326  1575 rpcz_store.cc:276] 0709 13:04:05.362395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.362663 (+   268us) service_pool.cc:224] Handling call
0709 13:04:05.381420 (+ 18757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.381647 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:06.185154 (+803507us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.127510 (+942356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:07.127585 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:07.127640 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.127736 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.127904 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30908,"prepare.queue_time_us":862997,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":36,"raft.queue_time_us":619712,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"thread_start_us":1689377,"threads_started":3,"wal-append.queue_time_us":485246}
I20260709 13:04:07.119192  8387 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:07.163636  1642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.014s	sys 0.023s
W20260709 13:04:07.161358  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 102689us
I20260709 13:04:07.125936  8461 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:07.176394  1642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.015s	sys 0.023s
W20260709 13:04:07.177821  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 140315us
I20260709 13:04:07.132848  8505 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:07.142042  8488 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:07.110195  8382 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.146148  8386 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.228987  7850 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.230921  8386 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:07.127310  8389 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:04:07.159873  8491 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:07.200493  8317 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:07.200766  8315 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:07.251766  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.145s (SENT)
I20260709 13:04:07.233281  8387 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:07.282933  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 472064us
I20260709 13:04:07.239202  8437 raft_consensus.cc:2804] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:04:07.310276  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 179981us
I20260709 13:04:07.273041  8491 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:07.304466  8471 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
I20260709 13:04:07.316751  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:07.323760  1688 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.903s (SENT)
I20260709 13:04:07.318102  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1024) took 1258 ms. Trace:
I20260709 13:04:07.346423  8382 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:07.221370  8505 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:07.353507  8490 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:04:07.354972  8481 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.356076  8481 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.312752  8315 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:07.358083  8425 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:07.312778  8437 raft_consensus.cc:697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:07.363358  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 954) took 2232 ms. Trace:
I20260709 13:04:07.363898  2378 rpcz_store.cc:276] 0709 13:04:05.131268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.239611 (+108343us) service_pool.cc:224] Handling call
0709 13:04:05.240147 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.289358 (+ 49211us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:05.916657 (+627299us) log.cc:844] Serialized 14 byte log entry
0709 13:04:06.463219 (+546562us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:06.538314 (+ 75095us) log.cc:844] Serialized 26 byte log entry
0709 13:04:06.538513 (+   199us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:06.538556 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.538635 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.362888 (+824253us) raft_consensus.cc:1685] finished
0709 13:04:07.362911 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.363242 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26980,"prepare.queue_time_us":931907,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":63048,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":38233,"thread_start_us":1061809,"threads_started":2,"wal-append.queue_time_us":848313}
I20260709 13:04:07.303750  8387 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:07.333906  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.313059  8494 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:07.358839  8315 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:07.390354  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56586.9us
I20260709 13:04:07.396629  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2906) took 1119 ms. Trace:
I20260709 13:04:07.397213  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.319728  8329 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:07.408571  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 84805.2us
I20260709 13:04:07.413489  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.366830  8492 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:07.422269  1539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.011s	sys 0.058s
I20260709 13:04:07.397053  1572 rpcz_store.cc:276] 0709 13:04:06.276809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.277013 (+   204us) service_pool.cc:224] Handling call
0709 13:04:06.277384 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.277645 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:06.304635 (+ 26990us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.388776 (+1084141us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.393016 (+  4240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:07.393120 (+   104us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:07.393209 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.393323 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.396544 (+  3221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131013,"prepare.queue_time_us":32008,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":385886,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":3364,"thread_start_us":951892,"threads_started":3,"wal-append.queue_time_us":576765}
I20260709 13:04:07.385838  8467 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:07.422782  1539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.011s	sys 0.058s
W20260709 13:04:07.423295  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 624127us
I20260709 13:04:07.360713  8437 consensus_queue.cc:237] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.395977  8494 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:07.375872  1567 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:07.428906  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 309040us
I20260709 13:04:07.436388  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:07.434885  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.418507  8507 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.443488  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:07.444264  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 268402us
I20260709 13:04:07.449535  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.402976  8488 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.419458  8329 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.543610  1576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2912) took 1221 ms. Trace:
W20260709 13:04:07.481959  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.522198  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 87305us
I20260709 13:04:07.593633  1576 rpcz_store.cc:276] 0709 13:04:06.322480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.613032 (+290552us) service_pool.cc:224] Handling call
0709 13:04:06.615398 (+  2366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.034938 (+419540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.034992 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.133700 (+ 98708us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.369895 (+236195us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:07.369970 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.370077 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.543322 (+173245us) raft_consensus.cc:1685] finished
0709 13:04:07.543348 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.543532 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4777,"raft.queue_time_us":484107,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":50,"thread_start_us":576783,"threads_started":2,"wal-append.queue_time_us":371322}
I20260709 13:04:07.551698  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1021) took 1493 ms. Trace:
I20260709 13:04:07.439311  8465 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.552186  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 953) took 2422 ms. Trace:
I20260709 13:04:06.834812  1651 rpcz_store.cc:276] 0709 13:04:04.863398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.867028 (+  3630us) service_pool.cc:224] Handling call
0709 13:04:04.867408 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.867727 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:05.889098 (+1021371us) log.cc:844] Serialized 14 byte log entry
0709 13:04:06.829912 (+940814us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.829999 (+    87us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:06.830074 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.830179 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.830358 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":854922,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":242734,"spinlock_wait_cycles":24104192,"thread_start_us":1909623,"threads_started":3}
I20260709 13:04:07.427296  8517 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:07.551407  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2649) took 1539 ms. Trace:
I20260709 13:04:07.634366  1647 rpcz_store.cc:276] 0709 13:04:06.011566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.486359 (+474793us) service_pool.cc:224] Handling call
0709 13:04:06.486816 (+   457us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:06.776971 (+290155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.550997 (+774026us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54000ba754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:07.551308 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612542848}
W20260709 13:04:07.536827  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.527s (SENT)
I20260709 13:04:07.638078  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:07.538283  8470 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:04:07.630920  1642 rpcz_store.cc:276] 0709 13:04:06.058653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.512206 (+453553us) service_pool.cc:224] Handling call
0709 13:04:06.512658 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.185023 (+672365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.185066 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.185660 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.424373 (+238713us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:07.440663 (+ 16290us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.447370 (+  6707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.447567 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.551473 (+103906us) raft_consensus.cc:1685] finished
0709 13:04:07.551501 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.551636 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16022,"prepare.queue_time_us":9400,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":209816,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":58774,"thread_start_us":830640,"threads_started":2,"wal-append.queue_time_us":326401}
I20260709 13:04:07.438831  8481 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:07.445639  8336 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:07.653352  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.470649  8507 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.446171  8317 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.208s
I20260709 13:04:07.635478  2380 rpcz_store.cc:276] 0709 13:04:05.130413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.158601 (+ 28188us) service_pool.cc:224] Handling call
0709 13:04:05.164971 (+  6370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:06.042826 (+877855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.042881 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:06.070661 (+ 27780us) log.cc:844] Serialized 26 byte log entry
0709 13:04:06.452173 (+381512us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:06.452217 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.452323 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.551934 (+1099611us) raft_consensus.cc:1685] finished
0709 13:04:07.551959 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.552113 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78859,"raft.queue_time_us":99674,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":53232,"thread_start_us":1056782,"threads_started":2,"wal-append.queue_time_us":1439954}
W20260709 13:04:07.642524  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 197833us
I20260709 13:04:07.449242  8382 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:07.537413  8503 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:04:07.547672  8319 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:07.487416  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.543958  7914 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.728s	user 0.016s	sys 0.000s
I20260709 13:04:07.432603  8467 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.539244  8298 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.662712  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:07.524581  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52467.5us
I20260709 13:04:07.666206  8527 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:07.667178  8456 consensus_meta.cc:287] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.986s	user 0.015s	sys 0.001s
I20260709 13:04:07.639829  8517 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.481352  8494 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.673822  8110 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.663337  8481 raft_consensus.cc:2764] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:07.714324  8110 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.717029  8110 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:07.673815  8382 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:07.721836  8382 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.649474  8465 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.727041  8465 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:07.727272  8399 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:07.658960  8507 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:07.693385  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51817.7us
W20260709 13:04:07.709688  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47076.2us
I20260709 13:04:07.710750  8456 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.711773  8517 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.712867  8494 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.713507  8401 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:07.693543  8319 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.447651  8224 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:07.707309  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 263864us
W20260709 13:04:07.737599  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64412us
I20260709 13:04:07.733749  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:07.737653  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 200817us
I20260709 13:04:07.738530  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.528664  8387 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.529143  8491 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:07.751426  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.762879  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.764438  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.744684  8528 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.767166  8456 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:07.768019  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.710750  8527 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.750218  8224 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:07.755173  8514 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.734706  7914 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.769702  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.777349  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.786302  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:07.787362  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.780736  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2913) took 1452 ms. Trace:
I20260709 13:04:07.788875  1568 rpcz_store.cc:276] 0709 13:04:06.328470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.617389 (+288919us) service_pool.cc:224] Handling call
0709 13:04:06.641850 (+ 24461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.021123 (+379273us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:07.171078 (+149955us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.642007 (+470929us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.656671 (+ 14664us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.656970 (+   299us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:07.657032 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.657183 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.780478 (+123295us) raft_consensus.cc:1685] finished
0709 13:04:07.780506 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.780674 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1803,"prepare.queue_time_us":276268,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":428109,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":9381,"thread_start_us":654766,"threads_started":3,"wal-append.queue_time_us":446801}
W20260709 13:04:07.772255  8510 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:07.807646  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:07.774374  8502 raft_consensus.cc:2764] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:04:07.780581  7914 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:07.777716  8298 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:07.773178  8317 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.776604  8519 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:07.814330  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 117913us
I20260709 13:04:07.775871  8527 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:07.809630  8315 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.823066  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.834801  1647 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:04:07.835435  1647 pending_rounds.cc:85] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 14
I20260709 13:04:07.835855  1647 pending_rounds.cc:107] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:04:07.836586  1647 raft_consensus.cc:2889] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.816679  8389 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:07.842598  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:07.845499  1651 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.846323  8520 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:07.848522  8530 consensus_queue.cc:1048] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:04:07.821601  8298 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:07.842598  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:07.842912  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78581.7us
W20260709 13:04:07.854094  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 89510.2us
I20260709 13:04:07.870879  8493 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.840034  8467 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:07.875026  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:07.879091  1568 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:07.881203  1570 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:07.845129  8389 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.852320  8502 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:04:07.886091  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63151us
I20260709 13:04:07.854607  8298 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.886812  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2914) took 1521 ms. Trace:
I20260709 13:04:07.891486  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:07.892678  1576 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
W20260709 13:04:07.877158  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 89871us
I20260709 13:04:07.892357  1569 rpcz_store.cc:276] 0709 13:04:06.365796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.684300 (+318504us) service_pool.cc:224] Handling call
0709 13:04:06.692408 (+  8108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.106325 (+413917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.106375 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.107202 (+   827us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.110409 (+  3207us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:07.110485 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.110587 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.886519 (+775932us) raft_consensus.cc:1685] finished
0709 13:04:07.886553 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.886726 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60634,"raft.queue_time_us":385243,"raft.run_cpu_time_us":40,"raft.run_wall_time_us":40,"thread_start_us":304551,"threads_started":2,"wal-append.queue_time_us":534444}
I20260709 13:04:07.897526  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:07.885516  7914 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.902567  8467 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:07.903388  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.892453  8502 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.904517  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:07.904932  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51657.6us
I20260709 13:04:07.907193  8507 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.888797  8389 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:04:07.919188  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 131306us
I20260709 13:04:07.926275  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:07.844661  8514 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:04:07.915045  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:07.924170  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:07.926278  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:07.929144  1572 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:07.931742  1568 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.931912  1576 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.918519  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:07.900681  8319 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:07.936147  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:07.935158  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:07.937652  1572 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:07.939445  1575 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:07.917948  8481 raft_consensus.cc:2764] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:04:07.943049  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:07.948382  1576 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:07.959995  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.962266  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:07.918316  8502 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.974229  8288 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:07.974901  8288 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.975361  8288 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.977164  8426 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.977730  8426 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.978111  8426 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:07.968614  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 75593us
I20260709 13:04:07.983310  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:07.984226  1575 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.917043  7914 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:07.963409  8319 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:07.973196  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:07.947950  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:07.990002  1651 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:04:07.942131  8530 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:07.948092  8488 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.994731  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:08.001675  1650 raft_consensus.cc:2468] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:07.993588  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:08.003167  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.005836  1574 raft_consensus.cc:2393] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 19: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:08.033142  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.034464  1647 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:08.003757  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:07.989573  8476 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 1.269s	user 0.000s	sys 0.003s
I20260709 13:04:08.045179  1576 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 13.
I20260709 13:04:08.040274  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.046340  1645 raft_consensus.cc:2368] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20.
I20260709 13:04:08.047001  1651 raft_consensus.cc:3060] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:08.048280  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 91542us
W20260709 13:04:08.048544  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:08.053102  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88738.9us
I20260709 13:04:08.066548  1572 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:08.074100  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:08.082769  1570 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:08.084504  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:08.085460  1570 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:08.101671  1651 raft_consensus.cc:2468] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:04:08.102809  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:08.045680  8252 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:08.107892  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134763us
I20260709 13:04:08.107188  1574 raft_consensus.cc:2410] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:04:08.124847  8481 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:08.126839  8481 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:08.142602  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:08.127163  1567 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:08.127727  8540 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:04:08.131319  8224 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:08.140286  1721 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:08.141777  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.144143  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.144490  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.145658  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.037703  8461 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 13:04:08.177806  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.069643  8467 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:04:08.184324  1723 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:08.199010  1720 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:08.057047  8517 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.188150  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44760 (request call id 2909) took 1857 ms. Trace:
I20260709 13:04:08.177806  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:08.179683  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.178123  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.201767  1724 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:04:08.208557  1567 rpcz_store.cc:276] 0709 13:04:06.300881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.302627 (+  1746us) service_pool.cc:224] Handling call
0709 13:04:08.125852 (+1823225us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54000c2954. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d65a3d 00007fc838d35cd0 00007fc8322ba254 00007fc8322ba1e2 00007fc8322ba111 00007fc8322ba08c 00007fc8322b9342 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:08.150602 (+ 24750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545869696}
I20260709 13:04:08.200197  1570 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:08.186173  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:08.214178  1574 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
W20260709 13:04:08.195974  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 53551.5us
I20260709 13:04:08.217559  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:08.225430  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.107453  8527 consensus_queue.cc:1059] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:04:08.226841  1723 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:04:08.226349  1717 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:08.199427  8350 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:08.228912  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:08.231844  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:08.233040  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.235581  1574 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:08.240962  1717 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:08.241219  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 143107us
W20260709 13:04:08.239017  8518 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:04:08.215726  8481 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:08.236255  2380 raft_consensus.cc:1275] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:08.237787  1724 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:08.242426  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:04:08.243549  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 3.179s (SENT)
I20260709 13:04:08.253440  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:08.282387  8512 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:08.283090  8512 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:08.253896  8530 raft_consensus.cc:3055] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:08.249511  8500 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:08.293704  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50176.8us
I20260709 13:04:08.252975  8481 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:08.281536  1688 leader_election.cc:400] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:04:08.307476  1688 leader_election.cc:403] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:08.309667  1723 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:08.311621  1717 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:08.270779  8437 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:08.268682  8514 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.287873  8389 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:08.292683  8530 raft_consensus.cc:740] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:08.318629  8437 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:08.363708  8509 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:08.369133  8549 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.236s	user 0.000s	sys 0.008s
I20260709 13:04:08.363719  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2921) took 1331 ms. Trace:
I20260709 13:04:08.371696  8389 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:08.370635  1573 rpcz_store.cc:276] 0709 13:04:07.032010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.055469 (+ 23459us) service_pool.cc:224] Handling call
0709 13:04:07.056021 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.056346 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:07.982885 (+926539us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b180036a780. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc8377663c7 00007fc833d789a7 00007fc833d880bf
0709 13:04:07.983044 (+   159us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.363184 (+380140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.363261 (+    77us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:08.363325 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.363430 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.363626 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60973,"prepare.queue_time_us":341715,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2140,"spinlock_wait_cycles":8603008,"thread_start_us":654707,"threads_started":3,"wal-append.queue_time_us":371816}
I20260709 13:04:08.323904  8315 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:04:08.333714  8520 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:08.376708  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95373.9us
I20260709 13:04:08.378935  1568 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:08.380587  1575 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:08.379972  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:08.327085  8507 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:08.338356  7914 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:08.399559  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.284216  8512 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.346148  8555 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18.
I20260709 13:04:08.383272  8437 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:08.405272  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 173507us
I20260709 13:04:08.407824  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.411228  1718 raft_consensus.cc:2376] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20. Re-sending same reply.
I20260709 13:04:08.411978  1573 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:08.391901  8558 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.417484  8426 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.419410  8505 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:08.339977  8288 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.421484  8426 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:08.422040  8288 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:08.422245  8298 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.002s
I20260709 13:04:08.425333  8540 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:08.425850  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:08.426762  8527 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:08.427160  8527 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:08.427276  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.427956  1718 raft_consensus.cc:2376] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19. Re-sending same reply.
I20260709 13:04:08.428121  8527 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.429488  1720 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:08.422577  1716 raft_consensus.cc:2368] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:04:08.431283  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.410800  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:08.437845  1575 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:08.439422  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:08.423408  8298 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.447499  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:08.448470  2380 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.392282  8425 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:08.455842  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:08.453277  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.457135  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.403110  8489 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:08.462426  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:08.462100  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.463148  1725 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:08.463411  1718 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:04:08.463488  1725 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:08.468492  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:08.473265  1723 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:08.459885  7914 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:08.496083  1539 connection.cc:256] RPC call timeout handler was delayed by -2.8563181979999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:04:08.497365  1539 connection.cc:256] RPC call timeout handler was delayed by -2.8248181980000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:04:08.498337  1539 connection.cc:256] RPC call timeout handler was delayed by -2.8014181980000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 13:04:08.499128  1539 connection.cc:256] RPC call timeout handler was delayed by -2.787918198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 13:04:08.500219  1539 connection.cc:256] RPC call timeout handler was delayed by -2.6808181979999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:04:08.501705  1539 connection.cc:256] RPC call timeout handler was delayed by -2.6691181980000351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260709 13:04:08.502735  1539 connection.cc:256] RPC call timeout handler was delayed by -2.6088181979999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:04:08.503386  1539 connection.cc:256] RPC call timeout handler was delayed by -2.6025181980000411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260709 13:04:08.503903  1539 connection.cc:256] RPC call timeout handler was delayed by -2.5953181980000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 13:04:08.504582  1539 connection.cc:256] RPC call timeout handler was delayed by -2.5044181979999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
I20260709 13:04:08.487890  8530 consensus_queue.cc:260] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.506791  8298 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:08.507526  8500 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:08.509142  8550 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:04:08.510525  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54837.7us
I20260709 13:04:08.446641  8345 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:08.527010  1719 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [14.9-16.10]
W20260709 13:04:08.513195  1539 connection.cc:256] RPC call timeout handler was delayed by -2.4891181980000283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
I20260709 13:04:08.461938  8426 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:04:08.495013  8514 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:08.488262  8555 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:04:08.530961  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:08.539196  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 99875.4us
I20260709 13:04:08.553555  1573 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:08.557348  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44891.4us
W20260709 13:04:08.544153  1539 connection.cc:256] RPC call timeout handler was delayed by -2.4288181980000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
I20260709 13:04:08.492830  8557 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:08.548909  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:08.549353  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99027.5us
I20260709 13:04:08.530594  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:08.494750  8517 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:08.553892  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:08.543018  8489 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:08.560518  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3622181980000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
W20260709 13:04:08.570894  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3586181980000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 13:04:08.571297  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3343181980000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 13:04:08.571599  1539 connection.cc:256] RPC call timeout handler was delayed by -2.3280181979999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 13:04:08.571892  1539 connection.cc:256] RPC call timeout handler was delayed by -2.2947181979999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
W20260709 13:04:08.572173  1539 connection.cc:256] RPC call timeout handler was delayed by -2.27311819800002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 13:04:08.572504  1539 connection.cc:256] RPC call timeout handler was delayed by -2.2362181980000173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 13:04:08.572791  1539 connection.cc:256] RPC call timeout handler was delayed by -2.2308181980000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 13:04:08.573077  1539 connection.cc:256] RPC call timeout handler was delayed by -2.0940181980000148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2388s for a response
W20260709 13:04:08.573391  1539 connection.cc:256] RPC call timeout handler was delayed by -1.912218198000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.259s for a response
W20260709 13:04:08.573680  1539 connection.cc:256] RPC call timeout handler was delayed by -1.5720181980000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2968s for a response
W20260709 13:04:08.573976  1539 connection.cc:256] RPC call timeout handler was delayed by -1.436118198000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31190000000000007s for a response
W20260709 13:04:08.574275  1539 connection.cc:256] RPC call timeout handler was delayed by -1.2525181980000184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33230000000000004s for a response
I20260709 13:04:08.532121  8500 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.591339  8500 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:08.550909  8530 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:08.550232  8514 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.605801  2380 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
W20260709 13:04:08.609845  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58225.1us
I20260709 13:04:08.576967  8489 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.607475  8514 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:08.576696  8557 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:08.641130  8489 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:08.686474  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.688263  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:08.648312  8556 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:04:08.651157  8566 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:04:08.656677  8345 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:08.657421  8502 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.682s	user 0.014s	sys 0.000s
W20260709 13:04:08.664463  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 106092us
I20260709 13:04:08.673404  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.678208  8568 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.679989  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.681073  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.682301  1645 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:04:08.683037  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.683863  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.647063  8559 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.689162  1646 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:04:08.689495  1573 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:08.705062  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:08.741580  8566 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:08.740641  8502 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:08.744549  1570 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:04:08.691207  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 116426us
I20260709 13:04:08.745967  8568 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:08.680045  8557 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:08.747709  8502 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:08.742825  8566 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:08.748569  1648 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:08.749557  8566 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:08.742192  8559 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:08.754194  1647 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:04:08.754845  8559 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:08.740628  1644 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:08.741698  1650 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:08.748057  8568 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:08.764006  8530 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:08.764293  8559 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:08.764549  8568 consensus_queue.cc:1048] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:08.767836  8345 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:08.776389  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 201255us
I20260709 13:04:08.777501  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:08.778369  8502 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.778834  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:08.779785  8489 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.782356  8489 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:08.780018  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:08.788053  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 234169us
I20260709 13:04:08.790568  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:08.791392  2372 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:08.791661  1644 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:08.795037  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:08.796330  2380 raft_consensus.cc:2410] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:08.798799  1648 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:08.810699  2372 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:08.811921  1688 leader_election.cc:400] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20.
I20260709 13:04:08.812381  1688 leader_election.cc:403] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:04:08.821902  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:08.786738  8502 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:08.692498  7914 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:08.801168  8557 raft_consensus.cc:2764] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:04:08.833429  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.834069  1651 raft_consensus.cc:2376] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18. Re-sending same reply.
I20260709 13:04:08.834808  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.835378  1651 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 21: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:08.836047  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.837314  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.837819  1648 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:08.785270  8437 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.839359  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.839916  1647 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:08.840734  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.838743  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.843542  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.845567  1651 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:08.842814  1646 raft_consensus.cc:2376] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21. Re-sending same reply.
I20260709 13:04:08.849151  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.850384  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.851266  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.852720  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:08.863984  1717 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:08.865360  2372 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:04:08.883051  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:08.843415  8500 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:08.838764  8577 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:08.883765  1646 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:09.156481  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.084867  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 290247us
I20260709 13:04:09.203569  2342 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:08.845638  8576 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20.
I20260709 13:04:09.198141  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:09.208247  1644 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
W20260709 13:04:09.206058  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 395003us
I20260709 13:04:09.207767  1651 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:09.211434  1651 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:04:09.218108  1651 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:09.218217  1644 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:04:08.850212  8345 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:08.853477  8517 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.012096  8499 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.045678  8437 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:07.334627  1643 rpcz_store.cc:276] 0709 13:04:06.060273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.550536 (+490263us) service_pool.cc:224] Handling call
0709 13:04:06.621693 (+ 71157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.621981 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:07.108089 (+486108us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.317493 (+209404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:07.317594 (+   101us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:07.317688 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.317804 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.318005 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2145,"prepare.queue_time_us":485675,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":2602,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":9592,"thread_start_us":661021,"threads_started":2}
I20260709 13:04:09.240684  1643 raft_consensus.cc:1275] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:04:09.249147  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 47879.4us
W20260709 13:04:09.250065  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46573.7us
I20260709 13:04:09.240187  8345 raft_consensus.cc:2804] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:09.252034  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.252290  1723 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:09.253072  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.222241  8577 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:09.236806  8517 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:08.897390  7914 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.263023  2372 raft_consensus.cc:2410] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:04:09.264637  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:09.266970  2372 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:09.263593  8345 raft_consensus.cc:697] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:09.268574  8345 consensus_queue.cc:237] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:09.269066  8195 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.277100  8195 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.279487  8195 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.288172  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:09.289004  2372 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:09.288916  1646 raft_consensus.cc:2468] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:09.294277  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:09.240065  8499 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.302297  1644 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:04:09.309405  1644 pending_rounds.cc:85] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 10
I20260709 13:04:09.309679  1644 pending_rounds.cc:107] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:04:09.309967  1644 raft_consensus.cc:2889] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.349911  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:09.302675  8500 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.358676  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.362149  8278 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:09.362409  8577 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.363948  8517 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:09.364941  8585 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:04:09.366186  8581 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:04:09.364874  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 111478us
I20260709 13:04:09.367496  2375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.367929  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:09.368196  2375 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:09.369580  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.369776  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:09.370857  8599 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.371565  8517 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:09.371409  8599 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.372232  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.372831  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:09.376693  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3230) took 1373 ms. Trace:
I20260709 13:04:09.377183  1570 rpcz_store.cc:276] 0709 13:04:08.002923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.214351 (+211428us) service_pool.cc:224] Handling call
0709 13:04:08.214776 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.745009 (+530233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.376393 (+631384us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54004209d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:09.376622 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217394944}
I20260709 13:04:09.378001  8599 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.380674  8577 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:09.385941  8387 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.386693  8387 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:09.390015  8514 consensus_meta.cc:287] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
I20260709 13:04:09.390553  8514 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:09.392083  8469 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.392640  8469 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:09.393486  2376 raft_consensus.cc:2393] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:09.394354  8514 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.391757  1725 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.407979  1725 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.408586  8387 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.394732  8469 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:09.395547  8481 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.409631  8481 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.410120  8329 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:09.411237  8500 raft_consensus.cc:2764] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:04:09.397925  2373 pending_rounds.cc:85] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Aborting all ops after (but not including) 12
I20260709 13:04:09.412590  2373 pending_rounds.cc:107] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:04:09.412933  2373 raft_consensus.cc:2889] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:09.413235  8552 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.205s	user 0.006s	sys 0.011s
W20260709 13:04:09.413816  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.670s (SENT)
I20260709 13:04:09.362519  8499 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:09.372267  8517 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:09.414338  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:09.411988  8500 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:09.434075  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 135416us
I20260709 13:04:09.435487  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:09.391140  8590 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:09.441783  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:09.434075  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:09.417104  8469 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:09.457226  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.713s (SENT)
I20260709 13:04:09.457259  8569 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:09.463541  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:09.466230  1725 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:09.467357  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55262 (request call id 2732) took 1013 ms. Trace:
I20260709 13:04:09.442556  8387 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:09.467756  1725 rpcz_store.cc:276] 0709 13:04:08.454043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.461764 (+  7721us) service_pool.cc:224] Handling call
0709 13:04:09.391372 (+929608us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b4c00206c40. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc83203aa20 00007fc83204ba29 00007fc832011644 00007fc832038f2d 00007fc832038b93 00007fc833d6918b 00007fc833d69cb5 00007fc833d64f30 00007fc833d667de 00007fc833d7227c 00007fc838d33b17
0709 13:04:09.467288 (+ 75916us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101294592}
I20260709 13:04:09.457517  8500 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.440315  8499 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:04:09.466944  8569 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.476248  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:09.455422  8590 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:09.516047  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:09.515358  1721 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:04:09.518123  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.470273  8589 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:09.509397  8468 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.680s	user 0.004s	sys 0.030s
I20260709 13:04:09.522289  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:09.474345  8500 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.472772  8595 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:09.530121  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:09.530787  1722 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:09.513562  8569 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.474206  8517 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 13:04:09.526433  8468 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.702s	user 0.005s	sys 0.031s
I20260709 13:04:09.490056  8195 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:04:09.541325  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:09.541889  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.541975  1645 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:09.529160  8370 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:04:09.553045  1722 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:04:09.554586  2342 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:09.555778  8469 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:09.556298  8469 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:09.557248  8469 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:09.561081  1645 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:09.530639  8595 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:09.567111  8590 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:09.542869  8589 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:09.539878  8568 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:09.570591  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62248us
I20260709 13:04:09.572901  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:09.573750  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:09.574857  8589 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:09.575855  8425 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:09.575906  8589 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.576581  8425 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.577332  8528 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:09.577845  8528 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:09.578830  8528 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.541414  8468 raft_consensus.cc:697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:09.581694  8468 consensus_queue.cc:237] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:09.583033  8553 log.cc:927] Time spent T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.385s	user 0.008s	sys 0.005s
I20260709 13:04:09.583133  1649 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:09.592023  1648 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:04:09.593065  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:09.597183  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:09.597914  1648 raft_consensus.cc:2376] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13. Re-sending same reply.
W20260709 13:04:09.609033  2344 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.869s (SENT)
I20260709 13:04:09.618585  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:09.625597  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 255807us
I20260709 13:04:09.571086  8590 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:09.569417  8595 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:09.631829  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:09.647106  1648 raft_consensus.cc:2368] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20.
I20260709 13:04:09.632416  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:09.642853  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:09.649228  1575 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.644019  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:09.650269  1572 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.644747  1573 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:09.645179  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:09.653743  1576 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.656935  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:09.657697  1568 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:09.625129  8568 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:09.652685  1573 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:09.741659  1573 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:09.691733  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:09.695722  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.747030  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59632.4us
I20260709 13:04:09.748145  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:09.749825  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:09.751433  1576 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:09.751899  1688 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:09.752234  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.755798  1572 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:04:09.625916  8401 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
W20260709 13:04:09.760293  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 131252us
I20260709 13:04:09.761330  1614 leader_election.cc:400] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:04:09.761713  1614 leader_election.cc:403] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:09.770361  1568 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:04:09.773267  1575 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:09.801301  1572 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:04:09.804220  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 177547us
W20260709 13:04:09.804780  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.804520  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 52666.2us
I20260709 13:04:09.805167  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:09.797019  1573 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.812392  1573 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.631191  8598 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:09.807569  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:09.824378  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:09.645746  8600 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:09.828699  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:09.836834  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:09.647783  8590 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:09.650154  8195 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.168s
I20260709 13:04:09.790973  8499 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:09.669629  8601 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:09.754707  8512 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:09.880116  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:09.873858  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:09.882812  1575 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:09.804473  8567 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.826134  8519 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:09.889474  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:09.823805  8437 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:09.892895  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:09.917511  1572 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:09.917696  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:09.870873  8565 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:09.841044  8425 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.875391  8278 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.877077  8496 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:09.926295  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:09.926470  1688 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:09.929764  1688 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:09.931802  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:09.940871  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:09.941715  1575 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:09.854534  8600 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:09.950377  8594 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:04:09.959712  1642 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:04:09.951220  8610 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:09.952291  1646 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:09.946992  7914 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:09.888940  8464 consensus_queue.cc:1059] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:04:09.947211  8481 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.002s
I20260709 13:04:09.947594  8319 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:09.946322  8565 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
W20260709 13:04:09.960266  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:09.987732  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.937970  8519 raft_consensus.cc:2804] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:09.930069  8468 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:09.938257  8601 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:09.933503  8500 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:09.968083  8598 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:09.973279  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 212058us
I20260709 13:04:10.016587  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.017364  1725 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:10.017920  1576 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:04:09.974174  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 169019us
I20260709 13:04:10.016587  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:09.976707  1646 raft_consensus.cc:1275] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:10.019851  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:09.924623  8110 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:04:09.895238  8567 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:09.986825  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.203s (SENT) [suppressed 24 similar messages]
I20260709 13:04:09.939193  8591 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20.
I20260709 13:04:09.952457  8499 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:09.931380  8512 consensus_queue.cc:1048] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:04:09.924697  8437 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:10.007715  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 247443us
I20260709 13:04:09.930238  8278 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:10.010191  1651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.007s	sys 0.014s
I20260709 13:04:09.977790  1573 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:04:10.015594  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:10.042809  1651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.007s	sys 0.014s
I20260709 13:04:10.042884  1573 pending_rounds.cc:85] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 8
I20260709 13:04:10.043329  1573 pending_rounds.cc:107] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:04:10.043603  1573 raft_consensus.cc:2889] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.043955  1573 pending_rounds.cc:107] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:04:09.982146  8504 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:10.017379  1721 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:09.938508  8612 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:04:09.981534  8302 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:10.031004  8603 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:04:10.024564  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:09.923982  8568 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.006116  8496 raft_consensus.cc:3055] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:10.026949  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.027755  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.031388  1646 pending_rounds.cc:85] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 10
I20260709 13:04:09.987066  7914 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.044209  1573 raft_consensus.cc:2889] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.021938  8500 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.046955  2378 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:10.050866  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.025893  8481 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.101768  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2495) took 1272 ms. Trace:
I20260709 13:04:10.090444  1646 pending_rounds.cc:107] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:04:10.102384  1646 raft_consensus.cc:2889] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.029249  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:10.097921  8604 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:04:10.099138  8500 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.099589  8278 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:10.094818  8596 log.cc:927] Time spent T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:04:10.100783  1539 leader_election.cc:400] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
I20260709 13:04:10.040199  8601 raft_consensus.cc:2764] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:04:10.044972  8590 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.089536  8496 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:10.010890  8468 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.051281  8514 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:10.102172  1647 rpcz_store.cc:276] 0709 13:04:08.829786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.202092 (+372306us) service_pool.cc:224] Handling call
0709 13:04:09.216703 (+ 14611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.216958 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.216987 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.228581 (+ 11594us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.523245 (+294664us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:09.750342 (+227097us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.823549 (+ 73207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.823630 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.101609 (+277979us) raft_consensus.cc:1685] finished
0709 13:04:10.101625 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.101719 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72930,"prepare.queue_time_us":44826,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":166721,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5347,"thread_start_us":494536,"threads_started":2,"wal-append.queue_time_us":357335}
I20260709 13:04:10.112298  1539 leader_election.cc:403] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:04:10.105108  1717 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:04:10.118271  8614 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:04:10.119035  8609 log.cc:927] Time spent T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:04:10.045176  8499 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.155294  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2731) took 1701 ms. Trace:
I20260709 13:04:10.143442  8567 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.144349  8601 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
W20260709 13:04:10.144405  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 94463.2us
I20260709 13:04:10.145838  8600 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:10.114283  8608 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:04:10.155766  1719 rpcz_store.cc:276] 0709 13:04:08.453603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.453892 (+   289us) service_pool.cc:224] Handling call
0709 13:04:08.463579 (+  9687us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:08.527558 (+ 63979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.154986 (+1627428us) spinlock_profiling.cc:243] Waited 63.1 ms on lock 0x7b5400087d54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:10.155228 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145028480}
I20260709 13:04:10.173130  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:10.089905  8504 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.095978  8302 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.171217  1572 raft_consensus.cc:2410] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:04:10.107775  8481 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.131129  8519 raft_consensus.cc:697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:10.207085  8624 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:04:10.208963  8606 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:04:10.214883  8481 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.127616  8598 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.160730  8514 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.188930  8615 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:10.223960  1719 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:04:10.231137  8627 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:04:10.235019  8468 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.237882  8527 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:10.239943  8569 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.000s
I20260709 13:04:10.224488  8500 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.242204  8514 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.244253  8437 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:10.245807  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.225217  8519 consensus_queue.cc:237] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.179200  8496 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:10.247787  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 85781us
I20260709 13:04:10.257277  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:10.258390  8603 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:10.264714  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 967) took 1475 ms. Trace:
I20260709 13:04:10.265187  1724 rpcz_store.cc:276] 0709 13:04:08.789559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.789700 (+   141us) service_pool.cc:224] Handling call
0709 13:04:08.810125 (+ 20425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:09.212754 (+402629us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:09.359114 (+146360us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.632659 (+273545us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:09.649059 (+ 16400us) log.cc:844] Serialized 52 byte log entry
0709 13:04:09.649203 (+   144us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:09.649551 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.649734 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.651044 (+  1310us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.264431 (+613387us) raft_consensus.cc:1685] finished
0709 13:04:10.264452 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.264622 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33813,"prepare.queue_time_us":230245,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":8344,"raft.queue_time_us":56163,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":45877,"thread_start_us":661190,"threads_started":3,"wal-append.queue_time_us":396971}
I20260709 13:04:10.236235  8504 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:10.254707  8499 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:10.266593  8500 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:10.277005  2344 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.523s (SENT)
I20260709 13:04:10.277712  2344 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.274281  8496 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.288100  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:10.292006  8329 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:10.292491  8329 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:10.293367  8329 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.294718  8329 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:10.296974  8464 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:04:10.297499  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.297431  8464 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:10.302932  8464 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.304509  8464 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.305665  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:10.306583  8615 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:10.307094  1539 leader_election.cc:400] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20.
I20260709 13:04:10.307315  8615 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:10.281801  8514 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.308153  1539 leader_election.cc:400] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:04:10.309938  1539 leader_election.cc:403] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:10.310730  8615 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.311200  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:10.316016  8555 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:04:10.317458  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:10.318825  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2504) took 1476 ms. Trace:
I20260709 13:04:10.319198  1644 rpcz_store.cc:276] 0709 13:04:08.842726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.219286 (+376560us) service_pool.cc:224] Handling call
0709 13:04:09.237966 (+ 18680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.310399 (+ 72433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.318597 (+1008198us) spinlock_profiling.cc:243] Waited 63.9 ms on lock 0x7b54000ba254. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:10.318771 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147049728}
I20260709 13:04:10.320863  8567 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:10.322606  8468 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:10.323262  8468 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.256958  8569 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.324252  8386 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.324793  8386 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.331153  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.331568  7914 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:10.331887  2373 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:10.333150  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.333813  2372 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 13.
I20260709 13:04:10.336864  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.338621  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:10.304095  8595 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.000s
I20260709 13:04:10.340833  8595 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.352521  1642 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:10.353516  8595 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:10.360249  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.363300  1576 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:10.343055  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44760 (request call id 2932) took 2440 ms (client timeout 1694 ms). Trace:
W20260709 13:04:10.364897  1569 rpcz_store.cc:269] 0709 13:04:07.903134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.904135 (+  1001us) service_pool.cc:224] Handling call
0709 13:04:10.342787 (+2438652us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400214dd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:10.342947 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4535845248}
I20260709 13:04:10.371235  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.283139  8319 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:10.380892  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63487.7us
I20260709 13:04:10.382182  1539 leader_election.cc:400] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20.
I20260709 13:04:10.382665  1539 leader_election.cc:403] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:04:10.383657  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.387118  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.387871  1575 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.386406  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.394212  1570 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.394382  1572 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:10.394819  1570 raft_consensus.cc:2410] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:04:10.395881  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.397292  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.657s (SENT)
I20260709 13:04:10.397907  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.403635  1572 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:10.405120  8509 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20.
I20260709 13:04:10.407657  1571 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:04:10.409825  1570 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:10.416640  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.412669  1575 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:04:10.428174  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 46176.4us
I20260709 13:04:10.402070  8410 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.435034  8615 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18.
I20260709 13:04:10.431929  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.467391  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:10.469573  8427 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:10.478219  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:10.478574  8500 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:10.479187  8509 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:10.479597  8509 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:10.479957  8509 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:10.480876  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:10.481900  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:10.487031  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.487308  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.487810  1575 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.488842  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.489486  1576 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.488128  1571 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:10.490262  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.476951  8493 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:10.490546  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.492812  1573 raft_consensus.cc:2410] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:04:10.476442  8615 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.490973  1570 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.493929  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.497668  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.488116  8468 raft_consensus.cc:2764] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:04:10.492254  8493 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:10.503247  8611 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.518s	user 0.002s	sys 0.000s
I20260709 13:04:10.502990  8493 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.504276  8493 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:10.505110  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.505666  8569 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:10.507068  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.511054  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.515342  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.515861  1575 raft_consensus.cc:1240] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:10.516650  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.517916  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.518500  1573 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:10.519721  1569 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:10.520977  1574 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:04:10.521826  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.522543  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.485394  8500 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:10.523201  1573 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:10.523356  8500 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:10.524318  8500 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.527727  8370 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.528235  8370 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.530398  8370 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:10.530705  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.536551  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.542277  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.540730  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.544176  1569 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:10.541931  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.545856  1575 raft_consensus.cc:2368] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a for earlier term 13. Current term is 14.
I20260709 13:04:10.535108  8509 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.546960  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.554246  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.554960  1568 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 14: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.548022  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.555714  1575 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.548303  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.557060  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.549302  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.557866  1568 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.558252  1574 raft_consensus.cc:2393] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.557076  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.558832  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.561347  8509 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.568248  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.573150  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.554144  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.569751  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.574184  2380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 975) took 1288 ms. Trace:
I20260709 13:04:10.582446  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:10.584478  1575 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:10.577939  2380 rpcz_store.cc:276] 0709 13:04:09.285772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.290069 (+  4297us) service_pool.cc:224] Handling call
0709 13:04:09.290336 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.297354 (+  7018us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:09.318368 (+ 21014us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.516497 (+198129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.516963 (+   466us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:09.517018 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.573956 (+1056938us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b54004ae054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:10.574004 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.574129 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":84255,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":129,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":9270,"spinlock_wait_cycles":15478016,"thread_start_us":218302,"threads_started":2,"wal-append.queue_time_us":332595}
I20260709 13:04:10.585217  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:10.552871  8345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.273s	user 0.005s	sys 0.005s
I20260709 13:04:10.586453  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:10.587841  8401 consensus_queue.cc:1048] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:10.588090  8623 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:10.589159  8623 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:10.589498  8623 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:10.590360  8623 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.591701  8623 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:10.593263  8302 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:10.551811  1573 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:10.599987  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.601284  1575 raft_consensus.cc:2393] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:10.602283  1725 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:10.602735  1642 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:10.604053  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.604079  1725 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:04:10.606698  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.606845  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.607300  2380 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 15.
I20260709 13:04:10.608038  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:10.608717  2371 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:10.610981  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.611299  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.612056  1721 raft_consensus.cc:2410] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:04:10.612274  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:10.615166  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.615917  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.616683  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.617254  1644 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:04:10.604704  1645 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:10.618568  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.622313  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.623306  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.624861  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:10.616153  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.629051  1725 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:10.632236  1642 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:10.625974  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:10.633885  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.634605  1573 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:04:10.635049  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:10.587256  8345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.307s	user 0.005s	sys 0.005s
I20260709 13:04:10.636767  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.640657  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:10.654844  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.587570  8370 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:10.658641  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:10.647274  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:10.660542  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.661104  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.648066  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:10.657173  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.646931  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.615284  8566 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:10.662607  1718 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:10.668763  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.669019  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:10.609388  8534 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:10.631137  8577 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:10.666119  8401 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:10.648576  8615 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.680665  8401 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:10.681124  8401 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.680929  8615 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.683023  8615 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.670992  1642 raft_consensus.cc:1240] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [13.7-17.9]
I20260709 13:04:10.684052  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.684738  1724 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:10.671932  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:10.685530  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.683845  8577 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.688740  1573 raft_consensus.cc:3055] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:10.689075  1573 raft_consensus.cc:740] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:10.672708  8468 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.689826  1573 consensus_queue.cc:260] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.690763  1573 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.647902  8509 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:10.696970  1575 raft_consensus.cc:1275] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:10.700583  2371 raft_consensus.cc:2410] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:04:10.702194  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:10.701215  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:10.670881  8345 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.688225  8577 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:10.717216  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.717235  1573 raft_consensus.cc:1275] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:04:10.719267  1573 pending_rounds.cc:85] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 13
I20260709 13:04:10.719544  1573 pending_rounds.cc:107] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:04:10.719995  1573 raft_consensus.cc:2889] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.707617  8615 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:10.708878  8481 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:10.708247  8509 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.738838  8509 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:10.729007  8481 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:10.740052  8481 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:10.722257  8468 consensus_queue.cc:1059] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:04:10.752981  8401 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.754777  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:10.756562  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 715263us
I20260709 13:04:10.707255  8600 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 13:04:10.764366  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.729920  8615 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:10.759557  8509 raft_consensus.cc:3055] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:10.775866  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 746354us
I20260709 13:04:10.779564  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2508) took 1240 ms. Trace:
I20260709 13:04:10.781261  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2503) took 1938 ms. Trace:
I20260709 13:04:10.786211  1651 rpcz_store.cc:276] 0709 13:04:08.842401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.219079 (+376678us) service_pool.cc:224] Handling call
0709 13:04:09.219373 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.418206 (+198833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.418249 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.418921 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.043326 (+624405us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:10.186139 (+142813us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.780509 (+594370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.780656 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.780888 (+   232us) raft_consensus.cc:1685] finished
0709 13:04:10.780912 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.781156 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":593920,"prepare.queue_time_us":75211,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":55493,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":904331,"threads_started":3,"wal-append.queue_time_us":624831}
I20260709 13:04:10.783726  1649 rpcz_store.cc:276] 0709 13:04:09.590974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.591111 (+   137us) service_pool.cc:224] Handling call
0709 13:04:09.625474 (+ 34363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.647085 (+ 21611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.647127 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.772253 (+125126us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.974866 (+202613us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:10.113140 (+138274us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.779165 (+666025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.779237 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.779292 (+    55us) raft_consensus.cc:1685] finished
0709 13:04:10.779303 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.779472 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":710901,"prepare.queue_time_us":132131,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":1886,"raft.queue_time_us":82607,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":52586,"thread_start_us":185662,"threads_started":2,"wal-append.queue_time_us":139942}
I20260709 13:04:10.782457  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.791756  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2523) took 1159 ms. Trace:
I20260709 13:04:10.778782  8626 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.783517  8509 raft_consensus.cc:740] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:10.804570  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2522) took 1173 ms. Trace:
I20260709 13:04:10.805763  1648 rpcz_store.cc:276] 0709 13:04:09.630725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.974923 (+344198us) service_pool.cc:224] Handling call
0709 13:04:09.975364 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.975650 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:10.174570 (+198920us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.804081 (+629511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.804148 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:10.804211 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.804289 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.804457 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":430696,"prepare.queue_time_us":133245,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"raft.queue_time_us":375086,"thread_start_us":378454,"threads_started":2,"wal-append.queue_time_us":291767}
I20260709 13:04:10.806990  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:10.811118  8401 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.813961  8386 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:10.821846  1572 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:04:10.817566  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2510) took 1225 ms. Trace:
I20260709 13:04:10.827431  8528 consensus_queue.cc:1048] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:10.795960  8500 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.823761  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:10.837412  8509 consensus_queue.cc:260] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.838984  8597 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:10.839620  8597 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:10.841840  8597 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.837483  8494 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:10.837416  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73301.2us
I20260709 13:04:10.847136  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.827206  1643 rpcz_store.cc:276] 0709 13:04:09.591641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.591801 (+   160us) service_pool.cc:224] Handling call
0709 13:04:09.592208 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:09.595662 (+  3454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.595701 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:09.622605 (+ 26904us) log.cc:844] Serialized 52 byte log entry
0709 13:04:09.959876 (+337271us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:10.010810 (+ 50934us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.107621 (+ 96811us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.326054 (+218433us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b180032f920. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:04:10.753689 (+427635us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b180032f920. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830bfbc0a 00007fc83a33b219 00007fc837c4258f 00007fc830c81571 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd
0709 13:04:10.816660 (+ 62971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.817097 (+   437us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.817154 (+    57us) raft_consensus.cc:1685] finished
0709 13:04:10.817177 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.817489 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":909162,"prepare.queue_time_us":165990,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":910,"raft.queue_time_us":748080,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":31875,"spinlock_wait_cycles":64642432,"thread_start_us":298476,"threads_started":1,"wal-append.queue_time_us":380860}
I20260709 13:04:10.849177  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:10.819857  8612 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:10.860249  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:10.870815  2372 raft_consensus.cc:1240] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:10.870244  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 163967us
I20260709 13:04:10.792161  1646 rpcz_store.cc:276] 0709 13:04:09.632364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.020969 (+388605us) service_pool.cc:224] Handling call
0709 13:04:10.031200 (+ 10231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.110825 (+ 79625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.110857 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.111259 (+   402us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.259086 (+147827us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:10.272542 (+ 13456us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.345337 (+ 72795us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b180031c0c0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:04:10.785664 (+440327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.785771 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.785842 (+    71us) raft_consensus.cc:1685] finished
0709 13:04:10.791539 (+  5697us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.791692 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":512794,"prepare.queue_time_us":14383,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":517579,"spinlock_wait_cycles":2394880,"thread_start_us":111484,"threads_started":1,"wal-append.queue_time_us":158786}
I20260709 13:04:10.873988  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:10.875350  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.871034  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:10.881253  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.882128  1570 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:04:10.890826  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.894238  1616 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.895205  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:10.897511  1718 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:10.899140  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:10.901121  8443 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:10.903091  8598 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:10.904585  8598 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:10.905009  8598 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:10.905797  8598 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.907017  8598 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.908885  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:10.909699  1574 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:10.910753  1569 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:10.895139  8600 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.914286  8600 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.914821  8600 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.915583  1724 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:10.883886  8401 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:10.917146  8401 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.918778  8509 raft_consensus.cc:3060] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:10.919582  8401 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:10.887466  8529 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.859248  8316 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:04:10.886308  8315 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:10.927480  8315 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.927924  8315 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.864012  8589 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.932768  8589 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:10.935300  8589 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:10.935428  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:10.942648  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68777.7us
I20260709 13:04:10.856842  8612 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:10.952993  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58844.3us
I20260709 13:04:10.957315  1570 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 8. (term mismatch)
I20260709 13:04:10.957979  1570 pending_rounds.cc:85] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 7
I20260709 13:04:10.959579  1570 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:04:10.960408  1570 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.960844  1570 pending_rounds.cc:107] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:04:10.961172  1570 raft_consensus.cc:2889] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.962224  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:10.965505  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:10.968967  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:10.868445  8629 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 13:04:10.973361  1724 raft_consensus.cc:2410] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:10.834810  8615 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:10.974869  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:10.989506  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:10.992945  2372 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:10.995769  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.971113  8500 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:04:11.002867  8509 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:11.003301  8500 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:10.900556  8577 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:11.010761  8577 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:10.967607  8427 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:11.011848  8427 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:10.974295  8110 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:04:11.016872  1690 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:10.964154  8529 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.963960  8599 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:10.999821  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:11.039173  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.044188  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 44660.9us
I20260709 13:04:11.051527  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:11.020049  8562 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:11.040273  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:11.075512  8654 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:04:11.075748  8382 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.971874  8589 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:04:11.025799  8570 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:11.085575  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 223485us
I20260709 13:04:11.107297  2372 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:11.020633  8278 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:04:11.115381  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.123994  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72538.3us
I20260709 13:04:11.020465  8541 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:04:11.015488  8481 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:11.069098  8562 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.056036  8599 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:11.081239  8598 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:11.078233  8302 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.032740  8529 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.184592  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:11.178681  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 98016.9us
I20260709 13:04:11.188105  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:11.154327  8329 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:11.149573  8600 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.155480  8387 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:11.163763  8562 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.167331  8598 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.171048  8386 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.148885  8569 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:04:11.189504  8657 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:11.149469  8382 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.192672  8329 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:11.211463  8600 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:04:11.228933  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 253057us
I20260709 13:04:11.218871  8598 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.193120  8526 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:04:11.238440  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:11.247200  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59183.7us
I20260709 13:04:11.222136  8657 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:11.225507  8329 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:11.223778  8252 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.211301  8387 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.269097  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:11.229087  8600 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:11.282140  8500 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.282729  8500 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.224485  8386 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.288968  2378 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 23.
W20260709 13:04:11.286224  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.265045  8319 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.330135  8529 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.333467  8671 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:11.335695  8657 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.338151  8657 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.338963  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1045) took 1294 ms. Trace:
I20260709 13:04:11.339398  1650 rpcz_store.cc:276] 0709 13:04:10.044447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.391810 (+347363us) service_pool.cc:224] Handling call
0709 13:04:10.421902 (+ 30092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.803476 (+381574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.803515 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.809548 (+  6033us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b48001c0158. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc838c97aa1 00007fc838c8eb37 00007fc838c8e9ca 00007fc838d8626f 00007fc838d85f37 00007fc838d9a55c 00007fc838d9a508 00007fc838d9a488 00007fc838d9a443 00007fc838d996ec 00007fc837cc3a96
0709 13:04:10.823216 (+ 13668us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.254627 (+431411us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:11.254669 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.254740 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.338644 (+ 83904us) raft_consensus.cc:1685] finished
0709 13:04:11.338666 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.338890 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":381062,"raft.queue_time_us":381894,"raft.run_cpu_time_us":2120,"raft.run_wall_time_us":5500,"spinlock_wait_cycles":5167488,"thread_start_us":390880,"threads_started":1,"wal-append.queue_time_us":490926}
I20260709 13:04:11.318710  8569 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:11.252810  8599 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.289474  8315 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.362162  8570 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:11.317778  8567 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:11.363299  8567 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:11.363677  8500 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.320964  8110 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.365010  8110 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.365706  8567 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:11.349470  8600 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:11.367542  8110 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:11.372484  8494 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:11.373087  8494 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:11.374181  8494 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.375801  8494 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:11.354979  8387 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.362668  8315 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.363174  8570 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:11.367686  1688 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.711s (SENT)
I20260709 13:04:11.358839  8329 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:11.356891  8386 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:11.370388  8481 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:11.355007  8382 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.360008  8569 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.405462  2371 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:11.409055  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.412236  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:11.414166  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 143523us
I20260709 13:04:11.367566  8600 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.427482  8569 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:11.447568  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.450326  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 265896us
W20260709 13:04:11.450482  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 212050us
I20260709 13:04:11.454931  8329 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:11.448936  8672 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:11.461413  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3239) took 1823 ms. Trace:
I20260709 13:04:11.461869  1567 rpcz_store.cc:276] 0709 13:04:09.637924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.644097 (+  6173us) service_pool.cc:224] Handling call
0709 13:04:09.658929 (+ 14832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.915708 (+1256779us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18003c8380. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc8339164bc 00007fc830c6448b 00007fc839abeab9 00007fc830c8069b 00007fc830c7e93f 00007fc830c7e455 00007fc833d66060 00007fc833d66bfb 00007fc833d7164b
0709 13:04:10.915855 (+   147us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:10.916182 (+   327us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.229975 (+313793us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.235170 (+  5195us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.235325 (+   155us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:11.235604 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.431066 (+195462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.431175 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.461154 (+ 29979us) raft_consensus.cc:1685] finished
0709 13:04:11.461184 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.461342 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8571,"prepare.run_cpu_time_us":229,"prepare.run_wall_time_us":9119,"raft.queue_time_us":437279,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":7842,"spinlock_wait_cycles":28002176,"thread_start_us":753555,"threads_started":3,"wal-append.queue_time_us":465621}
I20260709 13:04:11.460472  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:11.451445  8461 consensus_queue.cc:1059] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:04:11.473686  1570 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:11.459700  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:11.480232  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.480798  1570 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:04:11.481206  1570 pending_rounds.cc:85] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:04:11.481436  1570 pending_rounds.cc:107] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:04:11.481753  1570 raft_consensus.cc:2889] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.482551  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.484513  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:11.496951  1568 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:04:11.497246  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:11.498292  1717 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:11.501848  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:11.502825  2371 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:11.506696  2380 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:04:11.508461  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:11.510601  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.512310  1567 raft_consensus.cc:2410] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:04:11.467366  8110 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:11.469271  8319 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:11.491389  8302 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:11.515340  1575 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:11.516470  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:11.502278  8481 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:11.520560  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:11.494365  8677 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:11.524750  2380 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:11.527755  1724 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:04:11.527293  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44982.6us
I20260709 13:04:11.529755  1567 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:04:11.522447  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.531772  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:11.532235  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.533255  1569 raft_consensus.cc:2410] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:04:11.527665  8481 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.536784  8481 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:11.537348  8562 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.539801  8562 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.514786  8675 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:04:11.542244  8682 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.542762  8682 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.544499  8329 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:11.545258  8329 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:11.550616  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 101948us
W20260709 13:04:11.551826  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.024s (SENT)
I20260709 13:04:11.552383  2344 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.557273  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.562228  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.563143  1572 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 20.
I20260709 13:04:11.563572  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.565053  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:11.558969  8562 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.593829  8562 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.521171  8302 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.595804  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:11.601585  1569 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:11.560209  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66859.6us
I20260709 13:04:11.605971  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:11.546586  8461 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.598946  2380 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:11.597512  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.528141  8677 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.594676  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:11.622597  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:11.617746  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.613430  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.623678  1573 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.623667  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.624015  1574 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:11.629051  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.630049  1650 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:11.630450  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.630069  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.637398  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.637256  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.640749  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.609593  8302 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:11.654278  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.657032  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:11.658226  1572 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.658361  1645 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:04:11.659372  1645 pending_rounds.cc:85] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 10
I20260709 13:04:11.656139  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:11.659881  1645 pending_rounds.cc:107] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:04:11.660244  1645 raft_consensus.cc:2889] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.610392  8678 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:11.617909  8623 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.664306  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:11.664683  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.620162  8676 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:11.619913  8526 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:11.679646  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.679811  8526 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.680682  1647 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:11.681689  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.683722  8657 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.654699  8500 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.673622  8319 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:11.643090  8345 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.628763  8677 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:11.638193  8514 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:11.643587  8468 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:11.658947  8671 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:11.617599  8598 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.668648  8678 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.643888  8567 consensus_queue.cc:1048] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:04:11.643625  8370 consensus_queue.cc:1048] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:04:11.627856  8666 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.687376  8687 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:11.692948  8682 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:11.688439  8676 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.719707  8329 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.703212  8678 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.709275  8687 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:11.746722  8687 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.709542  8692 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:11.748627  8692 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.749101  8692 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.739419  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:11.753949  1648 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:04:11.741778  8598 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.673439  8675 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:11.790148  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.683115  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.806669  1569 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:04:11.811486  1573 raft_consensus.cc:2410] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:04:11.814110  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.804749  1647 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
W20260709 13:04:11.828855  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 222887us
I20260709 13:04:11.756778  8687 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:11.808204  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 260405us
W20260709 13:04:11.829536  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 193187us
W20260709 13:04:11.874414  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.876518  1647 raft_consensus.cc:1275] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:04:11.877023  1572 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:11.821085  8590 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:11.881197  1648 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:04:11.887804  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 231673us
I20260709 13:04:11.890539  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:11.900530  1574 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:11.902902  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.904918  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.905736  1574 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:11.911818  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:11.912644  1571 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:11.914673  1571 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 13:04:11.893138  8509 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.890s	user 0.011s	sys 0.011s
I20260709 13:04:11.873618  8626 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:11.883790  8427 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.869s	user 0.018s	sys 0.004s
I20260709 13:04:11.856268  8672 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:11.923012  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:11.886843  8689 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.893630  8110 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:11.919857  8500 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:11.880759  8666 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:11.936597  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:11.926528  8509 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:12.010701  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:11.897876  8597 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 13:04:11.894215  8590 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.025861  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:12.027448  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:12.028048  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 149362us
I20260709 13:04:11.941352  8500 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:12.030579  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:12.004931  8562 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.941390  8570 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:04:12.007316  8676 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:11.939217  8427 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.037870  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:12.002342  8689 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:11.934250  8626 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.056303  1648 raft_consensus.cc:2393] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 22: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:11.941460  8698 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:04:12.034221  8500 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:12.075500  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 154738us
I20260709 13:04:12.065434  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:12.059800  8509 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.082297  1718 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:12.043634  8562 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:12.037207  8526 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:12.066116  8702 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:04:12.003396  8666 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.084606  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:12.095026  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:12.088570  1648 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:12.065061  8461 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:12.084932  8494 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.005s
W20260709 13:04:12.084434  8481 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.543s	user 0.016s	sys 0.000s
I20260709 13:04:12.083678  8700 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:12.139845  2378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41544 (request call id 981) took 1294 ms (client timeout 1554 ms). Trace:
I20260709 13:04:12.089025  8712 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:12.150291  2378 rpcz_store.cc:269] 0709 13:04:10.845538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.846032 (+   494us) service_pool.cc:224] Handling call
0709 13:04:12.139457 (+1293425us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400457754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:12.139727 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352707840}
W20260709 13:04:12.184748  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 100504us
W20260709 13:04:12.136461  8699 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.252s	user 0.002s	sys 0.000s
I20260709 13:04:12.094049  8509 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.130282  8461 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.168864  8700 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.133193  8481 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:12.204800  8302 consensus_meta.cc:287] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.537s	user 0.016s	sys 0.001s
W20260709 13:04:12.148933  8704 log.cc:927] Time spent T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:04:12.142513  8676 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:12.134788  8708 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:04:12.301649  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.306710  8705 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.174s	user 0.004s	sys 0.000s
W20260709 13:04:12.318708  8701 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.378s	user 0.005s	sys 0.000s
I20260709 13:04:12.181798  8711 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:12.324899  8694 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.239s	user 0.004s	sys 0.000s
I20260709 13:04:12.056408  8672 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.180382  8494 consensus_queue.cc:1059] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:04:12.085282  8707 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:04:12.369755  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:12.371627  1646 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:04:12.371716  8569 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.920s	user 0.023s	sys 0.006s
W20260709 13:04:12.327795  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.004s	sys 0.005s
W20260709 13:04:12.323724  8658 log.cc:927] Time spent T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 1.248s	user 0.012s	sys 0.005s
I20260709 13:04:12.166435  8629 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:04:12.344475  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 413683us
I20260709 13:04:12.350243  8590 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:12.357147  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 214931us
I20260709 13:04:12.366551  8494 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.362934  8427 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:12.328591  8302 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:12.380365  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.005s	sys 0.006s
W20260709 13:04:12.448086  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 811251us
I20260709 13:04:12.392249  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:12.451889  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59706.5us
I20260709 13:04:12.136767  8709 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.355561  8626 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.359331  8714 raft_consensus.cc:2764] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:04:12.410446  1646 raft_consensus.cc:1240] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:12.357658  8711 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.378029  8718 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.414703  8461 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:12.474588  8715 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:12.476887  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1050) took 1867 ms. Trace:
I20260709 13:04:12.477480  1644 rpcz_store.cc:276] 0709 13:04:10.608959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.659820 (+ 50861us) service_pool.cc:224] Handling call
0709 13:04:10.667130 (+  7310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.808538 (+141408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.808571 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.809240 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.992426 (+183186us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:11.221050 (+228624us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.222888 (+  1838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.222994 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.084850 (+861856us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:04:12.476458 (+391608us) raft_consensus.cc:1685] finished
0709 13:04:12.476594 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.476811 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142484,"prepare.queue_time_us":334124,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":41,"raft.queue_time_us":156384,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":18084,"spinlock_wait_cycles":9745792,"thread_start_us":375960,"threads_started":2,"wal-append.queue_time_us":265245}
W20260709 13:04:12.462072  8687 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.000s
I20260709 13:04:12.388780  8569 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.460443  8570 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:12.461575  8689 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:12.459897  8494 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.521852  8712 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:12.486824  8718 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.466836  8709 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.525252  8493 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:12.494112  8714 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:12.494545  7850 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:12.631816  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2526) took 1002 ms. Trace:
I20260709 13:04:12.509101  8626 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.684072  8714 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.684823  8626 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.686738  8626 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.640035  8567 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.665979  1643 rpcz_store.cc:276] 0709 13:04:11.629547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.630413 (+   866us) service_pool.cc:224] Handling call
0709 13:04:11.637251 (+  6838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:11.643869 (+  6618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.643903 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:11.658258 (+ 14355us) log.cc:844] Serialized 52 byte log entry
0709 13:04:11.819558 (+161300us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:11.820256 (+   698us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.852627 (+ 32371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.852707 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.631464 (+778757us) raft_consensus.cc:1685] finished
0709 13:04:12.631495 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.631731 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31771,"prepare.queue_time_us":190,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":39699,"raft.run_cpu_time_us":501,"raft.run_wall_time_us":32859,"thread_start_us":125941,"threads_started":1,"wal-append.queue_time_us":221662}
I20260709 13:04:12.644431  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:12.646010  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:12.671408  8494 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.672616  8735 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.673175  8709 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.672919  8712 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:12.643965  8677 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.952s	user 0.013s	sys 0.009s
I20260709 13:04:12.674360  8302 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:12.675249  8678 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.930s	user 0.018s	sys 0.001s
I20260709 13:04:12.638481  8666 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.676061  8569 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:12.676141  8481 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.677132  8700 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.677685  8711 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:12.679526  8718 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.700870  8714 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:12.720839  8718 consensus_queue.cc:1059] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:12.722081  8718 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:12.680972  8674 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.641836  8692 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.892s	user 0.017s	sys 0.003s
I20260709 13:04:12.723650  8674 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.726332  8674 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.643424  8730 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.727501  8730 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:12.705056  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59156.7us
I20260709 13:04:12.730260  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:12.660413  8493 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:12.700982  8500 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.005s
I20260709 13:04:12.687599  8567 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.664429  8566 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:12.688088  8708 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:12.723719  8692 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:12.638127  8600 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.202s	user 0.019s	sys 0.008s
I20260709 13:04:12.698199  7850 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.735157  8500 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.747100  8692 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:12.747295  8386 raft_consensus.cc:3055] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:12.747694  8386 raft_consensus.cc:740] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:12.709626  8677 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.748515  8500 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:12.749519  8677 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.714468  8678 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.754977  8732 raft_consensus.cc:3055] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:12.707105  8735 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.760022  8730 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.739718  8708 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.762544  8732 raft_consensus.cc:740] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:12.769793  8735 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.770284  8708 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:12.771158  8732 consensus_queue.cc:260] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.773273  8732 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:12.646124  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 458393us
I20260709 13:04:12.819195  1572 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 13:04:12.825779  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.748524  8386 consensus_queue.cc:260] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.741081  2380 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:12.836175  8493 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:12.836175  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 105738us
I20260709 13:04:12.839752  8493 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:12.842120  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:12.738673  8566 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:12.849413  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.846086  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139941us
I20260709 13:04:12.846539  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:12.859192  8567 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.854064  8566 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.844228  7850 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:12.867787  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.868225  8674 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.869668  8674 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.874357  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:12.871956  8678 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.893026  8743 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:12.894107  8743 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:12.897337  8744 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:12.898690  8744 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.901019  8730 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:12.901633  8730 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:12.902112  8730 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:12.905495  8674 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.906284  8674 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.907513  8674 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.910636  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:12.915452  8720 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:12.916296  8720 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:12.916886  8720 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:12.919000  8629 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.919922  8629 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.920176  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3270) took 1905 ms. Trace:
I20260709 13:04:12.922111  1718 rpcz_store.cc:276] 0709 13:04:11.014726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.014917 (+   191us) service_pool.cc:224] Handling call
0709 13:04:11.015597 (+   680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.091948 (+1076351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.919391 (+827443us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400232b54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:12.919865 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435128192}
I20260709 13:04:12.923278  8629 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.927968  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:12.938575  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75628.4us
I20260709 13:04:12.940824  8678 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:12.946033  8302 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.947134  8302 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.950580  8723 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:12.896013  8743 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.954124  8690 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:12.959231  1651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2532) took 1326 ms. Trace:
I20260709 13:04:12.959980  1651 rpcz_store.cc:276] 0709 13:04:11.632667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.633042 (+   375us) service_pool.cc:224] Handling call
0709 13:04:12.922886 (+1289844us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b5800051698. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830c49b71 00007fc830c49a22 00007fc839b62c09 00007fc838cbcee0 00007fc838d860ab 00007fc838d86131 00007fc838d47fc0 00007fc838d32929 00007fc838d323d2 00007fc8322a9bd4
0709 13:04:12.923462 (+   576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.924038 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.924069 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.924134 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:12.924198 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.924262 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.958931 (+ 34669us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2606464}
I20260709 13:04:12.951649  8723 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.984277  8674 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.001652  8570 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:13.018101  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 77956.2us
I20260709 13:04:13.018416  8570 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.020231  8671 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:13.028921  8744 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.991446  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:13.032802  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 151016us
I20260709 13:04:13.033033  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2991) took 1133 ms. Trace:
I20260709 13:04:13.040190  8657 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:13.042315  8570 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.041702  8657 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:12.986608  8723 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.046379  8657 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.047749  1572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 2987) took 1152 ms. Trace:
I20260709 13:04:13.048381  1572 rpcz_store.cc:276] 0709 13:04:11.894902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.896021 (+  1119us) service_pool.cc:224] Handling call
0709 13:04:11.898118 (+  2097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.826938 (+928820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.047348 (+220410us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400125cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:13.047626 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114782080}
I20260709 13:04:13.047196  8657 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.051246  8657 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.040030  8715 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.955727  8690 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:12.986595  8509 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:12.991021  8302 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.064057  8336 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:13.093294  8336 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:13.105562  8336 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.106683  8302 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:13.055533  8715 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.121472  8715 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.090708  8723 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.124614  8723 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.128499  8723 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.133159  8707 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:13.111136  8567 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:13.139215  8567 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.143355  8567 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.144654  8673 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.145637  8673 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.151099  8673 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.113808  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.041302  1573 rpcz_store.cc:276] 0709 13:04:11.899539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.902061 (+  2522us) service_pool.cc:224] Handling call
0709 13:04:11.902402 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.141548 (+239146us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:12.187537 (+ 45989us) log.cc:844] Serialized 14 byte log entry
0709 13:04:12.495990 (+308453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.496049 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:12.496092 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.032466 (+536374us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400364cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:13.032541 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.032807 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4030,"prepare.queue_time_us":52901,"prepare.run_cpu_time_us":3121,"prepare.run_wall_time_us":9854,"raft.queue_time_us":172594,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":549104000,"thread_start_us":304008,"threads_started":1,"wal-append.queue_time_us":453338}
I20260709 13:04:13.137562  8570 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.154672  8570 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.021327  8612 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.157476  8612 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:13.158254  1648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2674) took 1304 ms. Trace:
I20260709 13:04:13.158913  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.158862  1648 rpcz_store.cc:276] 0709 13:04:11.854031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.065055 (+211024us) service_pool.cc:224] Handling call
0709 13:04:13.157750 (+1092695us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b54001bc6d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:13.157990 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16431488}
I20260709 13:04:13.147428  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.160913  1722 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:13.148054  8707 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:13.165055  8707 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.162926  8712 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:13.080595  8671 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.166774  8712 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.167533  8671 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.170146  8671 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.170179  8712 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:13.166764  8739 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.405s	user 0.000s	sys 0.004s
I20260709 13:04:13.173586  8570 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.165817  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:13.174108  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.175827  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.176980  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.170989  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.162724  8715 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.170993  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:13.107674  8302 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.192034  1572 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 22: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:13.196236  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.196755  1645 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:13.197420  1642 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:13.199361  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:11.880810   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:13.214793  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.190840  8690 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.220743  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.224001  1648 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:13.228008  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.228996  1722 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:04:13.223954  8715 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.235219  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.237346  1568 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:04:13.243984  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.248853  8710 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:13.249390  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:13.249910  8386 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.250407  1569 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:13.262857  1567 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:13.249871  8710 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.272519  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:13.240839  8715 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.292791  1572 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:13.287654  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:13.252856  8657 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.296785  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:13.305280  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.267163  8710 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.285774  8730 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.310040  8724 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:13.311167  8732 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.537s	user 0.004s	sys 0.018s
I20260709 13:04:13.322113  2371 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:13.322057  8724 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.322846  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:13.308091  8707 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:13.335558  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.336768  1568 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
W20260709 13:04:13.341920  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.344941  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.344686  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:13.344833  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.349659  1567 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.350907  1642 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:13.345953  1571 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:13.359253  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.360298  1690 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:13.357203  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.357935  1724 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 13.
I20260709 13:04:13.344094  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.360416  1465 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:13.373891  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.377465  1465 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 16, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:13.380753  1575 raft_consensus.cc:1240] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:13.380190  1465 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 11 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 13 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:13.382833  1574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44760 (request call id 3001) took 1420 ms. Trace:
I20260709 13:04:13.383306  1574 rpcz_store.cc:276] 0709 13:04:11.917313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.917440 (+   127us) service_pool.cc:224] Handling call
0709 13:04:11.917715 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.248220 (+1330505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.248268 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:13.248312 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:13.248377 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.319706 (+ 71329us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400026cd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:13.319781 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.337452 (+ 17671us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3051284352}
I20260709 13:04:13.382380  1465 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:13.395431  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.397393  1575 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:13.397982  1465 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:13.400527  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:13.402935  2372 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 23: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:13.391889  8615 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:13.386447  8730 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.403475  2371 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
W20260709 13:04:13.408746  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48857.8us
I20260709 13:04:13.401803  1465 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 11 to 16, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:13.413744  1569 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
W20260709 13:04:13.411233  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48426.9us
I20260709 13:04:13.419524  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:13.438220  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.440399  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.438308  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:13.445806  1571 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:13.441231  1645 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:13.447576  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.408409  8715 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:13.449836  1648 raft_consensus.cc:2393] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 22: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:13.450232  8494 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.451249  8468 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:13.454147  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.505970  1568 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:13.456952  8110 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:13.512925  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.463948  1465 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:13.466225  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:13.514107  2380 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:13.475113  8302 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.477808  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.481480  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.483202  8724 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.489630  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.508175  1575 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:13.550556  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 102858us
I20260709 13:04:13.552354  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.556913  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.535952  1616 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:13.527333  8110 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.462107  8690 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:13.567534  1648 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:13.556188  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.572540  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.564656  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:13.511744  1572 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:13.570283  1648 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:13.582037  1642 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:13.584530  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.569123  1465 catalog_manager.cc:5697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:13.592862  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 126905us
I20260709 13:04:13.590155  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.600605  1567 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:04:13.569784  8720 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.011s
I20260709 13:04:13.594477  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.580709  8110 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:13.626236  1648 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.627493  1648 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.628647  8570 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:13.630005  8570 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:13.630420  8570 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:13.544126  8715 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.591270  8657 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:13.638415  1570 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:13.647468  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.635815  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:13.650532  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 210546us
I20260709 13:04:13.650852  8708 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:13.654727  8709 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.656097  1465 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:13.592192  8742 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:04:13.660063  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:13.664135  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.664525  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.664954  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:13.633550  8468 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.667811  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.669520  1725 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:13.551272  8615 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.631729  8570 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.639598  8690 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.706972  8742 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:13.707465  8742 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:13.688407  8708 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:13.708557  8742 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.709487  8708 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.710767  8657 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.713268  8657 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:13.714236  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 54360.1us
I20260709 13:04:13.513281  8494 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.715973  1650 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:04:13.718520  8726 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:13.720532  2378 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:13.722050  8623 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.699996  8302 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:13.700711  1569 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:04:13.702621  8615 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.703249  8709 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.700090  1720 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.725462  8709 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.704746  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.726006  8715 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:13.726594  1570 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:13.726954  1575 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:13.728096  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.705562  1716 raft_consensus.cc:2410] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:13.693248  1568 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:04:13.729419  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.730202  1575 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:13.731130  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.731573  8468 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.723083  8302 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.732970  8302 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.713294  1717 raft_consensus.cc:2393] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:13.737630  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.722688  8623 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.739331  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.740116  1568 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:13.635881  8720 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.741317  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.742549  1568 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:04:13.743530  8720 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.743628  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:13.714346  8673 consensus_queue.cc:1059] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:13.746582  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.747349  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:13.748557  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2530) took 2117 ms (client timeout 1925 ms). Trace:
I20260709 13:04:13.727800  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:13.760437  1571 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 16: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:13.761651  1570 raft_consensus.cc:2410] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:04:13.766144  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:13.769546  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51802us
I20260709 13:04:13.780084  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:13.760514  1650 rpcz_store.cc:269] 0709 13:04:11.631049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.631254 (+   205us) service_pool.cc:224] Handling call
0709 13:04:13.748051 (+2116797us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54001bb7d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:13.748204 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4769856640}
I20260709 13:04:13.752161  8494 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.792106  8673 raft_consensus.cc:3055] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:13.792763  8673 raft_consensus.cc:740] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:13.793983  8673 consensus_queue.cc:260] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.799678  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:13.805706  8494 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:13.807562  8494 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:13.807981  8494 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:13.794557  1569 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:04:13.808710  2376 raft_consensus.cc:1240] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:13.811648  1576 raft_consensus.cc:1240] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:04:13.812105  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 139189us
I20260709 13:04:13.746996  8690 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:13.819080  1720 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:13.831743  1567 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 15.
I20260709 13:04:13.833004  8690 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:13.837209  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:13.841059  8716 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:04:13.846999  1575 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:13.847811  1568 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:13.848528  8535 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:13.849048  8535 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.849436  8535 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.854821  1574 raft_consensus.cc:1240] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:04:13.856853  1574 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:13.865135  8742 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.866067  2376 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:13.796031  8673 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.764832  8623 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:13.876096  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.877454  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:13.867954  8730 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.881042  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.882085  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.882848  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.883060  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.883860  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.889029  8494 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.890542  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:13.891206  1569 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:13.908981  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:13.895262  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.900694  8570 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.924835  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.926896  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.928592  1571 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: [18.9-20.10]
I20260709 13:04:13.928764  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.929520  1647 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:13.930684  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.887921  1572 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:04:13.931943  1572 pending_rounds.cc:85] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:04:13.932291  1572 pending_rounds.cc:107] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:04:13.932624  1572 raft_consensus.cc:2889] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.906857  1648 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 14. (term mismatch)
I20260709 13:04:13.908892  1725 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:04:13.893814  1465 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:13.901777  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.912693  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.913866  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.916538  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:13.921958  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.922803  2372 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:04:13.935977  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.924029  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.924301  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.937135  1642 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:13.935693  1722 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:04:13.933964  1648 pending_rounds.cc:85] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 13
I20260709 13:04:13.938925  1648 pending_rounds.cc:107] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 13:04:13.939248  1648 raft_consensus.cc:2889] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:13.937695  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 73230.1us
I20260709 13:04:13.941264  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.941283  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.945061  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.945415  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.872083  8386 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:13.948199  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.951331  1723 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:13.951445  1721 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:13.915679  8690 consensus_queue.cc:1048] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:04:13.944958  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:13.941488  8707 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.000s
W20260709 13:04:13.962393  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.941340  8726 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:13.974315  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.881091  8110 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:13.946188  8623 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:13.974316  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:13.975979  1650 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:04:13.972012  8492 consensus_queue.cc:1059] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 13:04:13.982441  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.982795  1719 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.983952  1647 raft_consensus.cc:2410] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:04:13.982986  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.989017  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:13.990514  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:13.941224  8730 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:13.972148  8612 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:13.991778  1646 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:13.972898  8726 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:13.998890  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:13.998984  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:13.998798  1649 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:13.987907  8494 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.011955  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.006863  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:13.964802  8716 consensus_queue.cc:1059] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:04:14.015193  1717 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:13.991744  8468 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:13.965394  8535 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.020330  2344 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:14.018404  2376 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:14.023504  8316 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.024304  8316 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.024885  8316 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.030154  1724 raft_consensus.cc:1240] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:04:14.038647  8709 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:14.039376  8709 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.039902  8709 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.994558  8386 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.005023  8726 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.037379  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.074733  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:14.013779  8715 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.080751  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:14.073232  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 80364.8us
I20260709 13:04:14.039973  8743 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.047937  8492 raft_consensus.cc:3055] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:14.011816  8302 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.097702  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.017390  8612 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.105432  1568 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:14.106244  1575 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:14.115792  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:14.116784  1721 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.119797  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:14.092273  8743 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.095284  8386 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:14.094712  8492 raft_consensus.cc:740] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:14.048249  8468 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.129135  1725 raft_consensus.cc:2393] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 19: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:14.147286  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.164022  1721 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:14.174431  8302 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:14.177139  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.180382  1721 raft_consensus.cc:1240] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:14.089380  8715 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.130333  8743 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.192678  1690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.018s
W20260709 13:04:14.189177  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.017s	sys 0.066s
W20260709 13:04:14.191633  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68365.5us
W20260709 13:04:14.194018  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.017s	sys 0.067s
I20260709 13:04:14.181995  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:14.193656  1690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.011s	sys 0.018s
I20260709 13:04:14.205700  1465 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:14.202607  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 523675us
I20260709 13:04:14.214597  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:14.221386  1616 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.223846  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.172067  8720 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.199388  8742 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.233968  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:14.206951  8615 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.238533  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.239698  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.243705  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:14.247480  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53059.4us
I20260709 13:04:14.245194  1576 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:04:14.249006  8730 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.253485  8716 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:14.255344  8716 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.255862  8716 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:14.253602  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:14.257184  1574 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.269519  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.236301  8742 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.275008  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.276299  1644 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:14.277237  1723 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:14.241142  8615 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.277097  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.262384  8110 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:14.283016  8535 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:14.246414  8316 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.278139  8742 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.293663  2373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.018s	sys 0.046s
W20260709 13:04:14.305816  2373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.018s	sys 0.046s
I20260709 13:04:14.306345  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.307590  1717 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:14.284596  8615 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.324801  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.239558  8492 consensus_queue.cc:260] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.330503  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:14.335500  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.309213  8494 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:14.353857  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.042s	sys 0.033s
W20260709 13:04:14.361255  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.042s	sys 0.033s
I20260709 13:04:14.362195  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:14.326650  8723 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.043s	sys 0.115s
I20260709 13:04:14.314625  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:14.351194  8494 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.384558  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:14.373117  8723 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.044s	sys 0.115s
I20260709 13:04:14.384735  8494 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.384415  8707 raft_consensus.cc:3055] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:14.383215  8612 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.376313  8468 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.432508  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:14.436874  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49472.1us
I20260709 13:04:14.447989  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.447144  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.467352  1719 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:04:14.445555  8708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.029s	sys 0.069s
I20260709 13:04:14.437683  8707 raft_consensus.cc:740] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:14.482085  1574 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:14.439582  8715 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.486438  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.451534  8468 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:14.481441  8708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.029s	sys 0.069s
I20260709 13:04:14.476619  8612 consensus_queue.cc:1059] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:04:14.493534  8468 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:14.501586  8675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.034s	sys 0.050s
I20260709 13:04:14.501901  8468 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:14.502206  8675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.034s	sys 0.051s
I20260709 13:04:14.502734  8709 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:14.504436  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.504684  1570 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:14.506280  8709 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.506750  8492 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.496711  8386 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:14.509275  8386 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:14.509711  8386 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:14.510428  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.511979  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.513355  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.515162  8302 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:14.515807  8316 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.518579  8713 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.519065  8713 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:14.519475  8713 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:14.519865  8623 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.001s
I20260709 13:04:14.520469  8623 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:14.525051  1465 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 15, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:14.527117  1465 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:04:14.534291  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.010s	sys 0.005s
I20260709 13:04:14.514806  1690 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.515951  8302 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:14.536226  2344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.009s
I20260709 13:04:14.490477  8715 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.541765  2344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.006s	sys 0.009s
W20260709 13:04:14.543406  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 523001us
W20260709 13:04:14.538930  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.010s	sys 0.005s
W20260709 13:04:14.538600  2342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.002s	sys 0.014s
W20260709 13:04:14.548605  2342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.002s	sys 0.015s
W20260709 13:04:14.549472  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 568417us
W20260709 13:04:14.552295  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.020s	sys 0.030s
W20260709 13:04:14.552810  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.020s	sys 0.030s
W20260709 13:04:14.553087  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 694275us
I20260709 13:04:14.545854  1465 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:14.556808  1690 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:14.561376  1614 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.018s
W20260709 13:04:14.562288  1614 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.000s	sys 0.019s
W20260709 13:04:14.563280  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 577382us
W20260709 13:04:14.561322  1689 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.003s	sys 0.015s
W20260709 13:04:14.583336  1689 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.003s	sys 0.015s
W20260709 13:04:14.585481  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.583777  1465 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 15, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.545193  8715 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.603593  8760 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.607458  8302 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.607853  8760 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.611918  8760 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:14.616084  8716 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:14.618250  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.619880  8468 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.621480  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.588151  8535 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:14.624748  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.622325  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:14.622987  1465 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 12 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 14 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:14.581050  8623 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:14.549093  8743 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.640869  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.644989  1723 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 23.
I20260709 13:04:14.643872  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:14.652241  1465 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:14.651274  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 423816us
I20260709 13:04:14.635778  8720 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:14.694389  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.583573  8707 consensus_queue.cc:260] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.642606  8709 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:14.710404  1465 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 5 to 14, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.694078  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.712159  8709 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.663501  8535 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:14.726938  1465 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 12 to 15, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:14.723017  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 291097us
I20260709 13:04:14.723891  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:14.739574  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 171002us
I20260709 13:04:14.662979  8623 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.759842  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.722653  8716 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:14.762043  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 63509.1us
W20260709 13:04:14.779186  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 188761us
W20260709 13:04:14.782256  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 232777us
I20260709 13:04:14.714149  8720 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.788278  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.726558  8386 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:14.804585  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 84742us
I20260709 13:04:14.777371  8110 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:14.771693  8767 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:14.785409  8707 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:14.798875  8761 log.cc:927] Time spent T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.264s	user 0.001s	sys 0.001s
I20260709 13:04:14.814460  1465 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.824270  8774 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:14.824837  8726 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.002s
W20260709 13:04:14.826967  8570 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.902s	user 0.011s	sys 0.005s
I20260709 13:04:14.830103  8720 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.830216  8468 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.831291  8713 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.810184  8675 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:14.846719  8110 raft_consensus.cc:697] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:14.800784  8773 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:14.833038  8715 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.833560  8708 raft_consensus.cc:3055] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:14.832125  8673 consensus_meta.cc:287] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.930s	user 0.015s	sys 0.000s
W20260709 13:04:14.861891  8758 log.cc:927] Time spent T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.186s	user 0.000s	sys 0.002s
W20260709 13:04:14.862267  1472 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 221) took 49169 ms (client timeout 15000 ms). Trace:
I20260709 13:04:14.863369  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 835) took 1418 ms. Trace:
I20260709 13:04:14.823374  8767 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:14.864897  1720 raft_consensus.cc:2410] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:04:14.831827  8386 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:14.847131  8774 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:14.940608  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.866416  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:14.887432  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 149818us
W20260709 13:04:14.867770  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.949747  1573 rpcz_store.cc:276] 0709 13:04:13.445477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.735678 (+290201us) service_pool.cc:224] Handling call
0709 13:04:13.736125 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.736687 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.736721 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:13.737421 (+   700us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.187647 (+450226us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:14.606407 (+418760us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.606667 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.606751 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.863148 (+256397us) raft_consensus.cc:1685] finished
0709 13:04:14.863171 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.863311 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":562540,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":1666,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":5870,"thread_start_us":791164,"threads_started":2,"wal-append.queue_time_us":713449}
I20260709 13:04:14.955435  1716 raft_consensus.cc:2410] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:14.866990  1465 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.899513  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:14.958448  1717 raft_consensus.cc:606] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:04:14.958837  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 136558us
I20260709 13:04:14.864382  8535 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.959786  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:14.952241  8774 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:14.898147  8764 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:04:14.962075  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3281) took 1095 ms. Trace:
I20260709 13:04:14.896652  8766 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.171s
I20260709 13:04:14.962492  1720 rpcz_store.cc:276] 0709 13:04:13.866372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.869732 (+  3360us) service_pool.cc:224] Handling call
0709 13:04:14.961835 (+1092103us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:14.961985 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2136411008}
I20260709 13:04:14.951751  8675 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.963821  8675 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:14.964059  8787 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:14.943938  8742 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 13:04:14.902381  8723 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:14.964694  8787 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:14.964740  8742 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.952054  8767 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:14.856204  8726 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.967239  8742 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.967705  8726 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.900663  8468 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:14.898999  8769 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:04:14.958866  1717 raft_consensus.cc:614] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:14.970239  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2687) took 1550 ms. Trace:
W20260709 13:04:14.972528  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.956478  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.969269  8787 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:14.899312  8615 consensus_meta.cc:287] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.538s	user 0.003s	sys 0.009s
I20260709 13:04:14.975998  8615 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.979013  8615 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.897478  8709 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.989300  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:14.867970  8492 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.903344  8623 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.896754  8743 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:14.972452  1643 rpcz_store.cc:276] 0709 13:04:13.419865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.420012 (+   147us) service_pool.cc:224] Handling call
0709 13:04:13.491953 (+ 71941us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:13.606369 (+114416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.606416 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:13.652583 (+ 46167us) log.cc:844] Serialized 52 byte log entry
0709 13:04:14.363391 (+710808us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:14.698297 (+334906us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.698610 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.698728 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.970009 (+271281us) raft_consensus.cc:1685] finished
0709 13:04:14.970032 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.970183 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54601,"prepare.queue_time_us":492909,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":132155,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":36799,"spinlock_wait_cycles":21248,"thread_start_us":1016402,"threads_started":2,"wal-append.queue_time_us":892333}
I20260709 13:04:14.952184  8708 raft_consensus.cc:740] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:14.996928  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 221265us
I20260709 13:04:14.971778  8715 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:15.003463  1690 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:14.992973  8707 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:14.994378  8302 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:14.995659  8766 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:15.000937  8494 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.001s
I20260709 13:04:14.968998  8468 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:14.998337  8492 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:15.044690  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84994.4us
I20260709 13:04:15.010993  8766 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.031906  8713 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:14.953660  8773 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.031670  8715 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:15.029269  8494 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.032135  8110 consensus_queue.cc:237] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.006930  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:15.257822  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:15.257925  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45556 (request call id 836) took 1728 ms. Trace:
I20260709 13:04:15.045693  8708 consensus_queue.cc:260] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.033094  8790 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.266511  8790 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.032509  8781 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:15.033890  8792 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:15.269485  8792 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:15.270035  8792 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:15.270964  1576 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 19 index: 17. (term mismatch)
I20260709 13:04:15.271369  1576 pending_rounds.cc:85] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 16
I20260709 13:04:15.271607  1576 pending_rounds.cc:107] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 18.17
I20260709 13:04:15.271827  1576 raft_consensus.cc:2889] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.005139  8690 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:15.273002  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:15.008761  8570 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 13:04:15.258231  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 252116us
I20260709 13:04:15.274662  8715 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.275106  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:15.276110  8708 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:15.258402  1567 rpcz_store.cc:276] 0709 13:04:13.530003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.836322 (+306319us) service_pool.cc:224] Handling call
0709 13:04:13.861060 (+ 24738us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:13.904295 (+ 43235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.904330 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:13.956865 (+ 52535us) log.cc:844] Serialized 52 byte log entry
0709 13:04:14.548149 (+591284us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:14.737796 (+189647us) log.cc:844] Serialized 14 byte log entry
0709 13:04:14.812930 (+ 75134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.813679 (+   749us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.257724 (+444045us) raft_consensus.cc:1685] finished
0709 13:04:15.257743 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.257870 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":250085,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2643,"raft.queue_time_us":151506,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":273,"thread_start_us":767542,"threads_started":3,"wal-append.queue_time_us":679513}
W20260709 13:04:15.254594  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3284) took 1377 ms (client timeout 1604 ms). Trace:
I20260709 13:04:15.253831  8766 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:15.275164  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:15.268548  8773 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.273157  8790 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.273710  1568 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:15.281539  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:15.257020  2344 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.719s (SENT) [suppressed 11 similar messages]
W20260709 13:04:15.286617  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 451043us
W20260709 13:04:15.244875  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.287570  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:15.257725  8709 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:15.285259  1716 rpcz_store.cc:269] 0709 13:04:13.876161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.877054 (+   893us) service_pool.cc:224] Handling call
0709 13:04:15.038141 (+1161087us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540010e0d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:15.253153 (+215012us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412928256}
I20260709 13:04:15.290897  1688 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:15.299787  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:15.308688  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:15.348676  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 57835.7us
I20260709 13:04:15.353461  1568 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.290874  8790 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:15.255352  8302 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.282526  8316 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.950297  1472 rpcz_store.cc:269] 0709 13:03:25.541127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.541344 (+   217us) service_pool.cc:224] Handling call
0709 13:03:25.763923 (+222579us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:04:14.681421 (+48917498us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:03.743253 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:03.744090 (+   837us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:03.744153 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:04:03.744198 (+    45us) tablet.cc:662] Decoding operations
0709 13:04:03.750181 (+  5983us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:03.750310 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:03.750342 (+    32us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:04:03.947299 (+196957us) tablet.cc:701] Row locks acquired
0709 13:04:03.947350 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:04:03.950190 (+  2840us) write_op.cc:270] Start()
0709 13:04:03.950406 (+   216us) write_op.cc:276] Timestamp: P: 1783602243947613 usec, L: 0
0709 13:04:03.950428 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:04.223103 (+272675us) log.cc:844] Serialized 7933 byte log entry
0709 13:04:04.330052 (+106949us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d69e28 00007fc8377a4581 00007fc8377a3d58 00007fc8377a89f1 00007fc8377a89a9 00007fc8377a8939 00007fc8377a8901 00007fc8377a7b2d 000000000038f354 000000000038f199
0709 13:04:06.068632 (+1738580us) op_driver.cc:464] REPLICATION: finished
0709 13:04:06.715921 (+647289us) write_op.cc:301] APPLY: starting
0709 13:04:06.778071 (+ 62150us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:07.933582 (+1155511us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b1800194fe0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc833aad0ec 00007fc833af9c0a 00007fc833af94f7 00007fc833af9490 00007fc833ab0f48 00007fc8377c51bb 00007fc8377c5532 00007fc83780cb93 00007fc8376b9de3
0709 13:04:07.976621 (+ 43039us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6b32a 00007fc833cf0f14 00007fc833cf0ec3 00007fc833cf0e38 00007fc833cf0df3 00007fc833cf00dc 00007fc833d19e16 00007fc833d01cbb 00007fc833cec39d 00007fc833cefd75
0709 13:04:08.183011 (+206390us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:08.183064 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:08.898034 (+714970us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:08.898581 (+   547us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:04:08.898646 (+    65us) write_op.cc:312] APPLY: finished
0709 13:04:08.899309 (+   663us) log.cc:844] Serialized 180 byte log entry
0709 13:04:09.244582 (+345273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:09.262331 (+ 17749us) write_op.cc:454] Released schema lock
0709 13:04:09.293579 (+ 31248us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:09.380623 (+ 87044us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:09.380656 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1744096,"reported_tablets":53,"spinlock_wait_cycles":34286976,"child_traces":[["op",{"apply.queue_time_us":572323,"apply.run_cpu_time_us":65283,"apply.run_wall_time_us":2677453,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87847,"num_ops":20,"prepare.queue_time_us":287890,"prepare.run_cpu_time_us":42114,"prepare.run_wall_time_us":587087,"raft.queue_time_us":1000545,"raft.run_cpu_time_us":36341,"raft.run_wall_time_us":1983108,"replication_time_us":2117992,"spinlock_wait_cycles":453146112,"thread_start_us":1764765,"threads_started":5,"wal-append.queue_time_us":609243,"wal-append.run_cpu_time_us":25088,"wal-append.run_wall_time_us":3514735}]]}
I20260709 13:04:15.305737  8724 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:15.289214  8690 raft_consensus.cc:697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:15.352084  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45750.3us
I20260709 13:04:15.352231  8336 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:15.284091  8615 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:15.388540  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 88811.5us
W20260709 13:04:15.389169  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 68485.5us
I20260709 13:04:15.351346  8570 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:15.395689  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:15.401199  8570 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:15.402788  8570 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.404460  8570 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:15.400970  8615 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:15.410384  8615 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:15.387261  8769 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:15.427780  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:15.356174  8790 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:15.387871  8386 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.433954  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 143899us
I20260709 13:04:15.305517  8773 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
W20260709 13:04:15.428859  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.392938  8743 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:15.350237  8709 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.427987  8789 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:15.433525  8790 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:15.452073  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.457335  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.433885  8623 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.438982  8743 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.391850  8316 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.461429  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1acf73e529394004842219637cff7cfc"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:15.466566  1716 raft_consensus.cc:606] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Received request to transfer leadership to TS da3b57321031416cba9db75450ef302a
I20260709 13:04:15.467048  1716 raft_consensus.cc:614] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:15.444396  8742 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:04:15.468683  8765 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:15.469831  8785 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.470374  8785 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.456692  8336 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:15.472585  8802 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.473246  1650 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:15.473142  8802 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.458154  1643 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:15.472812  8690 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.442963  8789 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:15.461903  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:15.436105  8730 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:04:15.438301  8386 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.477586  8743 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:15.460005  8796 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.500842  1643 raft_consensus.cc:1240] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:04:15.506497  1568 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:15.500324  8797 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:15.475502  8724 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.512435  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:15.513371  1576 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:15.487744  8789 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.509471  8468 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.483927  8802 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.518226  1568 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:15.502172  8767 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.536s	user 0.017s	sys 0.005s
W20260709 13:04:15.508793  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51509.5us
W20260709 13:04:15.501878  8774 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.010s
I20260709 13:04:15.510789  8709 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.494659  8492 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:15.501574  8675 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.002s
I20260709 13:04:15.465047  8760 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:04:15.519529  1643 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:15.505072  8713 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:15.513134  8796 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:15.523824  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96107.8us
I20260709 13:04:15.497705  8785 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.511288  8797 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.511344  8673 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.492401  8623 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.564055  8623 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.494678  8615 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.568264  8615 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.532303  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
W20260709 13:04:15.534634  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 149235us
I20260709 13:04:15.569046  1716 raft_consensus.cc:606] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:15.569413  1614 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:15.570297  1570 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:15.570667  8796 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.516146  8494 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.518790  8723 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:15.572789  8726 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:15.572896  8723 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.573403  8726 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:15.574317  8726 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.575083  8723 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:15.580219  8779 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.571s	user 0.008s	sys 0.011s
I20260709 13:04:15.580617  8792 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.582695  8792 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.576149  8743 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.541457  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:15.529440  8793 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
W20260709 13:04:15.546113  8795 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:04:15.526773  8468 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.532567  8675 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.561347  8797 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:15.561858  8673 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.585335  1567 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:15.572237  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:15.593797  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58022.5us
W20260709 13:04:15.592696  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 157721us
I20260709 13:04:15.533397  8713 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.529721  8814 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.517970  8724 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.614943  8468 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.619203  8302 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.621376  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.573415  8336 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:15.620234  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:15.626454  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54286.9us
I20260709 13:04:15.626803  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:15.664463  1716 raft_consensus.cc:606] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:15.628044  8675 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.629637  8673 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.630246  8811 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:15.619664  8792 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.633070  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.633845  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.634547  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.635558  1651 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:15.636531  1648 raft_consensus.cc:1240] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:15.633845  8797 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.637032  8781 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:15.637984  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:15.641220  8743 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.524959  8774 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.648352  1650 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23.
W20260709 13:04:15.655589  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 116920us
W20260709 13:04:15.662091  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.525243  8767 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.666698  8766 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:15.668713  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.671464  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.672851  1646 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.699630  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.647212  8713 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:15.677356  8468 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.678225  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.678907  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:15.683578  8316 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.708576  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.688655  8797 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.672101  8781 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:15.719508  8781 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.721091  8781 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:15.723933  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.689617  8386 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:15.725111  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 87179.8us
I20260709 13:04:15.725533  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.725869  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:15.727020  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:15.727902  8615 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:15.728721  8615 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:15.729125  8615 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:15.729516  8615 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.729812  8808 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:15.730229  8790 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.729529  8803 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:15.732788  8800 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 13:04:15.733486  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54662.5us
I20260709 13:04:15.734467  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:15.735875  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:15.736559  8811 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:15.737636  8811 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:15.738933  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:15.740182  8787 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:15.740469  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:15.742107  8787 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:15.742970  8787 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.743701  8787 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:15.703250  1644 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:15.709757  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.712805  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.713296  2378 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:04:15.713961  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.714579  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.715436  2373 raft_consensus.cc:1240] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:15.715930  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.689572  8792 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.747032  1572 raft_consensus.cc:1240] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:15.753183  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.704823  1690 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:15.754087  1649 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:04:15.754411  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49649.2us
I20260709 13:04:15.760615  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.760998  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.757959  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.762663  1648 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:15.755220  1646 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:15.748764  1651 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 23: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:15.766682  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.766628  1646 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:15.711549  8724 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.776698  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.776819  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.781646  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.782123  1643 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:15.777799  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.783053  1719 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:15.783439  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.779819  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:15.784794  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.787109  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.789903  1718 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:15.790053  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.791371  2374 raft_consensus.cc:1240] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:15.744319  8767 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.788800  1688 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:04:15.794926  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59128.6us
I20260709 13:04:15.798681  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.799248  1724 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.770529  8690 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:15.802906  1720 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 23: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:15.803236  1614 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:15.804292  1648 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:15.804555  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.805436  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.805752  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.806150  1648 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:15.806491  2372 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:15.806979  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.811554  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.692107  8814 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.811424  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:15.779040  8802 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:15.819957  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.820138  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.817113  1719 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 24.
I20260709 13:04:15.821475  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.819309  2371 raft_consensus.cc:2393] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 21: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:15.822623  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:15.822893  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.824435  1568 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:15.822428  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:15.823813  1572 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:15.825376  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.822293  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.739194  8615 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.801496  8724 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:04:15.842471  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 46538.4us
I20260709 13:04:15.843299  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:15.826473  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.844489  1651 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:15.844730  8675 consensus_queue.cc:1059] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:15.826722  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.845966  8535 raft_consensus.cc:3055] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:15.846128  1650 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:15.846369  8535 raft_consensus.cc:740] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:15.846470  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:15.827610  8774 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.851858  1570 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:15.828758  8784 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:04:15.829615  8792 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.831274  8766 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:15.789712  8760 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:04:15.789844  8673 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 13:04:15.810983  8817 consensus_queue.cc:1059] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:04:15.817914  8723 consensus_queue.cc:1059] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:04:15.897609  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.898739  2371 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:15.862161  8742 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:15.901638  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.902014  8760 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:15.902737  8760 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:15.904601  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.905565  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.905860  8817 raft_consensus.cc:3055] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:15.906419  8817 raft_consensus.cc:740] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:15.907631  8742 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.908615  8742 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.911465  8742 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.911022  8760 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.810628  8796 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:15.936910  8615 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.834533  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:15.836325  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.801678  8730 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:15.943869  8730 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.944478  8730 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:15.839675  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:15.949067  8708 consensus_meta.cc:287] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.002s
I20260709 13:04:15.949819  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.950589  1645 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.952628  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.954098  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.955327  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.955952  1642 raft_consensus.cc:2376] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23. Re-sending same reply.
I20260709 13:04:15.957538  1575 raft_consensus.cc:1240] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:15.958742  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:15.959399  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 69851.2us
I20260709 13:04:15.961396  8723 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:15.961941  8723 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.962349  8723 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:15.962471  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:15.839934  1572 raft_consensus.cc:1240] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:15.826020  1648 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:15.967218  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.968604  8781 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:15.969462  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.970160  1648 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:15.972520  1724 raft_consensus.cc:2468] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:04:15.973053  1690 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:15.973961  8796 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:15.974016  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:15.974589  8796 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:15.974326  8723 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.857494  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.992131  1644 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:15.860235  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:15.854812  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:15.818857  8690 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:15.868001  1576 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:04:15.872583  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.876713  1647 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:15.825091  8802 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.882606  8814 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.891973  1643 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:15.894902  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.999740  1716 raft_consensus.cc:2393] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:15.999902  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.858475  8316 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:16.005582  1651 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:15.913416  8813 raft_consensus.cc:3055] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:15.847290  8535 consensus_queue.cc:260] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:15.832798  8790 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.906515  1718 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:15.928273  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.907881  8817 consensus_queue.cc:260] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.934109  8386 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:15.935891  8789 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.026072  8802 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:16.027697  8789 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:15.826468  8782 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:16.029140  8782 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:16.021027  8386 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:16.029704  8782 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:16.030755  8386 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:15.939518  8760 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.924037  2376 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:15.940757  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.903805  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:15.975109  8796 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:15.976105  1568 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:15.976682  8787 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.042064  8723 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:16.043385  8787 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:15.979033  8675 raft_consensus.cc:993] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:04:16.044121  8817 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.045229  8675 raft_consensus.cc:1081] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:04:15.978925  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:16.047211  1724 raft_consensus.cc:2393] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:15.980183  8804 consensus_queue.cc:1059] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:15.982213  8765 consensus_queue.cc:1059] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:15.985270  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.052665  1567 raft_consensus.cc:1240] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: [19.17-19.17]
I20260709 13:04:15.986184  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:15.850692  1721 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:15.989234  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:15.993335  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 138594us
I20260709 13:04:15.993734  1571 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:16.054286  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.997052  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:15.896790  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.056114  1573 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:16.002517  1648 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:15.863066  8726 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:15.913329  1719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904"
dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:04:16.009569  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.011575  1649 raft_consensus.cc:2376] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19. Re-sending same reply.
I20260709 13:04:15.974396  8713 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.020731  8673 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:04:16.025098  8790 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.998771  8767 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:16.040184  1570 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:16.019984  1539 connection.cc:256] RPC call timeout handler was delayed by -1.33295302800002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 13:04:16.069653  1539 connection.cc:256] RPC call timeout handler was delayed by -1.3005530279999675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 13:04:16.061183  8804 raft_consensus.cc:3055] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:16.070034  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1268530279999709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 13:04:16.071007  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.002322  8468 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:16.009078  8316 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.012188  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.075730  1647 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 14.
I20260709 13:04:16.058347  1719 raft_consensus.cc:606] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:16.076620  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.011615  8813 raft_consensus.cc:740] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:16.059190  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.087539  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.063179  1645 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:16.066355  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.068671  1643 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:16.072862  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 83740us
I20260709 13:04:16.055543  1644 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:16.078003  1719 raft_consensus.cc:614] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:16.094092  1717 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:16.080444  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:16.096776  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.084744  8316 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:16.096390  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:16.095866  2342 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:16.082293  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.070370  8804 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:16.089496  8468 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.104465  8804 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.105618  8468 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:16.106119  8804 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:16.109390  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.109709  8813 consensus_queue.cc:260] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.110461  1571 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.111235  8813 raft_consensus.cc:3060] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:16.112262  1568 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:16.112571  8816 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:16.113284  8816 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:16.109269  8110 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.128870  1647 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:04:16.114468  8816 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.069370  8767 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.067226  8781 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.116144  1568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:04:16.102896  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.100448  8814 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:16.103749  8790 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.142556  1568 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:16.156992  1568 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:16.145583  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.142864  1645 raft_consensus.cc:2376] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17. Re-sending same reply.
W20260709 13:04:16.160362  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 89410.5us
W20260709 13:04:16.154309  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.163949  1573 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:16.160811  1722 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:16.148888  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.133164  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:16.178854  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.182808  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.184516  1647 raft_consensus.cc:2393] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:16.175736  8720 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:16.185361  8720 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:16.119199  8765 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:16.099184  8743 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.195838  1648 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:04:16.192757  1571 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23.
W20260709 13:04:16.201911  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 144781us
I20260709 13:04:16.130192  8110 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.141697  8767 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:16.202796  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.204373  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.138229  8785 raft_consensus.cc:3055] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:16.134207  8806 raft_consensus.cc:2804] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:16.212940  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.145817  8790 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:16.216435  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.218173  1650 raft_consensus.cc:2376] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19. Re-sending same reply.
I20260709 13:04:16.227564  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.228173  1650 raft_consensus.cc:2376] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19. Re-sending same reply.
I20260709 13:04:16.229274  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.224126  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.167351  8781 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:16.060631  8690 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.221846  1568 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.234638  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.226444  1651 raft_consensus.cc:2376] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23. Re-sending same reply.
I20260709 13:04:16.237674  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.239347  1642 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:16.240835  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.218081  8110 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.133890  8726 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:04:16.188050  8766 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:04:16.258797  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.254508  1568 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:16.221406  8785 raft_consensus.cc:740] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:16.268654  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.255409  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.220798  8765 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:16.276019  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.277590  1651 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 13:04:16.278293  1651 pending_rounds.cc:85] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 9
I20260709 13:04:16.278631  1651 pending_rounds.cc:107] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:04:16.279829  1651 raft_consensus.cc:2889] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.220225  8708 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
W20260709 13:04:16.301167  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.304332  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.306291  1642 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:16.186731  8720 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:16.328835  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84219.4us
I20260709 13:04:16.332386  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:16.243489  8781 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.336535  1645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
new_leader_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:42234
I20260709 13:04:16.342294  1645 raft_consensus.cc:606] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Received request to transfer leadership to TS 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:16.327569  8790 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:16.356360  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 134481us
I20260709 13:04:16.358273  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:16.360008  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:16.364858  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.366283  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.309675  8811 consensus_queue.cc:1059] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:04:16.369895  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.347676  8814 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:16.352977  8801 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:16.401299  1688 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.541s (SENT)
W20260709 13:04:16.407035  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.408013  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:16.408885  2374 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.408013  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.382535  8792 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:16.383249  8713 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:16.404177  8806 raft_consensus.cc:697] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:04:16.432910  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74737.3us
I20260709 13:04:16.429770  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.452832  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.478222  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.409934  8801 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.400329  8765 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.374001  8785 consensus_queue.cc:260] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.467224  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:16.469082  8781 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:16.495597  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:16.521209  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:16.424782  8832 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:16.471131  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.432075  8708 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:16.472513  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:16.473739  1571 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:16.430810  8790 consensus_queue.cc:1059] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:04:16.546059  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.454636  1643 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:04:16.507961  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:16.495597  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:16.508941  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:16.551139  1721 raft_consensus.cc:2368] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23.
I20260709 13:04:16.519528  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.520171  1716 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:16.498314  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:16.553313  1645 raft_consensus.cc:3060] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.496261  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:16.490041  8842 raft_consensus.cc:3055] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:16.490805  8836 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:04:16.540887  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.578748  1575 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:16.485935  8792 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:16.557250  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 429380us
I20260709 13:04:16.564468  1568 raft_consensus.cc:2393] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:16.579418  1574 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:16.549595  1573 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 14.
I20260709 13:04:16.582250  1725 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:16.491063  8814 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:16.471483  8787 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.605446  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:16.606189  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.485584  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:16.610615  1575 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
W20260709 13:04:16.582942  8830 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.064s	user 0.001s	sys 0.001s
W20260709 13:04:16.621520  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 196113us
I20260709 13:04:16.464109  8796 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.614393  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.626703  1568 raft_consensus.cc:3060] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.626672  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:16.634508  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:16.651737  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:16.656971  2374 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:16.604692  8814 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.652621  1725 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:04:16.669106  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45920.6us
I20260709 13:04:16.670814  1645 raft_consensus.cc:2468] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23.
I20260709 13:04:16.623334  8785 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.680879  1723 raft_consensus.cc:1240] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:16.578152  8842 raft_consensus.cc:740] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:16.615469  8787 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:16.665877  8822 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:04:16.678987  8782 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.649s	user 0.018s	sys 0.000s
W20260709 13:04:16.724993  8844 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:04:16.727238  8787 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.626956  8302 consensus_queue.cc:1059] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:04:16.605813  8806 consensus_queue.cc:237] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.562713  8832 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.741016  1575 raft_consensus.cc:1275] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:16.677983   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:16.780942  1574 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:16.678915  8814 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:16.777192   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:16.799494  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45937.9us
I20260709 13:04:16.678915  8817 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.811792  1614 leader_election.cc:400] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23.
I20260709 13:04:16.812314  1614 leader_election.cc:403] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:16.689785  8839 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.797983  8302 consensus_queue.cc:1243] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer cd6d377efe514bc58ae49c3f4a107232 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:04:16.811710  8535 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:16.841135  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 151733us
I20260709 13:04:16.723845  8782 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:16.840574  8826 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.832s	user 0.009s	sys 0.003s
I20260709 13:04:16.855782  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:16.860818  8824 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.850s	user 0.000s	sys 0.004s
I20260709 13:04:16.865483  1724 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
W20260709 13:04:16.873610  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 387992us
I20260709 13:04:16.874145  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:16.878156  1568 raft_consensus.cc:2468] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:16.841728  8836 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23.
I20260709 13:04:16.880172  1646 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:16.819386  8817 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:16.882437  1642 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:04:16.869017  8804 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.762s	user 0.014s	sys 0.010s
I20260709 13:04:16.893533  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:16.915277  8690 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:16.916967  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:16.777216  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.890352  1642 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:16.888687  8781 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:04:16.928977  1642 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:16.939231  1642 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:16.892971  8730 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.940s	user 0.013s	sys 0.013s
W20260709 13:04:16.944494  1651 consensus_meta.cc:287] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.012s
I20260709 13:04:16.945768  1651 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
W20260709 13:04:16.946317  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 90672.8us
I20260709 13:04:16.892032  8811 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:16.889317  8850 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:16.947621  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:16.951849  8851 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.956800  8801 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:16.969053  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.946647  8837 log.cc:927] Time spent T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
I20260709 13:04:17.013707  8850 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:16.949965  8811 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:17.016309  8811 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:17.019053  8743 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.806s	user 0.010s	sys 0.012s
I20260709 13:04:17.019783  8743 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:16.999039  8806 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:17.021443  8806 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:17.027693  8831 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.259s	user 0.001s	sys 0.001s
I20260709 13:04:17.029673  8468 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:17.030283  8468 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.030712  8468 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:17.027879  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.031886  1718 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:04:17.066347  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:16.995087  8766 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:16.888639  8848 consensus_queue.cc:1059] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.205s
I20260709 13:04:16.869025  8842 consensus_queue.cc:260] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.096561  8848 raft_consensus.cc:3055] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:16.834468  8839 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:17.089389  8838 log.cc:927] Time spent T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.482s	user 0.000s	sys 0.002s
I20260709 13:04:17.014559  8850 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:17.143348  1718 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 13:04:17.146056  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.146853  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 185725us
W20260709 13:04:17.132540  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 193436us
I20260709 13:04:17.000871  8851 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:17.099277  8856 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:17.097149  8848 raft_consensus.cc:740] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:17.163978  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.174069  1572 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:17.168267  1718 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:17.169183  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:17.166646  1690 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:17.191044  1568 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:04:17.191576  1568 pending_rounds.cc:85] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 11
I20260709 13:04:17.196969  1568 pending_rounds.cc:107] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:04:17.197345  1568 raft_consensus.cc:2889] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.194625  1690 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:17.181633  8302 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:04:17.193320  8690 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.205756  8302 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:04:17.206169  8813 consensus_meta.cc:287] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.094s	user 0.016s	sys 0.015s
I20260709 13:04:17.193668  8841 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:04:17.206533  8302 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:16.949242  8730 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:17.196036  8316 consensus_meta.cc:287] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.092s	user 0.021s	sys 0.014s
I20260709 13:04:17.204691  1642 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:04:17.220582  1642 pending_rounds.cc:85] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 15
I20260709 13:04:17.222085  1642 pending_rounds.cc:107] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 23.16
W20260709 13:04:17.221031  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 304051us
I20260709 13:04:17.194116  8856 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:17.228859  1649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2577) took 1258 ms. Trace:
I20260709 13:04:17.223563  1642 raft_consensus.cc:2889] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:17.225693  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:17.201233  8760 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:17.232565  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63452.1us
I20260709 13:04:17.171573  8804 consensus_queue.cc:1059] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:04:17.229436  1649 rpcz_store.cc:276] 0709 13:04:15.970211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.078637 (+108426us) service_pool.cc:224] Handling call
0709 13:04:16.101216 (+ 22579us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:16.187613 (+ 86397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.187653 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:16.198182 (+ 10529us) log.cc:844] Serialized 52 byte log entry
0709 13:04:16.389462 (+191280us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:16.389517 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.389605 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.228505 (+838900us) raft_consensus.cc:1685] finished
0709 13:04:17.228538 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.228759 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30561,"raft.queue_time_us":34169,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":26063,"thread_start_us":141127,"threads_started":1,"wal-append.queue_time_us":255093}
I20260709 13:04:17.233425  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:17.195899  8832 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.209677  8782 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.241107  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.244074  1568 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.227162  8316 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:17.250759  8854 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:04:17.232414  8723 raft_consensus.cc:2804] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:17.252281  8316 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.252880  8302 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.269326  8850 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:17.270036  8850 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:17.270629  8850 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:17.270102  8302 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:17.274542  8851 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:17.284544  8817 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.266218  8570 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.266316  8760 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:17.328589  8760 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:17.329656  8767 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 1.096s	user 0.016s	sys 0.001s
I20260709 13:04:17.329766  8690 raft_consensus.cc:3055] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:17.330456  8690 raft_consensus.cc:740] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:17.261063  1650 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:17.271581  8850 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.269748  8864 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.333319  8675 raft_consensus.cc:3055] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:17.296602  8806 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.299623  8730 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.301365  2376 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:17.302191  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.303926  2378 raft_consensus.cc:1240] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:17.304798  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.305817  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.309108  8839 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.311643  8673 raft_consensus.cc:993] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: : Instructing follower 4f737473248e4c059fa74af3b8e1f77e to start an election
W20260709 13:04:17.257396  8859 log.cc:927] Time spent T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:04:17.257524  8862 consensus_queue.cc:1059] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:04:17.238183  8848 consensus_queue.cc:260] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.258821  8708 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:04:17.286945  8856 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.377579  8570 raft_consensus.cc:1081] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Signalling peer 4f737473248e4c059fa74af3b8e1f77e to start an election
I20260709 13:04:17.262655  1690 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:17.360157  8110 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.090s	user 0.022s	sys 0.000s
I20260709 13:04:17.265738  8784 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:04:17.266038  8813 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:04:17.356452  8782 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:17.291680  8386 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.330253  8767 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:17.333648  8785 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.009s
I20260709 13:04:17.373165  1717 raft_consensus.cc:2468] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:17.331980  8690 consensus_queue.cc:260] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:17.265429  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:17.415463  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 150146us
I20260709 13:04:17.416385  1688 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:04:17.390036  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 127420us
I20260709 13:04:17.351277  8675 raft_consensus.cc:740] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:17.345860  8796 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.424789  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.336567  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49750 (request call id 2703) took 1533 ms. Trace:
I20260709 13:04:17.409906  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.360209  2371 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:17.416510  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.429442  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.430166  1572 raft_consensus.cc:2393] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:17.418259  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.430224  1574 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 14.
I20260709 13:04:17.395589  8570 raft_consensus.cc:2804] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:17.431372  1574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.430876  1567 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:17.435369  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.428583  1646 rpcz_store.cc:276] 0709 13:04:15.803079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.803223 (+   144us) service_pool.cc:224] Handling call
0709 13:04:17.320080 (+1516857us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003fc854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:17.336445 (+ 16365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435719936}
I20260709 13:04:17.433279  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.437258  1572 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:17.436031  1568 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 21.
I20260709 13:04:17.436071  1567 raft_consensus.cc:2376] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 18. Re-sending same reply.
I20260709 13:04:17.440706  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.442241  1567 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:17.451104  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:17.454087  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.456977  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 255162us
I20260709 13:04:17.419849  8765 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:17.356606  8864 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:17.460424  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.424018  8535 consensus_meta.cc:287] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.574s	user 0.019s	sys 0.009s
W20260709 13:04:17.460057  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.625s (SENT)
I20260709 13:04:17.467551  2344 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:17.396142  8110 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.471328  1568 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:17.473858  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:17.409376  8782 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.484438  1576 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:17.461761  8760 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.466645  8690 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:17.488437  8535 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:17.491052  8535 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:17.491495  8535 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:17.497066  8869 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:17.498592  8869 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:17.503378  8767 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.483394  8841 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:17.508222  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3"
dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
 from {username='slave'} at 127.0.0.1:44760
I20260709 13:04:17.508953  1569 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:17.509635  1569 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.496892  8861 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:17.508749  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.537449  1567 raft_consensus.cc:2376] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 21. Re-sending same reply.
I20260709 13:04:17.458148  8785 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:17.513043  1719 raft_consensus.cc:1240] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:17.470098  8570 raft_consensus.cc:697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:17.531805  8675 consensus_queue.cc:260] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.560542  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.561902  1575 raft_consensus.cc:2393] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:17.564812  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.461324  8723 raft_consensus.cc:697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:17.563968  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 103950us
W20260709 13:04:17.568252  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 151924us
I20260709 13:04:17.578823  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:17.577055  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 146336us
I20260709 13:04:17.582211  1616 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:17.577663  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.719s (SENT)
I20260709 13:04:17.584576  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:17.593145  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.742s (SENT)
I20260709 13:04:17.559943  8861 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.605198  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:17.573886  8787 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.846s	user 0.027s	sys 0.013s
I20260709 13:04:17.590261  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.560135  8828 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.589833  8110 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.553472  8781 raft_consensus.cc:3055] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:17.553900  8796 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.601524  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.603121  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:17.625051  1569 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.553489  8785 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:17.607128  8782 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:17.607290  8468 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.576s	user 0.020s	sys 0.016s
I20260709 13:04:17.536921  8760 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.644084  8468 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.614396  1572 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:17.597958  8876 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:17.598407  8841 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.619143  8861 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.620189  8874 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.602450  8871 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:17.611928  8675 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:17.648656  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:17.663463  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.654954  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.664774  1575 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
I20260709 13:04:17.665686  1572 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:17.654134  1569 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.570245  8723 consensus_queue.cc:237] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:17.674131  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 83960.1us
I20260709 13:04:17.675916  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.680843  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:17.636142  8767 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.683419  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:17.637032  8781 raft_consensus.cc:740] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:17.646721  8785 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:17.688221  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:17.689535  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:17.661923  8877 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.002s
I20260709 13:04:17.649400  8876 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.715193  8876 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:17.649175  8302 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:17.661796  8861 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.716928  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.646095  8828 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.720080  8861 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.652908  8796 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:17.629762  8787 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:17.671360  8814 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.839s	user 0.010s	sys 0.021s
I20260709 13:04:17.708599  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:17.712401  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 85319.5us
I20260709 13:04:17.714110  8885 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:17.683382  8817 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:17.722200  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.696312  8110 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:17.718528  8302 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:17.754570  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.768486  1571 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:04:17.755609  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.915s (SENT)
I20260709 13:04:17.769639  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:17.695540  8871 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:17.753137  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 137070us
I20260709 13:04:17.682248  8874 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.696225  8760 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.699805  8316 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.770668  8828 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:17.771281  8861 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:17.807753  8861 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.810195  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:17.810762  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.811398  1568 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:17.746644  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58502.3us
I20260709 13:04:17.779898  8767 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:17.710659  8864 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.779633  1573 raft_consensus.cc:1240] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:17.755059  8766 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.772373  8302 raft_consensus.cc:3060] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:17.815068  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.816601  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.817230  1571 raft_consensus.cc:2376] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da3b57321031416cba9db75450ef302a in term 18. Re-sending same reply.
W20260709 13:04:17.817808  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3310) took 1351 ms (client timeout 1670 ms). Trace:
I20260709 13:04:17.656222  8570 consensus_queue.cc:237] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:17.818668  1717 rpcz_store.cc:269] 0709 13:04:16.466651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.466902 (+   251us) service_pool.cc:224] Handling call
0709 13:04:17.817514 (+1350612us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54003aaf54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:17.817718 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588404480}
W20260709 13:04:17.825052  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.827270  8848 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:17.763351  8814 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.845678  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:17.801831  1569 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:17.760658  8885 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:17.848096  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 126298us
W20260709 13:04:17.848537  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 74158.7us
I20260709 13:04:17.802560  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:17.802119  1719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3311) took 1333 ms. Trace:
I20260709 13:04:17.849905  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:17.850730  1648 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:17.802788  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
W20260709 13:04:17.800959  1650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2596) took 1693 ms (client timeout 2176 ms). Trace:
I20260709 13:04:17.804883  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:17.812637  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:17.843281  1573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3318) took 1314 ms. Trace:
W20260709 13:04:17.852483  1650 rpcz_store.cc:269] 0709 13:04:16.107925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.228858 (+120933us) service_pool.cc:224] Handling call
0709 13:04:17.800662 (+1571804us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003fbe54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:17.800869 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369667968}
I20260709 13:04:17.828318  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.849282  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.805883  8874 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:17.797266  8811 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.022s
I20260709 13:04:17.846949  1569 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:17.847779  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:17.858947  1646 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:17.832962  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.849293  1614 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:17.811553  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.791478  8673 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:17.785764  8468 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:17.853468  1573 rpcz_store.cc:276] 0709 13:04:16.528587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.625092 (+ 96505us) service_pool.cc:224] Handling call
0709 13:04:16.625647 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.813741 (+1188094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.843004 (+ 29263us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000276d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:17.843209 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2652288256}
I20260709 13:04:17.843791  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:17.868184  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:17.897253  1571 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 17.
I20260709 13:04:17.855257  8782 consensus_queue.cc:1059] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:17.840587  8781 consensus_queue.cc:260] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.856796  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.849609  1719 rpcz_store.cc:276] 0709 13:04:16.469164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.469353 (+   189us) service_pool.cc:224] Handling call
0709 13:04:16.469753 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.539458 (+1069705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.801850 (+262392us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001a81d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:17.802049 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397428096}
W20260709 13:04:17.857576  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45057.3us
I20260709 13:04:17.864233  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:17.879833  1569 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:17.856264  8806 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:17.916455  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:17.830847  8862 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:17.888660  1568 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:04:17.888782  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:17.919013  1642 raft_consensus.cc:2393] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:17.877842  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.864838  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.889603  1570 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:17.899504  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.920531  2371 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:17.892899  8843 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.899924  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.933455  1721 raft_consensus.cc:2393] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:17.901057  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:17.902032  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.892990  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:17.894887  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:17.891975  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:17.935909  1645 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:17.936717  1643 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:17.866240  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:17.906072  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:17.937702  1650 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.893724  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:17.911104  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:17.890153  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.939957  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:17.940765  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 91511.5us
I20260709 13:04:17.941186  2376 raft_consensus.cc:2368] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19.
I20260709 13:04:17.924849  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:17.925639  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.927363  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:17.924558  8862 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:17.951151  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61038.4us
W20260709 13:04:17.857839  8790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.031s	sys 0.061s
I20260709 13:04:17.927472  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:17.928215  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.954692  1724 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:17.858366  8885 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:17.896286  8765 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:17.907421  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:17.930750  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:17.890221  8811 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:17.940924  1642 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:17.988241  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:17.810338  8861 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:18.001636  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:17.941581  1720 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:18.002840  1723 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:17.943112  2374 raft_consensus.cc:2393] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:17.858769  8888 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.948118  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.008921  1651 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:17.948984  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:17.889801  8673 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:17.949765  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.015374  1647 raft_consensus.cc:2468] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:17.955120  1573 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:17.950784  8468 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.955935  1570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44746 (request call id 3308) took 1453 ms. Trace:
W20260709 13:04:18.019263  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 77551.7us
I20260709 13:04:17.985146  8813 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:04:18.023540  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 96366.8us
I20260709 13:04:17.955116  1724 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:17.960325  1567 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:17.960875  8790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.031s	sys 0.061s
I20260709 13:04:17.962081  8535 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:17.962286  8869 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:17.966027  8842 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.968916  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:17.969223  8796 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:17.924011  1569 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.978910  1722 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:17.986932  1690 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:17.990458  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 83218.6us
W20260709 13:04:17.988271  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46413.9us
I20260709 13:04:18.038558  1650 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
W20260709 13:04:17.992082  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 61437.9us
I20260709 13:04:17.994428  1645 raft_consensus.cc:2468] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
I20260709 13:04:17.928181  8871 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.014509  2371 raft_consensus.cc:2393] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:18.025836  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:18.027345  1644 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:18.007218  8767 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:18.005558  8743 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.047127  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:18.048287  1651 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.973668  8876 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.017838  1570 rpcz_store.cc:276] 0709 13:04:16.502529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.523404 (+ 20875us) service_pool.cc:224] Handling call
0709 13:04:16.523801 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.920602 (+1396801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.955663 (+ 35061us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54001a9fd4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:17.955872 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139575680}
I20260709 13:04:18.033528  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:18.051052  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 64209.6us
I20260709 13:04:18.026702  1724 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.051716  1647 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:18.017194  8386 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.013s
I20260709 13:04:18.018010  8787 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:18.036000  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:18.037425  1642 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:18.038529  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:17.975065  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:18.066974  1718 raft_consensus.cc:2410] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 20 because replica has last-logged OpId of term: 19 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 13:04:18.050207  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:18.022023  8766 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:18.052528  1724 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.974171  8884 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:18.030268  1717 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:04:18.078915  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53126.6us
I20260709 13:04:18.064323  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:18.072053  2374 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:18.070251  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:18.048822  8767 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.078195   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:18.083040  1539 leader_election.cc:400] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19.
I20260709 13:04:18.091331  8889 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:18.095714  8781 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.097048  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:18.097927  8673 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.098671  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:18.100564  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.102064  8767 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:18.068675  8874 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:18.078572  8884 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.063349  8871 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.105659  8386 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.107214  8535 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:18.109164  1721 raft_consensus.cc:1240] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:18.110374  8766 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:18.059824  8888 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.112874  8790 consensus_queue.cc:1059] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:04:18.115191  8792 raft_consensus.cc:3055] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:18.054958  8862 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.117544  8742 raft_consensus.cc:3055] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:18.122090  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:18.124861  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44815.7us
I20260709 13:04:18.217440  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:18.126296  8811 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:18.127694  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63454.6us
I20260709 13:04:18.129770  1716 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:18.130581  1539 leader_election.cc:403] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:04:18.174755  8801 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.053736  8708 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.128496  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:18.226764  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:18.227093  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 106801us
I20260709 13:04:18.134687  1569 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:04:18.063812  8723 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:18.134133  1614 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:18.152169  1647 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:18.168255  1724 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:18.119726  8302 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.137586  8813 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:18.146751  8863 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:18.144405  8889 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:18.213780  1721 raft_consensus.cc:1275] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:18.215550  8792 raft_consensus.cc:740] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:04:18.220881  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 138045us
I20260709 13:04:18.216857  8814 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:18.223052  1722 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:18.216702  8742 raft_consensus.cc:740] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:18.227406  1573 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:18.227672  1569 pending_rounds.cc:85] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Aborting all ops after (but not including) 12
I20260709 13:04:18.266745  1569 pending_rounds.cc:107] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:04:18.267112  1569 raft_consensus.cc:2889] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:18.258395  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.228331  2342 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:18.268903  1576 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:18.176621  8884 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:18.215440  8673 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:18.240922  8813 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.145309  8861 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 13:04:18.215013  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.216722  8316 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.257429  8760 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:18.174856  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 221119us
I20260709 13:04:18.224470  1716 raft_consensus.cc:1240] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:18.250316  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:18.258384  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 124340us
I20260709 13:04:18.262854  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:18.227064  1720 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 22: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:18.271495  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 54164.2us
I20260709 13:04:18.229449  8468 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19.
I20260709 13:04:18.236606  8839 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:18.236065  8723 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.242323  8708 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.130623  1651 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
I20260709 13:04:18.248831  8889 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:18.273077  8811 raft_consensus.cc:2804] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:18.272928  8862 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:18.264379  8876 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:18.325839  8904 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:18.245100  8110 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:04:18.236690  8801 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.330669  8801 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:18.331121  8871 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:18.332648  8893 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:04:18.333371  1649 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:18.216658  8888 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.242672  8874 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:18.277591  8835 raft_consensus.cc:2804] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:18.306960  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:18.309978  1573 raft_consensus.cc:2468] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23.
I20260709 13:04:18.250447  8863 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.291330  8813 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:18.308482  8861 consensus_queue.cc:1243] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:04:18.290104  8841 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:04:18.311324  8723 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.355738  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:18.366626  2371 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:18.356738  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:18.356930  8863 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:18.354489  8896 log.cc:927] Time spent T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:04:18.354473  8835 raft_consensus.cc:697] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:18.398684  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:18.295018  8316 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.424831  1724 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:18.427105  1645 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:04:18.434191  1614 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.341877  8781 raft_consensus.cc:2764] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:04:18.441375  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:18.456620  1722 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:04:18.441973  8302 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:18.442272  8880 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:18.446775  8836 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.361488  8386 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:04:18.452922  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:18.363906  8708 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:18.463788  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 179668us
I20260709 13:04:18.353040  8811 raft_consensus.cc:697] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:18.311551  8883 consensus_queue.cc:1059] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.115s
W20260709 13:04:18.475696  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 131808us
I20260709 13:04:18.475958  1690 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:18.335952  8868 consensus_queue.cc:1059] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.124s
I20260709 13:04:18.495451  8883 consensus_queue.cc:1243] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:18.322579  8742 consensus_queue.cc:260] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.319332  8766 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.311750  8792 consensus_queue.cc:260] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.500073  1690 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:18.350440  8874 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:18.494799  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 217452us
I20260709 13:04:18.513902  8302 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:18.531167  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.534425  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 226928us
W20260709 13:04:18.543867  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43839.6us
I20260709 13:04:18.547304  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.548177  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.548296  1569 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:18.550571  1568 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:04:18.552145  1569 raft_consensus.cc:2368] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:04:18.549203  1570 raft_consensus.cc:2393] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 22: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:18.550803  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.488534  8904 consensus_queue.cc:1048] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:04:18.476831  8836 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.519398  8743 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:18.557494  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:18.529726  8785 consensus_meta.cc:287] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.779s	user 0.015s	sys 0.000s
I20260709 13:04:18.521724  8880 raft_consensus.cc:3055] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:18.561375  1647 raft_consensus.cc:1217] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 13.7->[14.8-19.10]   Dedup: 17.9->[19.10-19.10]
I20260709 13:04:18.563187  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.564076  1642 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:18.569149  1647 raft_consensus.cc:1240] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: [14.8-19.10]
I20260709 13:04:18.564468  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.565028  1570 raft_consensus.cc:2393] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 20: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:18.535905  8302 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.573287  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:18.569419  1576 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:18.574148  1649 raft_consensus.cc:2376] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19. Re-sending same reply.
I20260709 13:04:18.578256  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.579934  1651 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:18.559182  8792 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:18.634193  1651 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:18.567653  8880 raft_consensus.cc:740] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:18.582162  8302 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.629462  8570 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.584712  8906 raft_consensus.cc:2804] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:18.573550  8726 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:04:18.620324  8743 consensus_queue.cc:1048] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 13:04:18.650481  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.622694  8708 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.681466  8570 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.760955  8708 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.761622  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.762529  1575 raft_consensus.cc:2393] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:18.645267  8801 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.767104  8892 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.580s	user 0.004s	sys 0.001s
I20260709 13:04:18.766570  8801 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.767469  8806 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:18.767966  8806 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:04:18.653467  8885 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.683s	user 0.018s	sys 0.000s
I20260709 13:04:18.768880  8885 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.769164  8806 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.770741  8806 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:18.771647  8885 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:18.686730  8903 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.772676  8903 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.775143  8903 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:18.687219  8316 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:18.684208  1688 leader_election.cc:336] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Timed out: RequestConsensusVote RPC to 127.0.107.194:43325 timed out after 1.572s (SENT)
W20260709 13:04:18.776376  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92147.4us
I20260709 13:04:18.689055  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:18.780395  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.781701  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:18.782449  8767 consensus_meta.cc:287] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.004s
I20260709 13:04:18.783411  8767 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.689718  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.784781  1647 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:18.785982  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.786690  1647 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:18.763803  1575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:18.787160  8884 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.787825  1575 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:18.693471  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.697165  8888 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:18.697602  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:18.789168  1576 raft_consensus.cc:2368] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
I20260709 13:04:18.789856  8767 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:18.790388  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.791375  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:18.717270  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:18.704574  1574 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:18.793289  8708 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:18.722932  1725 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:18.797480  1690 leader_election.cc:400] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:04:18.797725  2371 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:18.703706  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.798869  8801 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:18.573350  8785 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.583032  8811 consensus_queue.cc:237] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.801194  1575 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:18.801592  8570 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:18.802703  8889 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.712951  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:18.808365  8792 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.809259  8792 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.810766  1649 raft_consensus.cc:2468] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:18.730624  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.814739  1569 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:04:18.815342  8792 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:18.732199  8850 raft_consensus.cc:3055] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:18.817402  8850 raft_consensus.cc:740] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:18.822360  8884 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:18.725701  8908 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:04:18.823770  8880 consensus_queue.cc:260] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.825934  8880 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:18.737115  8899 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.296s	user 0.000s	sys 0.002s
I20260709 13:04:18.742931  8781 raft_consensus.cc:3055] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:18.826719  8781 raft_consensus.cc:740] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:18.828235  8781 consensus_queue.cc:260] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.829676  8781 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.745870  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.746747  8742 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:18.831808  1571 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:18.799239  1567 raft_consensus.cc:2393] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:18.750360  8909 log.cc:927] Time spent T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:04:18.833758  1571 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:04:18.749943  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 499529us
I20260709 13:04:18.835372  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:18.837363  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:18.750739  8906 raft_consensus.cc:697] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:18.839953  8906 consensus_queue.cc:237] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.842128  8868 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:18.750888  8468 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:18.842896  8868 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.843298  8468 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.843938  8468 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.556228  8835 consensus_queue.cc:237] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.844587  8843 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.845245  8316 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.845498  8843 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.846695  8782 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.847251  8782 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.703972  1642 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:18.704666  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:18.707216  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:18.706243  1646 raft_consensus.cc:1275] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:04:18.755090  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.702848  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.756805  8760 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.757400  8817 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:18.757560  8904 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:18.756886  8790 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:18.710008  8874 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:18.760411  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.877086  1651 raft_consensus.cc:2468] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:18.787750  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.880730  1647 raft_consensus.cc:1813] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Handling vote request from an unknown peer 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:18.881348  1647 raft_consensus.cc:2368] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:04:18.884547  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:18.885543  1725 raft_consensus.cc:2410] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:04:18.691883  1643 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:18.885044  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.588s (SENT)
I20260709 13:04:18.886596  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:18.797994  1690 leader_election.cc:403] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:18.800541  1716 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:18.716866  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.888685  1572 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
W20260709 13:04:18.877373  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81489.2us
W20260709 13:04:18.748211  8905 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:04:18.845824  8316 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.850167  1650 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:18.850304  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:18.850778  1642 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:18.721168  8836 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:18.853740  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 297482us
I20260709 13:04:18.854712  1648 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:18.856571  1616 leader_election.cc:400] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 16. Current term is 17.
W20260709 13:04:18.735996  8842 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.685s	user 0.011s	sys 0.003s
I20260709 13:04:18.859769  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.862711  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.739131  8813 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.889019  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:18.910247  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:18.910895  1567 raft_consensus.cc:3060] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:18.911075  1720 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:18.844696  8885 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:18.894811  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:18.819139  8850 consensus_queue.cc:260] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.897614  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:18.894976  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:18.895148  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 139543us
I20260709 13:04:18.893676  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:18.901979  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 51819.1us
I20260709 13:04:18.914757  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:18.915541  1651 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:18.915944  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:04:18.901917  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45544.3us
I20260709 13:04:18.909771  1642 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.852716  8787 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:18.933786  1572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:18.934588  1718 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:18.934953  1572 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:04:18.933899  1717 raft_consensus.cc:2368] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18.
I20260709 13:04:18.936152  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:18.919301  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:18.920661  8869 raft_consensus.cc:3055] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:18.879034  8817 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.921833  8885 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.924858  8801 raft_consensus.cc:3060] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.925312  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:18.926293  8883 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:18.929836  1570 raft_consensus.cc:2410] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 22 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:04:18.862140  8708 raft_consensus.cc:3055] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:18.937803  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:18.992966  2371 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 18: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:18.866237  8792 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:18.994577  8792 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:18.995289  8792 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.863577  8843 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:18.865124  8785 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:18.926079  8316 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:18.999269  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74028.7us
I20260709 13:04:18.954406  1690 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
W20260709 13:04:18.969082  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.894850  8767 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.978580  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:18.981299  1642 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:18.985001  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.990191  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.990110  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52579us
I20260709 13:04:18.934124  8814 raft_consensus.cc:2764] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:04:19.018519  8814 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:18.934267  8874 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:19.021939  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.019501  8814 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.864210  8760 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.930047  8302 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:18.953812  1716 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 15.
I20260709 13:04:18.993597  8787 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:19.002353  8875 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.002480  8843 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:19.000460  1614 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:19.009094  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50799.9us
I20260709 13:04:19.017225  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:19.043614  1650 raft_consensus.cc:2393] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:19.010357  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56015.3us
I20260709 13:04:18.891700  8790 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:18.953784  8869 raft_consensus.cc:740] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:18.991801  8905 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:04:18.991355  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 212853us
I20260709 13:04:18.992148  8850 raft_consensus.cc:3060] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.003255  8911 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:19.032939  1645 raft_consensus.cc:1240] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:18.998222  8708 raft_consensus.cc:740] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:19.036067  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.040078  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.059286  2374 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:19.040537  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:19.047168  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46816.9us
I20260709 13:04:19.025410  8883 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:19.050513  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:19.066932  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.025614  8785 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:19.051215  1567 raft_consensus.cc:2468] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:19.063119  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:19.086633  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.063870  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.062352  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.087706  2374 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:19.078385  1572 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:19.055116  8876 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:19.100484  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.101864  1642 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:19.105126  1616 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.097749  1614 leader_election.cc:400] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18.
I20260709 13:04:19.105897  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.110742  1614 leader_election.cc:403] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:19.111888  8782 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.112582  8813 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:19.114636  8814 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.116384  8885 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.047916  8730 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:04:19.119778  1568 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:19.110996  1570 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:19.122870  1567 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:19.126235  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:19.128868  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.128664  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.129653  2374 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 23.
I20260709 13:04:19.094012  8785 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.132587  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:19.129506  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:19.051293  8316 consensus_queue.cc:1059] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:04:19.096799  8843 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:19.134235  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:19.134292  1616 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:19.136947  2371 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
W20260709 13:04:19.141268  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 222045us
I20260709 13:04:19.094906  8848 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:19.102926  8883 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:19.112440  8889 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.148711  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:19.065907  8796 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:04:19.048225  8875 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.157040  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.146145  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:19.159736  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
I20260709 13:04:19.187748  1576 raft_consensus.cc:2368] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:04:19.095325  8868 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.161605  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.186223  1569 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
W20260709 13:04:19.184422  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50221us
I20260709 13:04:19.191166  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.135859  8861 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:19.185159  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:19.157114  8723 consensus_meta.cc:287] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.004s
I20260709 13:04:19.135733  8885 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:19.156893  8895 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:04:19.135412  8913 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:19.144064  8785 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.135541  8876 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:19.197394  1616 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:19.199174  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.151391  8842 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:19.144807  8843 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.200377  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e"
W20260709 13:04:19.202010  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55982.8us
I20260709 13:04:19.205446  1567 raft_consensus.cc:3060] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.205703  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:19.158572  8883 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:19.209426  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 47911.1us
I20260709 13:04:19.207929  1688 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:19.209748  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:19.189664  8862 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:19.223263  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 181806us
I20260709 13:04:19.230480  1573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.240926  1573 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:19.192919  8874 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:19.239984  1571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.193197  8726 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:19.209116  8787 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:19.260960  1616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.260730  1573 raft_consensus.cc:1240] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:04:19.261587  1719 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:19.198050  8861 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:04:19.262740  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53059.6us
I20260709 13:04:19.238179  1567 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:19.266832  1688 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:04:19.267681  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100980us
I20260709 13:04:19.202934  8889 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:19.210493  8843 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.243067  8862 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.158042  8863 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:04:19.206663  8814 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:19.220405  8883 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 1
W20260709 13:04:19.243791  8916 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 13:04:19.302052  1690 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:04:19.239257  8675 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:04:19.239934  8868 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:19.192432  8468 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.243342  8781 raft_consensus.cc:2804] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:19.220520  8903 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.211442  8766 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:19.366302  8922 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:04:19.368525  8924 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:04:19.369278  8923 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 13:04:19.370680  8928 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:04:19.374634  8903 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:19.303998  8917 log.cc:927] Time spent T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 13:04:19.272003  8535 raft_consensus.cc:2804] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:19.206280  8723 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.383229  8723 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:19.366398  8570 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:19.385200  8570 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.385679  8468 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.265591  8787 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:19.386889  8787 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:19.338884  8784 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:19.388904  8760 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:19.389932  8908 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:19.332109  8814 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.390836  8925 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:19.391556  8880 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.007s
I20260709 13:04:19.325594  8862 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.305755  8110 consensus_queue.cc:1048] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:04:19.361405  8868 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.144361  8869 consensus_queue.cc:260] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:19.330384  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 63595.6us
I20260709 13:04:19.330593  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.302090  8861 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.273263  1576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.307931  8726 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.310169  8792 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.313895  8817 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.380012  8876 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.404546  1576 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:19.399683  1688 leader_election.cc:400] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:04:19.244550  8888 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:19.415987  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 85478.2us
I20260709 13:04:19.385692  8570 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.431169  8302 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.442572  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:19.397158  8868 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:19.454182  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:19.392307  8814 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.397414  8110 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:19.398173  8781 raft_consensus.cc:697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:19.420747  8468 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:19.480726  8468 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:19.481971  8468 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.482262  8708 consensus_queue.cc:260] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.483588  8708 raft_consensus.cc:3060] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:19.484689  8766 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:19.484895  8535 raft_consensus.cc:697] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:19.459694  8784 raft_consensus.cc:697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:19.486567  8784 consensus_queue.cc:237] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.486505  8535 consensus_queue.cc:237] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.464855  8110 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:19.489279  8110 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.484757  8817 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.490690  8110 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:19.490908  8817 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.419814  8726 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:19.499315  8926 log.cc:927] Time spent T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:04:19.500429  8781 consensus_queue.cc:237] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.449577  8888 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.388896  8904 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:04:19.449340  8790 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:19.514705  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68179.3us
I20260709 13:04:19.515635  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:19.516620  8903 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:19.518354  8903 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:19.518771  8903 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.519158  8903 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:19.460201  8925 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:19.485487  8814 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:19.553185  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.553373  1570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4f737473248e4c059fa74af3b8e1f77e" is_pre_election: true
I20260709 13:04:19.554847  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.569792  1572 raft_consensus.cc:1240] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:04:19.605403  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:19.606631  1647 raft_consensus.cc:2393] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:19.554167  1570 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:04:19.572911  1615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.575559  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44893.2us
I20260709 13:04:19.571245  8742 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:19.538409   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:04:19.578307  8895 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:19.584530  8817 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:19.588318  8675 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:19.589361  8842 consensus_queue.cc:1059] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:19.619720  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:19.620297  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:19.621680  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:19.622421  1646 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:19.575382  8880 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.603806  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:19.611938  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:19.623826  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.618770  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:19.634459  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:19.634667  1648 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:19.596771  8782 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.630376  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.638742  2374 raft_consensus.cc:1275] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:19.639142  2371 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:19.614573  1650 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:04:19.629690  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:19.641248  2376 raft_consensus.cc:2368] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:04:19.598882  8726 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.624754  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:19.575493  8766 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:19.672063  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 47385.9us
I20260709 13:04:19.651782  1650 raft_consensus.cc:1275] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:04:19.662297  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 300726us
I20260709 13:04:19.674256  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:19.672580  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:19.676647  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.676774  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:19.677529  2376 raft_consensus.cc:3060] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.675951  1651 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.638151  8792 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:19.680940  1690 leader_election.cc:400] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
W20260709 13:04:19.665513  8912 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:19.690663  1690 leader_election.cc:403] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:04:19.665207  8790 raft_consensus.cc:2804] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:19.698765  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:19.640221  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.694540  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.633894  8767 consensus_queue.cc:1059] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
W20260709 13:04:19.692924  8801 consensus_meta.cc:287] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.000s
I20260709 13:04:19.709234  1719 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:19.685009  8796 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:19.721441  1651 raft_consensus.cc:2468] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:19.645151  8880 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:19.712929  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 72680.9us
I20260709 13:04:19.709286  1690 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:19.730428  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.675827  8782 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.695462  8875 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.715557  8302 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:04:19.719314  8814 raft_consensus.cc:2764] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:04:19.739506  1690 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:19.711668  8888 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:19.697180  8790 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:19.724318  8796 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:19.747308  8801 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275) with higher term. Message: Invalid argument: T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19.
I20260709 13:04:19.761622  8675 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:19.781441  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.979s (SENT)
I20260709 13:04:19.723521  8911 consensus_queue.cc:1048] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:04:19.788710  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:19.740960  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.768154  8932 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:19.779911  8790 raft_consensus.cc:3060] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:19.810467  8782 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:19.814256  8934 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:04:19.801479  8814 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:19.784075  8796 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:19.804204  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 83148.9us
W20260709 13:04:19.796247  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55396.6us
I20260709 13:04:19.864892  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.777361  8765 raft_consensus.cc:2804] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:19.836131  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:19.788298  8675 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.859018  2376 raft_consensus.cc:2468] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:04:19.866328  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.877441  2372 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
I20260709 13:04:19.898465  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:19.831837  8836 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:19.880155  8885 consensus_meta.cc:287] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.674s	user 0.013s	sys 0.000s
I20260709 13:04:19.879046  8723 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.878847  8875 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.899705  8675 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:19.913184  8842 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.899312  8743 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:19.899642  8911 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.803529  8883 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:19.932229  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.998s (SENT)
I20260709 13:04:19.913158  8765 raft_consensus.cc:697] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:19.924988  8848 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:04:19.956171  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 91387.2us
I20260709 13:04:19.923754  8885 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 17. Current term is 18.
I20260709 13:04:19.959184  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.960310  8913 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.002569  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.004062  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:19.926827  8723 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:19.962834  8848 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:19.880596  8843 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.002s
I20260709 13:04:19.930241  8875 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.017374  1642 raft_consensus.cc:2468] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:19.959681  8814 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:19.933449  8842 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:19.975519  8766 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:19.953617  8911 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.034122  8848 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:20.075551  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.037859  8782 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:20.076848  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 57604.5us
I20260709 13:04:20.064656  8932 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:04:20.071740  8946 log.cc:927] Time spent T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
I20260709 13:04:20.020918  8913 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:20.082031  8861 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.006s
I20260709 13:04:20.110772  8861 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.114602  8839 raft_consensus.cc:3055] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:20.115031  8839 raft_consensus.cc:740] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:20.114400  8790 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.079871  8956 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:20.117282  1650 raft_consensus.cc:2410] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 13:04:20.101748  1616 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.117213  8790 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.121446  8861 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.043716  8843 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.123395  1646 raft_consensus.cc:1240] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:04:20.081555  8880 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.092753  8782 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:20.099548  8949 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 13:04:20.109563  8781 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.124733  8782 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.109030  8932 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.125119  8781 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.129781  8836 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:20.130439  8836 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:20.133476  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:04:20.105962  8951 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 13:04:20.106796  8952 log.cc:927] Time spent T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:04:20.100404  8955 raft_consensus.cc:2804] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:20.137207  8843 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.156235  8956 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:20.065513  8765 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.161919  8790 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.116012  8839 consensus_queue.cc:260] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.174569  8955 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:20.201960  8945 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.202670  8954 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:04:20.202777  8945 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.184778  8781 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:20.221453  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 204532us
I20260709 13:04:20.219951  8781 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:20.225178  8959 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:04:20.228932  8781 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.230258  8945 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:20.231945  8708 consensus_meta.cc:287] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.748s	user 0.010s	sys 0.003s
I20260709 13:04:20.232357  8956 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.208123  8110 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.167016  8814 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.205020  8955 raft_consensus.cc:3060] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.192157  8790 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:20.252671  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.208s (SENT)
I20260709 13:04:20.253299  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:20.237887  8839 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:20.241652  8883 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.244805  8875 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.247318  8110 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.269904  8953 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.270457  8953 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:20.270990  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.252465  8723 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.272078  8110 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:20.274887  8570 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.000s
I20260709 13:04:20.275326  8570 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.275925  8930 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:20.269284  1616 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 33 similar messages]
I20260709 13:04:20.258438  8885 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.258924  8842 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:20.277446  1616 leader_election.cc:336] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:20.263074  8889 consensus_meta.cc:287] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.946s	user 0.009s	sys 0.005s
I20260709 13:04:20.286136  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:20.296283  8787 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.909s	user 0.014s	sys 0.002s
I20260709 13:04:20.296774  8787 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.298848  8787 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:20.301337  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:20.302433  1616 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.304093  8843 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:20.307821  8903 consensus_meta.cc:287] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.004s
W20260709 13:04:20.312228  8785 consensus_meta.cc:287] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.070s	user 0.016s	sys 0.000s
I20260709 13:04:20.268664  8911 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.312757  8785 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.313911  1716 raft_consensus.cc:2368] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for earlier term 21. Current term is 22.
I20260709 13:04:20.314684  8570 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:20.316206  8850 consensus_meta.cc:287] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 1.254s	user 0.016s	sys 0.001s
W20260709 13:04:20.325640  8967 log.cc:927] Time spent T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:04:20.320086  8913 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.291193  8889 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:20.320909  8957 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:20.356701  1616 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:20.357514  1616 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:20.358505  1616 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:20.359364  1616 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:20.360703  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 402024us
I20260709 13:04:20.290186  8885 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.359820  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:04:20.354060  8868 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.887s	user 0.010s	sys 0.010s
I20260709 13:04:20.366708  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.367650  1647 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:20.366487  8868 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:20.367239  1616 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.322566  8790 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.373697  1644 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:20.373900  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:20.370919  1616 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.371668  1642 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:20.316809  8903 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.380036  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:20.382164  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.383390  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.384075  1647 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 20: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:20.375741  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:20.375818  1616 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.384594  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.384732  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:20.385620  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:04:20.386013  1616 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.386216  2376 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.379247  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:20.386763  1616 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.355403  8850 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:20.387693  1648 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:20.391650  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.389425  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.390797  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.389312  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.392424  1642 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.391733  1644 raft_consensus.cc:2410] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:20.395006  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:20.396667  1645 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:04:20.387451  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:20.399049  1617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.398466  1616 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.370975  8913 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.337093  8782 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.405862  2376 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:20.407208  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.408620  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.409755  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.375996  8953 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:20.411691  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1796402360000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 13:04:20.415450  1539 connection.cc:256] RPC call timeout handler was delayed by -1.1643402360000437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:04:20.361761  8796 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.416781  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:20.433524  8885 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.434543  8796 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.361027  8957 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.418598  8848 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.425885  8868 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.451860  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
W20260709 13:04:20.415079  8862 consensus_meta.cc:287] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.017s	user 0.011s	sys 0.005s
I20260709 13:04:20.416540  1642 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:20.406876  8842 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:20.440925  8790 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:20.471571  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:20.401232  8850 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.459129  8889 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:20.449120  8913 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.449458  8962 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:20.463955  8969 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:04:20.466163  8675 consensus_meta.cc:287] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.529s	user 0.015s	sys 0.000s
I20260709 13:04:20.461087  8885 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.462695  8787 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:20.480051  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:20.465797  8785 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:20.474084  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 221402us
I20260709 13:04:20.475858  8814 raft_consensus.cc:2764] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:04:20.488477  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:20.465993  8862 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.468967  8842 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:20.496263  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55527.2us
W20260709 13:04:20.489598  8925 consensus_meta.cc:287] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.964s	user 0.017s	sys 0.004s
I20260709 13:04:20.483037  8968 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:20.480247  8790 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.490367  8932 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:20.516530  1651 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.497574  8787 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:20.485455  8889 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.483765  8913 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:20.506871  8792 consensus_meta.cc:287] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.809s	user 0.000s	sys 0.019s
I20260709 13:04:20.487385  8675 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.523088  8862 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.495307  8836 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.530869  8932 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:20.485714  8962 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.506342  8925 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:20.489097  8885 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.497319  8918 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:20.530297  8903 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.541527  1651 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
W20260709 13:04:20.573983  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.574076  8885 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:20.577046  8977 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:04:20.577894  8790 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.538213  8792 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.536005  8974 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:20.518963  8869 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.537117  8787 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:20.586454  8925 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:20.539491  8911 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:20.564013  8785 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.593505  8836 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.569553  8953 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.567554  8906 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:20.596467  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:20.599324  8925 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.600672  8675 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.597888  8850 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.602559  8842 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.605007  8913 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:20.610265  1690 leader_election.cc:336] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:20.614182  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 927836us
I20260709 13:04:20.627743  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:20.620388  8962 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.620864  8468 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:20.624250  8906 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.621058  8787 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.632193  8848 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.623279  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:20.597846  8974 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.619346  8984 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:20.635599  8984 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:20.618216  8911 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.636648  8984 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.638300  8911 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.638770  8842 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:20.639040  8782 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.639552  8842 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.642146  8842 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.644207  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2640) took 1030 ms. Trace:
I20260709 13:04:20.644699  1645 rpcz_store.cc:276] 0709 13:04:19.613312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.613877 (+   565us) service_pool.cc:224] Handling call
0709 13:04:19.636877 (+ 23000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.397096 (+760219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.643957 (+246861us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b5400306e54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:20.644142 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1746505472}
I20260709 13:04:20.636113  8974 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.652297  8742 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.653210  1649 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 13:04:20.624902  2344 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.652940  8742 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.616297  8785 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.654920  1643 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:20.655201  8906 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.619418  8953 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.656457  8925 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.656965  8889 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.661089  8766 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.679222  8766 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.679642  8766 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.680924  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:20.683307  8979 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:04:20.684973  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:20.685783  2376 raft_consensus.cc:2468] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:04:20.685987  8971 log.cc:927] Time spent T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:04:20.686257  8953 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.660949  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.660845  8957 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:20.661839  8302 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:20.687431  8760 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.688046  8760 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.663444  1688 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:20.690519  8760 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.624629  8790 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:04:20.691107  8760 raft_consensus.cc:2804] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:20.694015  8863 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:20.694152  8903 consensus_queue.cc:1048] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:20.695349  8863 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:20.697417  8863 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:20.691092  1645 raft_consensus.cc:2468] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
I20260709 13:04:20.698953  8766 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.699329  8863 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:20.667490  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.697714  8760 raft_consensus.cc:697] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:20.701400  8760 consensus_queue.cc:237] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:20.667834  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.704439  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:20.706019  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:20.706255  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:20.707334  1688 leader_election.cc:400] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 14 election: Vote denied by peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309) with higher term. Message: Invalid argument: T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for earlier term 14. Current term is 15.
I20260709 13:04:20.700841  1647 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:20.709045  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:20.714138  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:20.715282  1719 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.716651  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:20.719385  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:20.670152  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.676427  8742 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.678304  8785 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.741029  8863 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.746157  8957 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.750878  8730 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.742090  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.742748  8766 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:20.743499  1717 raft_consensus.cc:1240] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:20.745446  1720 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:20.758844  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2648) took 1085 ms. Trace:
I20260709 13:04:20.765439  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:20.766341  2376 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 19: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:20.771663  1723 raft_consensus.cc:2410] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 13:04:20.773554  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 64583.5us
I20260709 13:04:20.759604  1650 rpcz_store.cc:276] 0709 13:04:19.673748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.673932 (+   184us) service_pool.cc:224] Handling call
0709 13:04:20.758545 (+1084613us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b5400307854. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:20.758752 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014903552}
I20260709 13:04:20.774505  1688 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:20.775895  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:20.776578  2376 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:20.779742  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:20.780113  2378 raft_consensus.cc:1275] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:20.781140  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:20.781831  2374 raft_consensus.cc:3060] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.786007  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:20.786772  2371 raft_consensus.cc:3060] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:20.787575  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68240.4us
I20260709 13:04:20.788995  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.789242  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:20.789032  1642 raft_consensus.cc:1275] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:20.790547  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.789726  1644 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:20.793980  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.795466  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.796741  1642 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:20.797557  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:20.798556   431 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 1587)
I20260709 13:04:20.805353  1644 raft_consensus.cc:1275] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:20.799705  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.807762  1642 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:20.807145  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.799957  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.809953  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:20.817167  1643 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.813597  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.821229  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.825058  1642 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.823657  1644 raft_consensus.cc:2393] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:20.826947  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:20.831141  2372 raft_consensus.cc:3060] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:20.832602  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.833518  1644 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:20.804150  8386 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:20.835331  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.811030  8785 raft_consensus.cc:2804] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:20.839104  8932 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:20.839465  8781 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:20.840404  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34482 (request call id 1115) took 1212 ms. Trace:
I20260709 13:04:20.882977  1646 rpcz_store.cc:276] 0709 13:04:19.627645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.635139 (+  7494us) service_pool.cc:224] Handling call
0709 13:04:19.670321 (+ 35182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.124018 (+453697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.840123 (+716105us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54000915d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:20.840325 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041054592}
I20260709 13:04:20.804525  8790 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.841964  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.842478  2376 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 24.
I20260709 13:04:20.884596  1648 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:20.845629  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:20.828222  8955 consensus_meta.cc:287] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.007s
I20260709 13:04:20.885828  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:20.823316  8957 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:20.898437  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.915194  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.916051  1725 raft_consensus.cc:2468] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:20.917503  1643 raft_consensus.cc:2468] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:20.918650  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.919448  1643 raft_consensus.cc:2393] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:20.920377  1642 raft_consensus.cc:2468] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:20.855962  1719 raft_consensus.cc:2410] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:04:20.864225  2371 raft_consensus.cc:2468] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:20.876855  2378 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:20.833110  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:20.842514  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3351) took 1713 ms (client timeout 1522 ms). Trace:
I20260709 13:04:20.854454  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.888720  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:20.925577  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 92516.3us
I20260709 13:04:20.926267  1717 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:20.889284  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:20.925364  1716 rpcz_store.cc:269] 0709 13:04:19.128742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.129116 (+   374us) service_pool.cc:224] Handling call
0709 13:04:20.842243 (+1713127us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400088754. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:20.842420 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706125568}
I20260709 13:04:20.929155  2374 raft_consensus.cc:2468] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:20.933143  8973 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.933737  8973 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.937356  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:20.938629  2380 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:20.941144  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:20.877259  8842 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.959700  1645 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:20.966053  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.969158  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.972851  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:20.978813  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:20.979902  1646 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:20.979980  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:20.981601  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:20.982605  1649 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.982757  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:20.971325  8570 consensus_queue.cc:1048] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:04:20.988323  8784 consensus_queue.cc:1048] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:20.992156  2373 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:20.993278  8801 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:20.994197  8801 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.992561  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.005211  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:20.888788  8785 raft_consensus.cc:697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:21.006518  1720 raft_consensus.cc:2393] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:20.994665  8801 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:20.959712  8790 raft_consensus.cc:2804] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.975555  8880 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.993551  8965 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:20.957598  8316 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.940917  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:21.037318  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 96439.9us
I20260709 13:04:21.038054  1614 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:20.864234  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:21.040771  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 176571us
I20260709 13:04:20.954663  1643 raft_consensus.cc:1240] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:04:21.056634  1690 leader_election.cc:336] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:21.057904  1690 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:21.058708  1690 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.031143  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.007349  8911 consensus_queue.cc:1059] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:04:21.007750  8884 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:20.906203  8955 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:20.926726  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.039192  1614 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:21.069612  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.045908  8726 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:21.051576  1649 raft_consensus.cc:2468] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:21.015476  8760 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:21.088488  8760 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.012953  8973 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.015745  8913 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:21.089686  1722 raft_consensus.cc:1275] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:04:21.090358  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 163729us
I20260709 13:04:21.021632  8790 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.035112  2372 raft_consensus.cc:2468] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:21.062851  1720 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:04:21.059051  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:20.985478  8842 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.065122  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.065888  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:21.112665  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 46804us
I20260709 13:04:21.064519  8904 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.042330  8965 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.015538  8944 consensus_queue.cc:1048] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:04:21.046132  8836 raft_consensus.cc:2804] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:04:21.017541  8956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.033s	sys 0.029s
I20260709 13:04:21.071292  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:21.088930  8760 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:21.055516  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:21.090874  8787 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:21.046471  8316 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.092555  8726 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:21.089567  8913 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.205091  1646 raft_consensus.cc:2393] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 19: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
I20260709 13:04:21.066780  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:21.112879  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54188.5us
I20260709 13:04:21.043326  8880 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.046255  8869 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:04:21.114181  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:21.131227  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.095835  8811 raft_consensus.cc:2804] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:21.133085  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:21.212994  1720 raft_consensus.cc:3060] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.185698  1721 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:04:21.107326  8730 raft_consensus.cc:3055] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:21.127848  8965 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.188879  8787 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.120522  8386 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:21.193207  8468 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:21.227352  8468 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.230474  8468 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:21.113405  8792 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.112375  8790 raft_consensus.cc:3060] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:21.199515  1723 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
W20260709 13:04:21.190960  8956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.040s	sys 0.031s
I20260709 13:04:21.122496  8957 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.097368  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:21.247294  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 149956us
W20260709 13:04:21.206826  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.113677  8863 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.252916  8468 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:21.127501  8962 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.253259  8990 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:04:21.183398  8984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.057s	sys 0.126s
I20260709 13:04:21.253562  8468 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:21.253720  8989 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 13:04:21.254022  8984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.057s	sys 0.126s
I20260709 13:04:21.193723  8842 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:21.219363  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 105246us
I20260709 13:04:21.256727  1688 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.257393  1725 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 22: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:21.110446  8925 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.257737  1688 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.258240  8925 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.223953  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.129331  8904 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.125209  8932 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.108634  8785 consensus_queue.cc:237] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.260720  8932 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.263017  8932 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:21.248417  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.264555  1719 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
W20260709 13:04:21.264955  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:21.265679  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.265864  8766 raft_consensus.cc:2804] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:21.254091  8468 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.266496  8766 raft_consensus.cc:697] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:21.267477  8766 consensus_queue.cc:237] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.193969  8836 raft_consensus.cc:697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:21.268296  8955 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.269043  8836 consensus_queue.cc:237] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.270287  8880 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:21.225631  8796 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:21.270969  8796 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:21.271034  8925 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.271440  8796 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.265682  8847 raft_consensus.cc:2804] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:21.217396  8730 raft_consensus.cc:740] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:21.252663  8957 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.276638  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:21.275823  1616 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.225862  8787 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:21.277686  1723 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:21.225847  8811 raft_consensus.cc:697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:21.274948  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.280110  1720 raft_consensus.cc:2468] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:21.280700  1722 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
W20260709 13:04:21.274283  1690 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.281534  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:21.282253  2372 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:21.284734  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:21.299443  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.301533  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.302459  1722 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 24.
I20260709 13:04:21.303638  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:21.304874  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.278278  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:21.308185  1616 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:21.306702  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.309052  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:21.282343  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.321045  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.322057  1720 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:21.323864  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:21.327652  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:21.329586  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:21.273568  1688 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:21.243171  8386 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:21.339501  1647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.034s	sys 0.033s
W20260709 13:04:21.340008  1647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.034s	sys 0.034s
W20260709 13:04:21.334115  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.333318  8913 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:21.335630  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 62170.6us
I20260709 13:04:21.244284  8792 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.272442  8847 raft_consensus.cc:697] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:21.347863  1649 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:04:21.357132  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:21.349539  1645 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:21.349814  1722 raft_consensus.cc:3060] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:21.318142  8842 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.355687  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.359524  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:21.360548  1643 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:21.362596  1643 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:04:21.311385  8767 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.368006  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.368942  1719 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:21.312104  8904 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:21.345780  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.358712  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:21.318462  8880 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:21.347421  8984 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.381527  8984 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.383786  8984 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.384680  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:21.385540  1649 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 18: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:21.377825  8302 consensus_queue.cc:1059] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:04:21.377270  8842 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.388351  8984 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.389432  8984 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.387646  8302 raft_consensus.cc:3055] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:21.333765  8956 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:21.390390  8302 raft_consensus.cc:740] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:21.345559  8863 raft_consensus.cc:2804] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:21.391042  8956 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:21.391484  8863 raft_consensus.cc:697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:21.391547  8302 consensus_queue.cc:260] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.392203  8956 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.392550  8863 consensus_queue.cc:237] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.393816  8302 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.395571  8889 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.396258  8889 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.396878  8730 consensus_queue.cc:260] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.398473  8730 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.377200  8386 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.401336  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:21.264830  8726 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.334136  8925 raft_consensus.cc:2804] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:21.376911  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.404841  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.415597  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.253746  8962 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.345474  8973 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.372476  1688 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:21.425362  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53026.2us
I20260709 13:04:21.350389  8932 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.431568  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:21.432811  1650 raft_consensus.cc:2393] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:21.432953  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.377084  8813 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:21.402679  8944 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.403223  8880 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.445796  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.391055  8842 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.416729  8925 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:21.456871  8847 consensus_queue.cc:237] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.441612  8813 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.442323  8811 consensus_queue.cc:237] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.461521  8784 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:21.402908  8767 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.466584  8962 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:21.480000  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:21.463786  8468 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.439134  8904 raft_consensus.cc:2804] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:21.430151  8973 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:21.530067  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 50275.3us
I20260709 13:04:21.450626  8880 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:21.479048  8925 raft_consensus.cc:3060] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:21.537678  2372 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 16.
I20260709 13:04:21.478807  8841 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:21.538955  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.543622  8792 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.482378  8842 consensus_queue.cc:1048] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:04:21.547662  8767 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:21.548732  8889 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.548934  1722 raft_consensus.cc:2468] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:21.552028  8913 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:21.552781  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:21.448143  8932 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.516134  8813 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:21.558297  8962 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.533682  8782 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:21.601222  1648 raft_consensus.cc:2393] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:21.618913  1647 raft_consensus.cc:1275] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:21.524852  8984 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.520541  8965 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.538929  8904 raft_consensus.cc:697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:21.633754  1647 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:21.469276  8944 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.675350  1717 raft_consensus.cc:2393] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 20: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:21.617069  8962 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.597657  8841 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.612965  1614 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:21.643548  8957 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.644384  8932 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:21.707939  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 95110.8us
I20260709 13:04:21.709362  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:04:21.753623  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44402.3us
I20260709 13:04:21.754958  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:21.717860  8913 raft_consensus.cc:2804] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:21.704439  8880 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.760382  8945 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:21.760262  8848 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:21.762048  8316 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.781536  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:21.712119  8790 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:21.794391  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.008s	sys 0.035s
W20260709 13:04:21.796190  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.008s	sys 0.035s
I20260709 13:04:21.716012  8904 consensus_queue.cc:237] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.799823  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.774889  8913 raft_consensus.cc:697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:21.804908  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:21.787593  8868 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:21.807188  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:21.788214  8848 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.793102  8468 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:21.800446  8836 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.017s	sys 0.027s
I20260709 13:04:21.819882  8842 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.805578  8973 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:21.813459  8848 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:21.806421  8781 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:21.824409  8836 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.018s	sys 0.027s
I20260709 13:04:21.811321  8884 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:21.853431  8884 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.854005  8884 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.836418  8781 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.856730  8781 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.858256  8790 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.833186  8965 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:21.882332  8841 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:21.911543  8995 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:04:21.913043  8760 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.712s	user 0.013s	sys 0.000s
I20260709 13:04:21.913666  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2667) took 1055 ms. Trace:
I20260709 13:04:21.914243  1642 rpcz_store.cc:276] 0709 13:04:20.857994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.921560 (+ 63566us) service_pool.cc:224] Handling call
0709 13:04:20.922062 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.017740 (+ 95678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.017778 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.018342 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.506683 (+488341us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:21.539750 (+ 33067us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.540121 (+   371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.540211 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.913355 (+373144us) raft_consensus.cc:1685] finished
0709 13:04:21.913382 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.913585 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102891,"prepare.queue_time_us":36946,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":98403,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":9473,"thread_start_us":444335,"threads_started":1,"wal-append.queue_time_us":717788}
W20260709 13:04:21.917590  8801 consensus_meta.cc:287] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.885s	user 0.013s	sys 0.000s
I20260709 13:04:21.915501  8781 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.891265  8962 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.920125  8781 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:21.927392  1646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.011s	sys 0.010s
W20260709 13:04:21.932468  1646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.012s	sys 0.010s
I20260709 13:04:21.933063  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.934870  1723 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.908005  8884 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.936043  8863 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:21.944959  8863 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:21.913610  8760 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.951735  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:21.925362  8796 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.653s	user 0.015s	sys 0.000s
I20260709 13:04:21.950510  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.953909  1719 raft_consensus.cc:2393] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da3b57321031416cba9db75450ef302a in current term 17: Already voted for candidate 4f737473248e4c059fa74af3b8e1f77e in this term.
W20260709 13:04:21.951117  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.009s	sys 0.013s
W20260709 13:04:21.955133  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.010s	sys 0.013s
W20260709 13:04:21.956915  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 529282us
I20260709 13:04:21.958592  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.962985  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:21.965749  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 158627us
I20260709 13:04:21.968578  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:21.966849  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:21.973191  1722 raft_consensus.cc:1240] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:21.883291  8913 consensus_queue.cc:237] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.918133  8801 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:21.981029  1614 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:21.954871  8884 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.968639  8316 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:04:21.985267  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:21.993526  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 605131us
W20260709 13:04:21.994586  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:21.983093  8302 consensus_meta.cc:287] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.585s	user 0.015s	sys 0.000s
I20260709 13:04:22.006414  1721 raft_consensus.cc:2393] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:21.992961  8836 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:22.020737  8836 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.021147  8836 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:22.022176  8760 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:21.985436  8880 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:21.976764  8944 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:21.960255  8796 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:22.002857  1690 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.032809  1690 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.034509  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:22.035360  1690 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.035893  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:22.026382  8781 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:22.037914  1614 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:22.059520  1725 raft_consensus.cc:2393] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:22.027448  9007 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:22.085399  1690 leader_election.cc:336] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.086472  1690 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.073304  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.095983  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:22.097643  1690 leader_election.cc:336] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.102696  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:22.103639  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 141638us
W20260709 13:04:22.103276  1690 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.108614  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 150071us
I20260709 13:04:22.108479  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:22.084190  9007 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.063163  8801 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:22.122624  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:22.124703  9015 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:04:22.095899  9017 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:22.141211  8785 consensus_queue.cc:1048] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:04:22.108101  9019 raft_consensus.cc:2804] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:22.095045  8813 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:22.160665  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60093.8us
I20260709 13:04:22.133054  8953 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:22.135053  9007 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:22.182984  1616 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.184103  1616 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.185022  1616 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.185845  1616 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:22.186641  1616 leader_election.cc:336] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.187008  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:22.150156  8863 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:22.209075  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 74014.2us
I20260709 13:04:22.214046  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.177209  9019 raft_consensus.cc:697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:22.238909  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:22.239547  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52893.5us
I20260709 13:04:22.240161  2372 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 16.
W20260709 13:04:22.240665  1616 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.241252  1616 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:22.248373  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:22.249390  2378 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:22.220009  9013 log.cc:927] Time spent T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
I20260709 13:04:22.266813  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:22.267547  1719 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:22.269059  1616 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.269650  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:22.265893  8781 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.273168  9006 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.157s	user 0.000s	sys 0.002s
I20260709 13:04:22.273093  8781 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:22.275046  9009 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:04:22.275746  8781 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:22.276593  8869 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.273232  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:22.282900  1616 leader_election.cc:336] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.283663  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:22.289760  8973 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:22.257673  8944 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:04:22.270481  8787 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.934s	user 0.017s	sys 0.000s
I20260709 13:04:22.264752  8796 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:22.316210  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:22.286347  8781 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:22.320739  2378 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 18.
I20260709 13:04:22.309507  8842 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:22.301657  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96955.1us
I20260709 13:04:22.327986  1719 raft_consensus.cc:2468] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
W20260709 13:04:22.305313  9004 log.cc:927] Time spent T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:04:22.300302  8925 consensus_meta.cc:287] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.697s	user 0.016s	sys 0.000s
I20260709 13:04:22.302553  8906 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:22.344177  1690 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.344698  8906 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:22.344815  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.345769  8906 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.346810  8906 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.347621  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.347123  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:22.348793  9002 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.349402  9002 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.350407  1719 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:04:22.350831  1719 pending_rounds.cc:85] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 14
I20260709 13:04:22.351181  1719 pending_rounds.cc:107] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:04:22.351416  1719 raft_consensus.cc:2889] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:22.352797  8813 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:22.354964  8925 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:22.310277  8787 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.374718  8787 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:22.296736  8973 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:22.333736  9014 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:04:22.376502  1645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2674) took 1036 ms. Trace:
I20260709 13:04:22.376771  1650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2676) took 1036 ms. Trace:
I20260709 13:04:22.342948  9019 consensus_queue.cc:237] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:22.378260  9011 log.cc:927] Time spent T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.518s	user 0.003s	sys 0.000s
W20260709 13:04:22.333736  9020 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:04:22.387008  8973 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:22.377072  1645 rpcz_store.cc:276] 0709 13:04:21.340119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.378957 (+ 38838us) service_pool.cc:224] Handling call
0709 13:04:21.379351 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.379784 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:21.379814 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.380389 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.800597 (+420208us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:21.800640 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.800710 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.376085 (+575375us) raft_consensus.cc:1685] finished
0709 13:04:22.376114 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.376309 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21506,"raft.queue_time_us":1247,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":33101,"thread_start_us":419738,"threads_started":1,"wal-append.queue_time_us":708243}
I20260709 13:04:22.377278  1650 rpcz_store.cc:276] 0709 13:04:21.340965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.451804 (+110839us) service_pool.cc:224] Handling call
0709 13:04:21.500638 (+ 48834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:21.501192 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.501237 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:21.510008 (+  8771us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.938574 (+428566us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:21.938615 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.938680 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.376537 (+437857us) raft_consensus.cc:1685] finished
0709 13:04:22.376557 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.376706 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":264,"raft.queue_time_us":1376,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":8070,"thread_start_us":381809,"threads_started":1,"wal-append.queue_time_us":687898}
I20260709 13:04:22.465539  2378 raft_consensus.cc:1275] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:22.467350  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:22.465368  2374 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:22.462077  8811 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:22.474100  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:22.433055  8787 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:22.459193  8962 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.450915  8955 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:22.487726  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:22.469794  8730 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:22.488768  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 176101us
W20260709 13:04:22.491935  1616 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.473618  8811 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.477445  8925 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:22.492570  8962 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.537865  9035 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:22.538625  9035 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.539367  8869 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:22.539990  8869 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.541584  9035 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:22.542513  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 250377us
I20260709 13:04:22.549706  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.549283  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.560878  1643 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:22.561416  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.528398  8925 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:22.562052  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.562762  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.562059  1645 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.563265  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.563783  1649 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.564711  1719 raft_consensus.cc:2393] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 23: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:22.565833  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.561342  1644 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:22.484915  8787 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:22.523388  8944 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:22.595216  1648 raft_consensus.cc:1275] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:04:22.503783  8730 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:22.492857  8955 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.563609  8316 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:22.562476  8925 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:22.579278  1645 raft_consensus.cc:2468] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:22.627209  1721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3371) took 1410 ms. Trace:
I20260709 13:04:22.582069  1644 raft_consensus.cc:2468] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:22.584898  1725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3385) took 1152 ms. Trace:
I20260709 13:04:22.585065  1723 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:22.521251  8868 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:22.590953  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3361) took 1537 ms. Trace:
I20260709 13:04:22.636884  1717 rpcz_store.cc:276] 0709 13:04:21.053368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.053646 (+   278us) service_pool.cc:224] Handling call
0709 13:04:22.590624 (+1536978us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400222c54. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:22.590864 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389872512}
I20260709 13:04:22.628796  1725 rpcz_store.cc:276] 0709 13:04:21.432546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.432694 (+   148us) service_pool.cc:224] Handling call
0709 13:04:22.584558 (+1151864us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54001716d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:22.584805 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438257408}
W20260709 13:04:22.598738  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.558439  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.639539  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.570461  1642 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.549773  1614 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:22.620949  2374 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:22.644450  1649 raft_consensus.cc:2468] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:22.646168  1649 raft_consensus.cc:1275] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:04:22.626822  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 210788us
I20260709 13:04:22.649312  8811 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:22.628019  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.627702  1721 rpcz_store.cc:276] 0709 13:04:21.217010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.220176 (+  3166us) service_pool.cc:224] Handling call
0709 13:04:22.626944 (+1406768us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54003ac354. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:22.627145 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716902272}
I20260709 13:04:22.650331  8730 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:22.652014  1642 raft_consensus.cc:1275] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:22.652640  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:22.653494  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.655256  8962 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:22.656463  1690 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.656201  8868 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.656934  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:22.658159  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.658304  1719 raft_consensus.cc:1240] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.658893  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.659278  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.659857  1722 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:22.660312  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:22.661180  1690 leader_election.cc:336] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.662364  1644 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:22.663641  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:22.666599  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 116818us
I20260709 13:04:22.668941  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.667742  1614 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:22.626343  8316 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:22.672125  1642 raft_consensus.cc:2393] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 22: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:22.672668  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:22.629439  9002 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:22.675850  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.680495  1645 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 21.
I20260709 13:04:22.677768  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.681394  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.658790  8868 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:22.682384  1648 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:22.683382  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:22.682556  1690 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:22.684294  1725 raft_consensus.cc:2368] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
I20260709 13:04:22.682235  1642 raft_consensus.cc:2368] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
W20260709 13:04:22.672361  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 206184us
I20260709 13:04:22.684379  1690 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:22.616611  8302 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:04:22.666848  9028 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:22.687050  8836 consensus_meta.cc:287] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.004s
W20260709 13:04:22.688171  8848 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.820s	user 0.022s	sys 0.008s
I20260709 13:04:22.669796  9034 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:22.612821  8944 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:22.660197  8945 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
W20260709 13:04:22.841763  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 159213us
W20260709 13:04:22.669272  8847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.004s	sys 0.027s
I20260709 13:04:22.687311  8962 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.877792  9005 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.472s	user 0.002s	sys 0.001s
I20260709 13:04:22.879534  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:22.840909  8913 raft_consensus.cc:2804] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:22.850920  9022 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:22.909332  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.014218  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 175474us
I20260709 13:04:22.789222  1649 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:23.005412  1651 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 17.
W20260709 13:04:22.880962  8847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.004s	sys 0.027s
I20260709 13:04:22.871851  8945 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.888863  9002 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.055474  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:22.885728  9034 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:23.067986  1724 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:22.866546  8836 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:23.030778  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.857713  9019 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.158s
I20260709 13:04:23.098136  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:23.049105  9027 log.cc:927] Time spent T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.542s	user 0.002s	sys 0.000s
I20260709 13:04:22.870337  8848 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:23.037192  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:23.136839  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 99706.5us
I20260709 13:04:23.042375  8730 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.101827  1645 raft_consensus.cc:2393] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 19: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:23.052786  8913 raft_consensus.cc:697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:23.125350  8955 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.041122  8841 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.133249  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.321s (SENT)
I20260709 13:04:23.157814  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:23.006247  8962 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.102973  8925 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:23.211241  1647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.001s	sys 0.027s
W20260709 13:04:23.291152  1647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.002s	sys 0.028s
W20260709 13:04:23.291745  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 612649us
I20260709 13:04:23.212057  1643 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 23: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:23.164987  8785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.007s	sys 0.011s
I20260709 13:04:23.066430  9002 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.241930  9034 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:23.220325  9007 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.027s	user 0.013s	sys 0.004s
I20260709 13:04:23.331959  9007 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.334201  1718 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:23.244966  9037 log.cc:927] Time spent T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:04:23.226615  9043 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.338230  9043 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.252429  9049 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:23.159408  8730 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.306883  9023 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.308905  9022 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.214404  8841 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:23.314543  9038 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:04:23.315421  8925 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.315716  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 299505us
I20260709 13:04:23.317273  8944 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.321229  8962 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.323812  8787 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:23.232602  9005 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:23.377430  9023 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.335958  9007 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:23.295848  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:23.316192  8848 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.328171  9002 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.382570  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49750 (request call id 2748) took 2058 ms. Trace:
I20260709 13:04:23.386133  8925 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.359584  8836 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.397295  9049 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.399874  8730 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:23.404793  9022 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.413707  8841 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.244153  8913 consensus_queue.cc:237] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.439496  8869 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.383265  1646 rpcz_store.cc:276] 0709 13:04:21.324101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.325045 (+   944us) service_pool.cc:224] Handling call
0709 13:04:21.351046 (+ 26001us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:21.351507 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.351538 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:21.360116 (+  8578us) log.cc:844] Serialized 52 byte log entry
0709 13:04:21.934469 (+574353us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:21.952141 (+ 17672us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.952411 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.952487 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.024517 (+ 72030us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b18002616a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:04:22.405169 (+380652us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b18002616a0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:04:22.566386 (+161217us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:04:23.381580 (+815194us) raft_consensus.cc:1685] finished
0709 13:04:23.381637 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.382199 (+   562us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4726,"prepare.queue_time_us":17405,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":7196,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":44213,"spinlock_wait_cycles":105602560,"thread_start_us":549108,"threads_started":1,"wal-append.queue_time_us":570565}
I20260709 13:04:23.446539  9023 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.448457  9049 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.451996  9043 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:23.446615  9032 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.779s	user 0.014s	sys 0.014s
I20260709 13:04:23.464301  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55266 (request call id 3394) took 1224 ms. Trace:
I20260709 13:04:23.475371  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:23.469520  9022 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:23.477851  9031 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 1.077s	user 0.013s	sys 0.030s
I20260709 13:04:23.467769  1718 rpcz_store.cc:276] 0709 13:04:22.238008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.290995 (+ 52987us) service_pool.cc:224] Handling call
0709 13:04:23.462222 (+1171227us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400224554. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:23.462424 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2312373632}
I20260709 13:04:23.492595  9041 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.493858  1723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60218 (request call id 1054) took 1041 ms. Trace:
I20260709 13:04:23.493599  9041 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.494280  1723 rpcz_store.cc:276] 0709 13:04:22.452162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.452315 (+   153us) service_pool.cc:224] Handling call
0709 13:04:22.584786 (+132471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.630123 (+ 45337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.493279 (+863156us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001716d0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6d8ad 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:23.493592 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303676800}
I20260709 13:04:23.496773  1724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55262 (request call id 2821) took 1562 ms. Trace:
I20260709 13:04:23.497265  1724 rpcz_store.cc:276] 0709 13:04:21.934613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.942580 (+  7967us) service_pool.cc:224] Handling call
0709 13:04:21.942893 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.068888 (+1125995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.496150 (+427262us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400233054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:23.496543 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586183424}
I20260709 13:04:23.498513  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3389) took 1310 ms. Trace:
I20260709 13:04:23.487954  2377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41546 (request call id 842) took 1022 ms. Trace:
I20260709 13:04:23.498984  1716 rpcz_store.cc:276] 0709 13:04:22.188168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.188320 (+   152us) service_pool.cc:224] Handling call
0709 13:04:22.198597 (+ 10277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.203065 (+  4468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.203106 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.203754 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.478430 (+274676us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:22.478495 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.478602 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.445614 (+967012us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:04:23.497980 (+ 52366us) raft_consensus.cc:1685] finished
0709 13:04:23.498002 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.498341 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6336,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3663,"spinlock_wait_cycles":4986112,"thread_start_us":253700,"threads_started":1,"wal-append.queue_time_us":317713}
I20260709 13:04:23.533071  9048 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:23.535737  1720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55266 (request call id 3388) took 1356 ms. Trace:
I20260709 13:04:23.504873  2377 rpcz_store.cc:276] 0709 13:04:22.465634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.469103 (+  3469us) service_pool.cc:224] Handling call
0709 13:04:22.478889 (+  9786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.651080 (+172191us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:23.146694 (+495614us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.317151 (+170457us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:23.317307 (+   156us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:23.317497 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.487375 (+169878us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54005c30d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:23.487452 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.487731 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3779,"prepare.queue_time_us":587589,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":926,"raft.queue_time_us":62531,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":88591,"spinlock_wait_cycles":395244416,"thread_start_us":658753,"threads_started":2,"wal-append.queue_time_us":111002}
I20260709 13:04:23.536216  1720 rpcz_store.cc:276] 0709 13:04:22.179915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.180125 (+   210us) service_pool.cc:224] Handling call
0709 13:04:22.196971 (+ 16846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.197430 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.197465 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.198041 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.327286 (+129245us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:22.327352 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.328614 (+  1262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.476870 (+1148256us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:04:23.535143 (+ 58273us) raft_consensus.cc:1685] finished
0709 13:04:23.535169 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.535543 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":677,"raft.queue_time_us":865,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":4616,"spinlock_wait_cycles":19168000,"thread_start_us":107963,"threads_started":1,"wal-append.queue_time_us":191329}
W20260709 13:04:23.541299  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 407893us
I20260709 13:04:23.480062  8925 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:23.555944  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49766 (request call id 2693) took 1002 ms. Trace:
I20260709 13:04:23.556438  1643 rpcz_store.cc:276] 0709 13:04:22.553163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.561832 (+  8669us) service_pool.cc:224] Handling call
0709 13:04:23.555566 (+993734us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54002e2054. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d722fa 00007fc838d33b17 00007fc8322ae6d4 00007fc8322ae662 00007fc8322ae591 00007fc8322ae50c 00007fc8322ad7c2 00007fc8320a948c 00007fc8320a88c6 00007fc8320a826f
0709 13:04:23.555764 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492571392}
W20260709 13:04:23.558624  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 263028us
I20260709 13:04:23.559706  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:23.562263  9050 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:23.565171  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.585259  8302 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:23.591010  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:23.591665  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:23.591820  1648 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:23.592589  1642 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 18.
W20260709 13:04:23.594027  1691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.586974  8302 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:23.614151  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54662.9us
I20260709 13:04:23.615315  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:23.615748  8302 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:23.616408  8836 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:23.617257  8842 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:23.621707  1643 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:23.637526  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.640410  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:23.653028  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:23.655057  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.660157  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:23.662111  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:23.505198  9041 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:23.665153  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:23.669886  1646 raft_consensus.cc:2393] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 17: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:23.676357  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 61255.8us
I20260709 13:04:23.682103  1648 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:23.682897  9019 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.683530  9019 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.684927  9007 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.685680  9007 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.687656  9019 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.687801  9007 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:23.688304  8302 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.689091  8302 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.689656  9022 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:23.690374  9022 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.691336  8302 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:23.691859  8302 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.692428  8302 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.693079  8570 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:23.693836  8570 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.694666  8302 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:23.696018  8570 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:23.697407  8570 consensus_queue.cc:1048] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:23.700982  9022 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:23.754758  1690 leader_election.cc:336] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:23.758042  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 80145.5us
W20260709 13:04:23.792533  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.792666  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:23.793771  1690 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.794457  1647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2688) took 1243 ms. Trace:
W20260709 13:04:23.794557  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.795394  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.795044  1647 rpcz_store.cc:276] 0709 13:04:22.551131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.551389 (+   258us) service_pool.cc:224] Handling call
0709 13:04:22.573307 (+ 21918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.292447 (+719140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.292554 (+   107us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.294120 (+  1566us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.374092 (+ 79972us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:23.374151 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.407793 (+ 33642us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.794229 (+386436us) raft_consensus.cc:1685] finished
0709 13:04:23.794248 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.794400 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1047,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":797549,"threads_started":2,"wal-append.queue_time_us":465032}
W20260709 13:04:23.796659  1690 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:23.796950  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:23.797310  9058 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:23.798286  1690 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:23.798651  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:23.799209  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.799377  1716 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 24.
I20260709 13:04:23.765911  8925 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.800650  1690 leader_election.cc:336] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:23.801515  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:23.802198  1690 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:23.798354  9058 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:23.802907  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.800969  8925 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:23.800819  8973 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.413s	user 0.012s	sys 0.014s
I20260709 13:04:23.774945  9043 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.799306  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:23.776274  8962 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.813774  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.510s (SENT)
I20260709 13:04:23.816679  9041 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:23.818990  9050 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:23.819455  8906 consensus_meta.cc:287] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.472s	user 0.017s	sys 0.010s
I20260709 13:04:23.819796  2374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 1041) took 1072 ms. Trace:
I20260709 13:04:23.821727  8925 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:23.824391  1465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 210) took 53623 ms (client timeout 15000 ms). Trace:
I20260709 13:04:23.773571  8836 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:23.826913  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:23.808768  8973 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.838111  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:23.836097  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:23.809422  8787 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.844213  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:23.838861  9002 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.858485  1718 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:23.865947  8787 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.858129  8570 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.834985  2374 rpcz_store.cc:276] 0709 13:04:22.678294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.678426 (+   132us) service_pool.cc:224] Handling call
0709 13:04:22.678790 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.679049 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:23.185678 (+506629us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.449095 (+263417us) spinlock_profiling.cc:243] Waited 896 us on lock 0x7b180021a240. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:04:23.462244 (+ 13149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:23.471975 (+  9731us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:23.508020 (+ 36045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.526095 (+ 18075us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.750515 (+224420us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31364,"prepare.queue_time_us":572838,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":71462,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":269,"spinlock_wait_cycles":2060544,"thread_start_us":531649,"threads_started":2}
W20260709 13:04:23.868995  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 169809us
I20260709 13:04:23.870275  8836 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.873209  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:23.874234  1649 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 21: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:23.878515  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71855.1us
W20260709 13:04:23.879522  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.857839  9041 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.886682  8784 consensus_queue.cc:1059] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:23.888670  8784 raft_consensus.cc:3055] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:23.889112  8784 raft_consensus.cc:740] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:23.889153  1651 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:04:23.837246  1465 rpcz_store.cc:269] 0709 13:03:30.200805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.436951 (+11236146us) service_pool.cc:224] Handling call
0709 13:04:23.824047 (+42387096us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:15.640791 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:15.641532 (+   741us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:15.641589 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:04:15.641612 (+    23us) tablet.cc:662] Decoding operations
0709 13:04:15.643677 (+  2065us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:15.643766 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:15.643799 (+    33us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:04:15.736175 (+ 92376us) tablet.cc:701] Row locks acquired
0709 13:04:15.736223 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:04:15.736592 (+   369us) write_op.cc:270] Start()
0709 13:04:15.736770 (+   178us) write_op.cc:276] Timestamp: P: 1783602255736508 usec, L: 0
0709 13:04:15.737018 (+   248us) op_driver.cc:348] REPLICATION: starting
0709 13:04:15.740998 (+  3980us) log.cc:844] Serialized 8254 byte log entry
0709 13:04:16.144921 (+403923us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d69e28 00007fc8377a4581 00007fc8377a3d58 00007fc8377a89f1 00007fc8377a89a9 00007fc8377a8939 00007fc8377a8901 00007fc8377a7b2d 000000000038f354 000000000038f199
0709 13:04:16.721544 (+576623us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b4400022b4c. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc830b09db1 00007fc830ba7c85 00007fc830ba86a2 00007fc83391a1db 00007fc833915aaf 00007fc83390986e 00007fc833908bb6 00007fc8339087d8 00007fc833912221 00007fc8339121d9 00007fc833912169
0709 13:04:17.253143 (+531599us) op_driver.cc:464] REPLICATION: finished
0709 13:04:17.569287 (+316144us) write_op.cc:301] APPLY: starting
0709 13:04:17.569428 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:17.921911 (+352483us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:17.921970 (+    59us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:19.378691 (+1456721us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:19.379212 (+   521us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:04:19.379265 (+    53us) write_op.cc:312] APPLY: finished
0709 13:04:19.511780 (+132515us) log.cc:844] Serialized 188 byte log entry
0709 13:04:19.710016 (+198236us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:19.737943 (+ 27927us) write_op.cc:454] Released schema lock
0709 13:04:19.767731 (+ 29788us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:19.815313 (+ 47582us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:19.815352 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":509095,"reported_tablets":52,"spinlock_wait_cycles":91008,"child_traces":[["op",{"apply.queue_time_us":315663,"apply.run_cpu_time_us":86336,"apply.run_wall_time_us":2245969,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92033,"num_ops":21,"prepare.queue_time_us":114038,"prepare.run_cpu_time_us":43134,"prepare.run_wall_time_us":504456,"raft.queue_time_us":393660,"raft.run_cpu_time_us":46840,"raft.run_wall_time_us":324486,"replication_time_us":1515871,"spinlock_wait_cycles":9603328,"thread_start_us":1073337,"threads_started":5,"wal-append.queue_time_us":492176,"wal-append.run_cpu_time_us":177990,"wal-append.run_wall_time_us":3625950}]]}
W20260709 13:04:23.897431  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53274.9us
I20260709 13:04:23.899181  9061 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:23.900122  9061 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:23.902012  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57035.3us
I20260709 13:04:23.900535  9061 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.904013  9061 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:23.901911  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:23.918123  1647 raft_consensus.cc:1240] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:23.855947  9043 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.921186  8848 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.915858  9002 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.922083  8848 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.924026  9002 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:23.925594  8973 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:23.890368  8784 consensus_queue.cc:260] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.927620  8784 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:23.928781  1466 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.905143  8962 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:23.931236  1466 catalog_manager.cc:5697] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:04:23.935784  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 117) took 103269 ms (client timeout 15000 ms). Trace:
I20260709 13:04:23.936077  1643 raft_consensus.cc:1275] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:23.937391  8570 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:23.938154  8570 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:23.939215  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:23.939901  1644 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:23.940645  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:23.941567  1721 raft_consensus.cc:3060] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:23.941991  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:23.942309  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.942683  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.942310  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.943657  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.944947  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:23.945653  1688 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:23.946707  8847 consensus_queue.cc:1048] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:23.941591  1724 raft_consensus.cc:1275] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:23.942864  1723 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 23: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:23.947880  9029 raft_consensus.cc:2804] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:23.955338  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:23.962212  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43859.4us
W20260709 13:04:23.936302  1468 rpcz_store.cc:269] 0709 13:02:40.666323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.666649 (+   326us) service_pool.cc:224] Handling call
0709 13:04:23.935562 (+103268913us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":7202,"mutex_wait_us":103237,"reported_tablets":24}
I20260709 13:04:23.964558  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:24.027626  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.313992  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 350939us
W20260709 13:04:24.072023  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 126449us
W20260709 13:04:24.037173  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 63069.8us
I20260709 13:04:24.398576  1721 raft_consensus.cc:2468] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 23.
I20260709 13:04:24.018582  9029 raft_consensus.cc:697] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:23.932241  8962 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:24.056677  1466 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 15 to 19, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:24.416826  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:24.337255  8945 consensus_meta.cc:287] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 1.072s	user 0.018s	sys 0.006s
W20260709 13:04:24.040009  8316 consensus_meta.cc:287] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.203s	user 0.018s	sys 0.005s
I20260709 13:04:24.391810  8570 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:24.019491  9043 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:23.964845  9058 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:23.965018  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 121) took 15373 ms (client timeout 15000 ms). Trace:
I20260709 13:04:24.362339  9048 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:24.446009  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:24.041275  9041 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:24.074951  8848 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:24.393968  8836 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:24.251827  8973 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:23.996294  8787 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:24.429509  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:24.443759  2374 raft_consensus.cc:1275] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:04:24.279863  9034 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:24.460985  8973 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.462021  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:24.416325  8842 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.463807  8842 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.464784  8847 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.465466  8847 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.467020  8842 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:24.470306  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:24.479511  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 50707us
W20260709 13:04:24.482095  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51046.6us
W20260709 13:04:24.483538  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 53358.6us
I20260709 13:04:24.445365  8869 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.478462  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:24.492463  1650 raft_consensus.cc:2468] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:24.470263  8973 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:24.444813  1468 rpcz_store.cc:269] 0709 13:04:08.591337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.964686 (+15373349us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:23.964922 (+   236us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:24.504627  2377 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 24.
W20260709 13:04:24.501288  8784 consensus_meta.cc:287] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.573s	user 0.014s	sys 0.000s
I20260709 13:04:24.446949  8316 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:24.455927  8945 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.513306  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:24.524499  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:24.538441  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 389029us
I20260709 13:04:24.544061  1718 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:04:24.545032  1718 pending_rounds.cc:85] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 11
I20260709 13:04:24.545290  1718 pending_rounds.cc:107] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:04:24.545698  1718 raft_consensus.cc:2889] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:24.545887  9061 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.620s	user 0.012s	sys 0.003s
I20260709 13:04:24.546339  9061 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:24.549640  8847 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:24.556460  9029 consensus_queue.cc:237] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.485728  9048 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:24.561451  1466 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.564162  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:24.512009  9034 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.565596  9048 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:24.522117  8869 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.569231  8869 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:24.571465  8316 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:24.583221  9034 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:24.586169  1466 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:24.593912  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:24.590420  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 74478.3us
I20260709 13:04:24.591810  1651 raft_consensus.cc:2393] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 25: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:24.596908  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:24.599857  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:24.598022  2377 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:24.606117  2380 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:04:24.603803  2374 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
I20260709 13:04:24.614991  1722 raft_consensus.cc:1275] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:24.618124  1614 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:24.643844  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 136990us
I20260709 13:04:24.621234  8570 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:24.616998  8787 consensus_queue.cc:1059] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:24.662441  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:24.663460  1718 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:24.664144  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:24.651796  9050 raft_consensus.cc:2804] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:24.668599  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:24.664915  1722 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:24.623006  9034 consensus_queue.cc:1048] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:04:24.657946  1466 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 16, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:24.632203  9019 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:04:24.631930  9043 consensus_queue.cc:1059] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:04:24.674664  9061 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:24.689950  1725 raft_consensus.cc:1275] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:24.667071  9048 consensus_queue.cc:1059] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:04:24.681676  9050 raft_consensus.cc:697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:24.691345  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:24.692116  1466 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.693813  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:24.709404  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:24.712195  2372 raft_consensus.cc:1275] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:24.712096  8781 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:24.716532  8781 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.717447  8781 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:24.717765  9050 consensus_queue.cc:237] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.719158  8962 raft_consensus.cc:3055] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:24.719655  8962 raft_consensus.cc:740] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:24.694561  8869 consensus_queue.cc:1048] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:24.692246  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:24.713493  2377 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 19.
I20260709 13:04:24.731045  8869 consensus_queue.cc:1048] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:24.732064  8869 consensus_queue.cc:1243] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:04:24.713977  9076 log.cc:927] Time spent T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:04:24.694800  8570 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:24.736310  1466 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 11 to 16, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:24.742377  8570 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:24.735972  9072 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.092s	user 0.007s	sys 0.010s
I20260709 13:04:24.742004  2374 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:24.742955  1466 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 17, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.749516  1688 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
W20260709 13:04:24.766743  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 356233us
I20260709 13:04:24.728116  8945 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:24.795648  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 350853us
W20260709 13:04:24.783708  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:24.793958  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:24.799656  1722 pending_rounds.cc:85] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Aborting all ops after (but not including) 11
I20260709 13:04:24.806212  1722 pending_rounds.cc:107] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:04:24.806568  1722 raft_consensus.cc:2889] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:24.801326  1466 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.810941  8869 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:04:24.813090  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:24.829787  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 95890.8us
W20260709 13:04:24.867197  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 194221us
W20260709 13:04:24.868818  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 207267us
I20260709 13:04:24.796239  9083 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:24.834453  1466 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:24.871662  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52479.1us
W20260709 13:04:24.868656  1646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.006s	sys 0.006s
W20260709 13:04:24.922470  1646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.007s	sys 0.006s
I20260709 13:04:24.873430  8847 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:24.918133  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:24.932497  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:24.959707  1688 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:24.948818  1646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 2697) took 1135 ms. Trace:
W20260709 13:04:24.956532  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 84796.7us
W20260709 13:04:24.958751  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 76769.2us
I20260709 13:04:24.908951  8570 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:24.964238  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.960640  1646 rpcz_store.cc:276] 0709 13:04:23.813134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.830072 (+ 16938us) service_pool.cc:224] Handling call
0709 13:04:23.859519 (+ 29447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.884792 (+ 25273us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:24.185365 (+300573us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.928025 (+742660us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.947903 (+ 19878us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:24.948272 (+   369us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:24.948403 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.948511 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.948724 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28495,"prepare.queue_time_us":632223,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":116,"raft.queue_time_us":19698,"thread_start_us":949523,"threads_started":2}
I20260709 13:04:24.963459  1466 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.970496  1718 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 25.
I20260709 13:04:24.805994  8962 consensus_queue.cc:260] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:24.969879  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:24.980491  8787 raft_consensus.cc:3055] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:04:24.987380  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 1acf73e529394004842219637cff7cfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:24.988201  8913 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:24.988354  9061 raft_consensus.cc:3055] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:24.941458  9041 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.991328  9077 log.cc:927] Time spent T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.330s	user 0.002s	sys 0.000s
I20260709 13:04:24.999518  8570 raft_consensus.cc:2804] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:25.001422  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 82039.2us
I20260709 13:04:25.002180  8869 consensus_queue.cc:1048] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:25.002584  8842 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.040953  8842 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.008426  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:25.009104  9058 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:25.008960  1466 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:24.991961  8847 raft_consensus.cc:697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:25.048946  1466 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 18, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:25.052140  8847 consensus_queue.cc:237] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.052459  1466 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 15, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:25.053187  8781 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:25.045907  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 86238.3us
I20260709 13:04:24.983361  8953 consensus_queue.cc:1048] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:04:25.048910  8842 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:25.050030  9048 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:25.066720  9048 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.067535  9048 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:25.050312  9041 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:25.054559  1466 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 11 to 14, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.998656  8787 raft_consensus.cc:740] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:25.047521  1614 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:25.016186  8913 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:25.021426  9061 raft_consensus.cc:740] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:25.116582  8787 consensus_queue.cc:260] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.133774  1466 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.156697  8787 raft_consensus.cc:3060] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:25.158752  8945 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:25.159587  8945 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.161073  1466 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:25.162710  8945 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:25.163352  1466 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.169766  1466 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.175578  9061 consensus_queue.cc:260] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:25.151470  8913 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:25.179883  9061 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.188064  1466 catalog_manager.cc:5697] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 15 to 18, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.107448  8570 raft_consensus.cc:697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:25.199079  8570 consensus_queue.cc:237] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.205065  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:25.198684  1466 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 10 to 11, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.208052  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 93893.5us
I20260709 13:04:25.207646  8787 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:25.212678  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 291812us
W20260709 13:04:25.214113  1616 leader_election.cc:336] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:25.214794  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:25.216982  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.671s (SENT)
I20260709 13:04:25.217401  1466 catalog_manager.cc:5697] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 16, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.230232  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 104699us
I20260709 13:04:25.230046  1466 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.238245  1466 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.245827  1466 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 18, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:25.250186  2344 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:25.251498  1466 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.256073  1466 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 15, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 15 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:25.262082  1466 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.266765  1466 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 17, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:25.272764  1466 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:25.279925  1466 catalog_manager.cc:5697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 9 to 13, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.284087  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.285187  9096 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 13:04:25.284504  1466 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 13 to 16, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:25.293615  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 246097us
W20260709 13:04:25.293851  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79838.6us
I20260709 13:04:25.294534  8842 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:04:25.294863  1616 leader_election.cc:336] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:25.295248  8842 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:25.295401  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:25.296415  8842 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:25.297119  1616 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 106 similar messages]
W20260709 13:04:25.300716  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 68871.6us
W20260709 13:04:25.249359  9092 log.cc:927] Time spent T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.270s	user 0.005s	sys 0.000s
I20260709 13:04:25.312726  9034 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:25.320633  8848 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:25.315512  9022 consensus_queue.cc:1048] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 13:04:25.330600  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 287502us
W20260709 13:04:25.359835  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.343641  2374 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.001s
I20260709 13:04:25.380126  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:25.499881  2374 raft_consensus.cc:2468] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 17.
I20260709 13:04:25.386329  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:25.330160  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.404920  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:25.328651  1688 connection.cc:256] RPC call timeout handler was delayed by -1.297897493999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
I20260709 13:04:25.530920  1721 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:25.401512  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 184563us
W20260709 13:04:25.460844  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.533975  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.534514  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 141641us
W20260709 13:04:25.538426  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.679s (SENT)
I20260709 13:04:25.548789  1725 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:25.435600  8913 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:25.552362  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:25.374245  8962 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:25.421180  9088 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:25.393458  9041 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:25.538539  8316 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:25.535207  9019 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.595250  1616 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:25.603494  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:25.610291  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79241.4us
I20260709 13:04:25.559684  9007 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.592691  8787 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:25.633440  1725 raft_consensus.cc:1275] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:04:25.632421  9029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.072s	user 0.075s	sys 0.104s
W20260709 13:04:25.636274  9029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.075s	sys 0.104s
I20260709 13:04:25.623528  8781 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:25.623584  9041 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:25.633008  8913 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:25.677124  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 350169us
I20260709 13:04:25.681516  1724 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:25.663082  8869 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:25.622527  8316 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.702863  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:25.705752  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:25.706948  8913 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:25.707631  8913 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.709038  9097 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:25.709501  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:25.709689  9097 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:25.710493  8913 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:25.711366  9043 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:25.712466  9099 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:25.712514  9043 raft_consensus.cc:2804] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:25.713481  9043 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:25.713404  9099 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:25.714180  9043 raft_consensus.cc:3060] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.713645  9050 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:25.716862  9050 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:25.716646  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 123431us
W20260709 13:04:25.718204  1616 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:25.632719  9019 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:25.692267  9103 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:25.727288  9103 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:25.729148  9098 log.cc:927] Time spent T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:04:25.730618  9103 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:25.747483  1721 raft_consensus.cc:2468] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:25.717481  9050 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:25.654060  9007 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.721264  8781 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.750180  9072 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:04:25.764444  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 46288.3us
I20260709 13:04:25.776415  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:25.824028  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 184903us
W20260709 13:04:25.829138  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 119729us
W20260709 13:04:25.850195  9101 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 13:04:25.868703  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.871953  8913 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:25.875893  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:25.877285  1646 raft_consensus.cc:1240] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:25.879143  8316 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:25.880331  8913 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:25.881965  1651 raft_consensus.cc:1240] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 LEADER]: Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:25.888022  8913 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:25.889043  1616 leader_election.cc:336] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:25.890201  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 48049.5us
W20260709 13:04:25.883993  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 351091us
I20260709 13:04:25.893133  9097 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:25.893831  9048 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.826s	user 0.016s	sys 0.006s
W20260709 13:04:25.902535  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.916474  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:25.918955  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:25.920485  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:25.937530  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:25.945526  1614 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:25.954571  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:25.911371  8781 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:25.960525  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.910482  8316 consensus_queue.cc:1048] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:04:25.917491  9048 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:25.959743  1690 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:25.971830  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:25.934020  9099 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:25.946755  8913 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:25.929905  9022 consensus_queue.cc:1048] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 13:04:25.980602  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:25.983493  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:25.984321  2380 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:25.985713  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:25.987655  1651 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:25.980602  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 274969us
I20260709 13:04:25.968276  9081 raft_consensus.cc:2804] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:26.001582  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:26.002707  2378 raft_consensus.cc:1240] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Rejecting Update request from peer cd6d377efe514bc58ae49c3f4a107232 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:04:25.959579  9058 consensus_queue.cc:1059] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 13:04:26.005321  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.005954  9058 raft_consensus.cc:3055] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:26.006592  9058 raft_consensus.cc:740] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
W20260709 13:04:26.010061  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 121046us
W20260709 13:04:26.013319  1616 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:25.993336  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.978237  9007 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:26.021622  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.025328  2378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41544 (request call id 1051) took 1021 ms. Trace:
I20260709 13:04:26.027170  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:26.025868  2378 rpcz_store.cc:276] 0709 13:04:25.003584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.016581 (+ 12997us) service_pool.cc:224] Handling call
0709 13:04:25.056011 (+ 39430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.003280 (+947269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.021323 (+ 18043us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54000a87d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d6ee01 00007fc833d6d83e 00007fc838d32419 00007fc8322a9bd4 00007fc8322a9b62 00007fc8322a9a91 00007fc8322a9a0c 00007fc8322a8cc2 00007fc8320a948c 00007fc8320a88c6
0709 13:04:26.025225 (+  3902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153721600}
I20260709 13:04:26.042596  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:26.050154  1690 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.051074  1690 leader_election.cc:336] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.051921  1690 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.063199  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.066465  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:26.071923  8781 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:26.030195  9019 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:26.073762  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60501.8us
W20260709 13:04:26.075589  1616 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.075845  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:26.076972  1720 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
W20260709 13:04:26.078644  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 1acf73e529394004842219637cff7cfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:26.079057  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51703.4us
I20260709 13:04:26.076207  1616 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:26.102974  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:26.107985  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 98190.7us
I20260709 13:04:26.109093  1614 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:26.109529  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:26.097651  8962 raft_consensus.cc:2804] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:26.121506  8962 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:26.127943  8962 raft_consensus.cc:3060] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:26.141824  8962 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:26.029533  9058 consensus_queue.cc:260] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:26.168812  8962 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:26.169399  9058 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:26.172623  9061 consensus_queue.cc:1175] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd6d377efe514bc58ae49c3f4a107232" responder_term: 25 status { last_received { term: 25 index: 17 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer da3b57321031416cba9db75450ef302a for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:26.144840  9019 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:26.148145  2380 raft_consensus.cc:2468] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 20.
I20260709 13:04:26.152638  8781 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:26.175195  8781 raft_consensus.cc:697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:26.176584  8781 consensus_queue.cc:237] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:26.158571  9043 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:26.191251  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:26.206621  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.204573  1651 raft_consensus.cc:2468] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 19.
W20260709 13:04:26.208334  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 231369us
W20260709 13:04:26.189850  9041 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.006s
W20260709 13:04:26.200248  9108 log.cc:927] Time spent T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.291s	user 0.001s	sys 0.001s
I20260709 13:04:26.208639  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:26.192323  8962 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.224754  2380 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:26.207521  9099 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.233896  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:26.237134  1690 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.240403  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 164764us
W20260709 13:04:26.244722  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.215205  9081 raft_consensus.cc:697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:26.218400  9116 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:26.228255  9041 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:26.230424  8962 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:26.244889  9019 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:26.266600  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 157103us
W20260709 13:04:26.269404  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 150719us
I20260709 13:04:26.269194  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:26.297804  9043 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:26.298491  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:26.300787  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:26.301841  2378 raft_consensus.cc:2393] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 19: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:26.303721  9116 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:26.304993  9116 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:26.309108  9109 log.cc:927] Time spent T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.208s	user 0.000s	sys 0.004s
I20260709 13:04:26.269294  2342 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:26.304993  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:26.231630  9050 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:26.316687  9105 log.cc:927] Time spent T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.405s	user 0.004s	sys 0.002s
I20260709 13:04:26.310093  9081 consensus_queue.cc:237] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:26.270143  9019 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.269487  1614 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:26.313869  2371 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:26.265157  9048 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 25 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:26.351060  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81962.7us
I20260709 13:04:26.355782  9110 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.358282  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
W20260709 13:04:26.360211  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 90798.6us
I20260709 13:04:26.360489  9022 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:26.361668  9022 consensus_queue.cc:1048] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:26.352308  9019 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:26.363773  9007 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:26.364553  9007 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:26.365121  9115 raft_consensus.cc:2804] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:26.365984  9115 raft_consensus.cc:697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:26.365741  9007 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:26.368673  8869 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:26.371866  2373 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:04:26.367565  9115 consensus_queue.cc:237] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:26.376377  9050 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:26.357227  9088 raft_consensus.cc:2804] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:26.379981  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:26.383149  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:26.387303  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:26.388064  1719 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:26.351410  9034 consensus_queue.cc:1059] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:04:26.389132  9034 raft_consensus.cc:3055] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:26.389784  9034 raft_consensus.cc:740] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:26.391043  9034 consensus_queue.cc:260] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:26.395026  8962 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:26.400524  9088 raft_consensus.cc:697] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:26.402243  9088 consensus_queue.cc:237] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:26.411499  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.368027  9007 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:26.417982  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:26.447506  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.450608  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:26.451406  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.451653  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.452647  1645 raft_consensus.cc:2410] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:04:26.452740  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:26.453527  1650 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:26.453470  9058 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:26.456307  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.457993  1647 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:26.459234  1643 raft_consensus.cc:1275] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:26.460073  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.461099  1649 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 23: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:26.461357  1647 raft_consensus.cc:1275] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:04:26.461755  1647 pending_rounds.cc:85] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Aborting all ops after (but not including) 9
I20260709 13:04:26.462069  1647 pending_rounds.cc:107] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:04:26.462543  1647 raft_consensus.cc:2889] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:26.463092  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:26.464085  1649 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 23: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:26.465826  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:26.466558  1649 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:26.469084  1649 raft_consensus.cc:1275] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:04:26.470340  8836 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.470954  8836 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:26.474545  8836 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:26.480176  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 129700us
I20260709 13:04:26.482101  9127 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:26.475539  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:26.492540  1650 raft_consensus.cc:2468] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:26.530196  9043 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.445497  9034 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:26.546854  1719 raft_consensus.cc:2468] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 23.
I20260709 13:04:26.549543  1614 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:26.554136  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:26.483211  8962 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
W20260709 13:04:26.571794  1690 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.573051  1690 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.573714  1690 leader_election.cc:336] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.574129  1690 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:26.575119  9097 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:26.593348  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 59324.7us
I20260709 13:04:26.528954  8836 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
I20260709 13:04:26.594632  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: da3b57321031416cba9db75450ef302a
I20260709 13:04:26.549629  9043 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:26.555399  9126 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:26.598145  9116 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:26.846098  9116 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:26.846900  9116 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:26.903237  9116 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:26.910597  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.880s (SENT)
I20260709 13:04:26.913095  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:26.917788  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.917424  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 304433us
W20260709 13:04:26.918785  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.940578  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 346014us
I20260709 13:04:26.938042  9041 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:26.990710  9137 log.cc:927] Time spent T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.398s	user 0.002s	sys 0.009s
I20260709 13:04:26.990247  9126 raft_consensus.cc:697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:27.063285  9126 consensus_queue.cc:237] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.049484  9043 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:27.049681  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 561581us
I20260709 13:04:27.049073  9144 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.017202  8962 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.061424  9007 consensus_meta.cc:287] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.005s
W20260709 13:04:27.043140  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.070290  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.632s (SENT)
W20260709 13:04:27.061305  9140 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:04:27.023010  9139 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:04:27.085059  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.104923  1688 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.001s	sys 0.009s
W20260709 13:04:27.073529  9019 consensus_meta.cc:287] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.007s
I20260709 13:04:27.106302  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:27.116550  1616 leader_election.cc:336] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.116312  1688 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.002s	sys 0.009s
I20260709 13:04:27.167548  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:27.167837  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 619657us
W20260709 13:04:27.132772   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 1601)
W20260709 13:04:27.121209  9136 log.cc:927] Time spent T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.188s	user 0.000s	sys 0.002s
W20260709 13:04:27.184165  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 127372us
I20260709 13:04:27.105384  9081 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:27.182654  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:27.208578  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 138675us
I20260709 13:04:27.099339  9144 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.213318  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:27.105700  8962 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.177132  9127 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:27.170724  9116 consensus_queue.cc:1059] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:04:27.221479  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 105119us
I20260709 13:04:27.173235  9019 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:27.237187  8906 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:27.211941  9081 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.237938  8906 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:27.207324  9124 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:27.229647  1616 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.246173  2374 raft_consensus.cc:1275] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:27.244300  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.221999  8847 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:27.244426  8906 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.267074  2374 raft_consensus.cc:1275] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:04:27.275601  9138 log.cc:927] Time spent T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.678s	user 0.005s	sys 0.008s
I20260709 13:04:27.278033  9148 consensus_queue.cc:1059] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:27.278048  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2; no voters: 
I20260709 13:04:27.280485  9144 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:27.286959  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 73762.5us
W20260709 13:04:27.297688  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70187.9us
I20260709 13:04:27.295962  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:27.294639  8842 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:27.299299  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.312292  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:27.328904  1616 leader_election.cc:336] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.330138  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.304535  8847 consensus_queue.cc:1048] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:27.336673  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:27.328755  9097 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:27.340693  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 62730.3us
I20260709 13:04:27.341290  9154 consensus_queue.cc:1048] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:04:27.343217  1539 leader_election.cc:400] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
I20260709 13:04:27.343611  1539 leader_election.cc:403] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:04:27.346506  9081 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:27.351496  8842 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.354573  9150 raft_consensus.cc:3055] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:27.355172  9150 raft_consensus.cc:740] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:04:27.357201  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 781404us
W20260709 13:04:27.359234  1690 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.362906  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.364621  9162 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.366693  8962 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:27.329648  9123 consensus_queue.cc:1048] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:04:27.370448  9007 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:27.375085  9019 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:27.380791  9116 raft_consensus.cc:2804] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:27.380482  8869 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:27.381352  9116 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.381839  9116 raft_consensus.cc:3060] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:27.374948  9146 raft_consensus.cc:3055] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:27.370364  9029 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:27.393327  1723 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:04:27.396629  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67762.8us
W20260709 13:04:27.397708  1616 leader_election.cc:336] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.398413  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.398855  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:27.399912  2374 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.400941  1721 raft_consensus.cc:1275] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:27.404788  9081 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:27.409335  2374 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
I20260709 13:04:27.410364  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:27.411041  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.411448  1718 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.417297  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:27.412906  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.422942  8869 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:27.423804  1614 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:27.425850  8869 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:27.426741  8869 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:27.428274  9019 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:27.428515  8869 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:27.391017  9146 raft_consensus.cc:740] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:27.392052  9029 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:27.469250  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 71604.2us
I20260709 13:04:27.408962  9144 raft_consensus.cc:3060] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:27.473778  1690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.449764  8781 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:27.471827  1616 leader_election.cc:336] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.478091  1616 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:27.474877  8953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.041s	sys 0.035s
I20260709 13:04:27.474404  9029 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.420571  8962 consensus_queue.cc:1059] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
W20260709 13:04:27.494882  8953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.042s	sys 0.035s
I20260709 13:04:27.488848  8913 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:27.505946  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.416234  9150 consensus_queue.cc:260] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:27.513501  1690 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.515827  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:27.519956  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.536451  1616 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.537293  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.517916  8962 consensus_queue.cc:1243] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:04:27.525885  9158 log.cc:927] Time spent T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.294s	user 0.008s	sys 0.000s
I20260709 13:04:27.551011  1718 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
W20260709 13:04:27.552688  1616 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.525687  9150 raft_consensus.cc:3060] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.553170  1616 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.509670  9146 consensus_queue.cc:260] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:27.564800  1616 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.555316  9123 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:27.566349  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.569659  8848 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.567996  9156 consensus_queue.cc:1048] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
W20260709 13:04:27.572402  9163 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:04:27.576109  9043 raft_consensus.cc:3055] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:27.542140  9148 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:04:27.617959  9029 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.621985  9029 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:27.556964  9007 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:27.597879  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.585318  9034 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 1.022s	user 0.015s	sys 0.003s
I20260709 13:04:27.648310  9034 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:27.648989  9034 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:27.661067  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 147556us
W20260709 13:04:27.662663  1690 leader_election.cc:336] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.663337  1690 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:04:27.665705  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.670403  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:27.672542  9097 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.672596  9156 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:27.675012  9156 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:27.675259  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 996f59fd8f2340a49403b3e9ef4c8e86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader da3b57321031416cba9db75450ef302a (127.0.107.196:45309) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica da3b57321031416cba9db75450ef302a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:27.627249  9043 raft_consensus.cc:740] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:27.685897  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:27.698284  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:27.665285  8848 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.705083  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:27.706321  9043 consensus_queue.cc:260] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.707265  9081 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.707711  9043 raft_consensus.cc:3060] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.708220  9081 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:27.711876  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.930s (SENT)
I20260709 13:04:27.715795  2344 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.716284  2373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:27.724699  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:27.728073  2373 raft_consensus.cc:2393] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 25: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:27.736065  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:27.739421  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:27.740554  1650 raft_consensus.cc:2410] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:04:27.744122  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:27.744688  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:27.748417  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 62619.9us
I20260709 13:04:27.728467  9058 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.754045  1642 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 21: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:27.728498  8913 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:27.767724  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 55926.3us
I20260709 13:04:27.766692  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:27.731632  9106 raft_consensus.cc:493] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:27.705071  8848 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.736380  9173 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:27.752032  9156 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:27.795176  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.801937  1616 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.826097  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.802903  1616 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.774358  8913 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:27.778659  9106 raft_consensus.cc:515] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:27.863363  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:27.879065  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:27.884370  1645 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.882742  1642 raft_consensus.cc:2376] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21. Re-sending same reply.
I20260709 13:04:27.868412  9155 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:27.887396  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 89746.3us
I20260709 13:04:27.892530  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:27.896791  2376 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 17.
I20260709 13:04:27.792853  9173 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:27.895519  9146 raft_consensus.cc:3060] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:27.920800  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 118898us
I20260709 13:04:27.892416  9034 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:27.926963  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:27.927268  1616 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.896670  9081 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:27.937193  1616 leader_election.cc:336] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.937943  1616 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:27.959484  1616 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.962896  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.915861  9156 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:27.988832  9156 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:27.976230  1616 leader_election.cc:336] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.993033  1616 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:27.973986  9148 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.001375  1616 leader_election.cc:336] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.004141  1616 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:28.010048  9178 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:04:28.003389  9179 log.cc:927] Time spent T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
W20260709 13:04:28.004263  9116 consensus_meta.cc:287] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.622s	user 0.014s	sys 0.004s
W20260709 13:04:28.015718  1616 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:28.042716  1616 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.043342  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.003146  9125 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.005434  9182 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:28.047382  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 125503us
I20260709 13:04:28.000669  9177 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:28.025929  9155 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:28.035802  8962 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.074898  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.039458  9154 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:28.011540  9148 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.096165  9125 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.073591  8836 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:28.097134  9125 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:28.113188  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70619.3us
W20260709 13:04:28.114380  1616 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.052403  9116 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:28.116092  1616 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.120008  9180 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:28.123188  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 55470.4us
W20260709 13:04:28.120628  9175 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.264s	user 0.002s	sys 0.000s
I20260709 13:04:28.102178  8953 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.123294  1616 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.114861  9106 leader_election.cc:290] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:28.100637  8913 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:28.129921  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.085673  8962 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.067093  9182 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:28.138163  9150 consensus_meta.cc:287] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.000s
W20260709 13:04:28.139452  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.112605  8848 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.112479  9177 raft_consensus.cc:3060] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:28.148010  8906 consensus_meta.cc:287] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.011s
W20260709 13:04:28.132803  1616 leader_election.cc:336] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.168744  1616 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.168677  8906 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:28.171139  1616 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.171778  1616 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:28.105929  9154 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:28.155117  9148 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.155491  9173 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.133471  8953 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.157025  9150 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:28.131237  9180 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.159626  8962 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:28.177888   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 1601)
W20260709 13:04:28.181996  1616 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.185916  1616 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.183319  9150 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:28.172266  8906 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:28.186139  1650 raft_consensus.cc:1275] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:28.186122  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:28.188262  1616 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.189476  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.191129  1688 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:28.191485  1616 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:28.192420  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.192276  1616 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:28.195585  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:28.175725  8848 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.178299  9081 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.203567  8953 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:28.204165  9106 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.206717  9180 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:28.207248  9029 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.193442  9190 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:28.188515  9150 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:28.210950  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:28.213413  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 83601.1us
I20260709 13:04:28.196770  9197 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.188795  8316 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.222299  1539 leader_election.cc:400] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
W20260709 13:04:28.211423  1616 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.179492  9156 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.225116  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.215646  8906 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:28.237725  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.230685  8953 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.240693  9029 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:28.241202  9043 consensus_meta.cc:287] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.533s	user 0.016s	sys 0.000s
I20260709 13:04:28.232653  9198 consensus_queue.cc:1048] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
W20260709 13:04:28.255798  9144 consensus_meta.cc:287] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.005s
I20260709 13:04:28.278214  1688 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:28.296030  1645 raft_consensus.cc:2468] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:28.272662  9116 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.273559  9144 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) with higher term. Message: Invalid argument: T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21.
I20260709 13:04:28.305116  9172 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.327045  9177 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.329265  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.331856  9144 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:28.333022  9174 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.456s	user 0.004s	sys 0.004s
I20260709 13:04:28.332834  9144 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:28.332019  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:28.345196  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.929s (SENT)
I20260709 13:04:28.345784  2344 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.347816  9196 raft_consensus.cc:2804] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:28.301640  9154 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:28.380738  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 48803.7us
I20260709 13:04:28.344023  9034 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.390792  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:28.391574  1651 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:28.389066  9197 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.394284  9197 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:28.403097  9197 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:28.403988  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.405043  9116 raft_consensus.cc:3060] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:28.410571  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.412178  9043 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:28.412853  9043 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:28.413280  9043 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.357373  9194 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.360152  9144 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.360545  9177 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:28.426118  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.427156  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:28.431138  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:28.431296  1649 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.434031  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:28.429497  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:28.434506  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.386093  9195 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:28.440577  1616 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.390038  9019 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:28.441201  1616 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.441318  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.443163  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:28.429497  9185 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:28.438009  9177 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:28.457266  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.417323  9196 raft_consensus.cc:697] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:28.474009  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.475685  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:28.432237  9198 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:28.491923  2372 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:28.494266  1718 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:04:28.494480  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.462288  9081 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:28.501638  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 72235us
W20260709 13:04:28.479919  9146 consensus_meta.cc:287] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 13:04:28.504081  1688 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:28.506011  2378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363173567dd940e99e459f02148a42e0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:28.513707  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:28.475514  9125 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:28.520116  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.984s (SENT)
I20260709 13:04:28.500507  9198 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:28.530480  1723 raft_consensus.cc:1275] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:28.533152  9171 raft_consensus.cc:2804] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:28.533895  9204 log.cc:927] Time spent T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:28.534015  9171 raft_consensus.cc:697] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:28.473681  9185 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.535322  9171 consensus_queue.cc:237] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:28.537257  9206 log.cc:927] Time spent T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:04:28.537997  9185 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:28.467814  9195 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:28.543749  9195 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.544508  9183 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:28.484705  9196 consensus_queue.cc:237] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.475724  9019 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.550679  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:28.566527  9156 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:28.566655  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:28.567335  9156 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:28.567812  9148 consensus_queue.cc:1048] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:28.568537  9156 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.569869  9148 raft_consensus.cc:3060] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.528867  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.573740  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.552323  9081 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:28.569767  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.540074  9177 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.514657  2372 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 25.
I20260709 13:04:28.561852  9183 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.588469  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.589560  1719 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 21.
I20260709 13:04:28.589686  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.594558  1649 raft_consensus.cc:2468] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:28.598155  1614 leader_election.cc:304] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:28.601722  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 81092.5us
I20260709 13:04:28.566509  9019 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.582823  9183 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.575933  9198 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.576386  9150 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.605533  9177 raft_consensus.cc:2804] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:28.612021  9116 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:28.625977  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.614132  9186 raft_consensus.cc:2749] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.623435  9177 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting leader election (detected failure of leader da3b57321031416cba9db75450ef302a)
W20260709 13:04:28.661940  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 153780us
I20260709 13:04:28.686975  9177 raft_consensus.cc:3060] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:28.689767  1690 leader_election.cc:336] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.625684  9126 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:04:28.665360  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.674785  9182 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.676760  9146 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.677318  9144 consensus_queue.cc:1048] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:28.680018  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:28.685940  9150 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:28.659729  9043 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:28.731050  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:28.726617  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:28.736876  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.740816  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:28.744231  1690 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:28.745667  1690 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:28.746376  1690 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.759497  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:28.763437  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.749130  8836 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:28.723482  9146 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:28.799212  9146 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:28.800756  9106 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:28.801405  9106 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.801864  9106 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:28.804926  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:28.770918  9150 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:28.812709  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 1acf73e529394004842219637cff7cfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:28.816934  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.732512  8316 consensus_queue.cc:1048] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:04:28.832696  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 108665us
I20260709 13:04:28.835985  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:28.841651  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.846423  9144 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.851359  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:28.814980  8836 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:28.832876  8906 consensus_meta.cc:287] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.004s
I20260709 13:04:28.860118  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:28.868511  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.873840  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:28.848567  9195 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.853168  9198 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:28.871409  8836 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:28.871117  8906 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:28.896867  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 141778us
I20260709 13:04:28.891189  9058 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:28.916426  1720 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 25.
I20260709 13:04:28.920640  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:28.929150  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.981127  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 78777.2us
I20260709 13:04:28.987425  9156 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.988457  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.005604  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.037307  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.049238  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.057201  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.061092  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.001798  9156 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.073217  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.076961  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.033428  9043 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.042068  9144 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.086454  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.087903  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.089784  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.098896  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.106572  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.109386  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:29.111330  2343 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.112301  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.114177  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.123198  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.133436  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:29.134343  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:29.125118  9218 log.cc:927] Time spent T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:04:29.138844  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.136147  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.140717  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.141471  1644 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 23.
I20260709 13:04:29.147044  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.150629  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.103389  9144 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.152429  1719 raft_consensus.cc:1217] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Deduplicated request from leader. Original: 17.11->[18.12-18.12]   Dedup: 18.12->[]
I20260709 13:04:29.153633  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.154315  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.154959  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:29.139294  9148 consensus_meta.cc:287] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 13:04:29.156437  1719 raft_consensus.cc:1240] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:04:29.162108  1649 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:04:29.163688  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:29.174237  9222 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:04:29.175071  1725 raft_consensus.cc:1240] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 22. Current term is 23. Ops: [22.11-22.11]
W20260709 13:04:29.200327  9183 consensus_meta.cc:287] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: Time spent flushing consensus metadata: real 0.570s	user 0.014s	sys 0.000s
W20260709 13:04:29.202250  9221 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:04:29.207417  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:29.217020  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.217655  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 740763us
W20260709 13:04:29.218462  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.219506  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:29.189344  9148 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:29.226243  9144 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:29.226511  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.771s (SENT)
I20260709 13:04:29.227480  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.185539  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.168380  9106 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:29.229890  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.167248  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:29.229874  1719 raft_consensus.cc:1217] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Deduplicated request from leader. Original: 20.13->[21.14-21.14]   Dedup: 21.14->[]
I20260709 13:04:29.196750  1716 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:29.227694  1717 raft_consensus.cc:1217] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Deduplicated request from leader. Original: 19.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 13:04:29.231786  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:29.232430  9177 consensus_meta.cc:287] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.006s
I20260709 13:04:29.194401  1720 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 23.
W20260709 13:04:29.220146  1616 leader_election.cc:336] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:29.233388  9182 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:29.220077  9219 log.cc:927] Time spent T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.191s	user 0.002s	sys 0.000s
I20260709 13:04:29.234632  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:29.235301  2373 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:29.235888  2378 raft_consensus.cc:1217] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Deduplicated request from leader. Original: 17.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:04:29.235319  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.236569  9106 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 election: Requested vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:29.236783  2371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:29.237591  2344 leader_election.cc:336] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.780s (SENT)
I20260709 13:04:29.278967  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:29.282557  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45018.5us
I20260709 13:04:29.282802  2378 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:29.238545  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.239234  1718 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:29.239514  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.186493   431 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 1601)
I20260709 13:04:29.291513  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.234645  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.231484  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.253674  1717 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:04:29.200999  9183 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:29.274194  1719 raft_consensus.cc:1240] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 13:04:29.232075  9216 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
I20260709 13:04:29.279152  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:29.301952  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.276064  1616 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:29.230376  9156 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.303751  1722 raft_consensus.cc:2393] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 21: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
W20260709 13:04:29.306118  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 85987.1us
I20260709 13:04:29.310580  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.279145  8913 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:29.302508  9048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.008s	sys 0.017s
I20260709 13:04:29.324749  1724 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:29.279937  9177 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:29.325133  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:29.340049  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.343580  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:29.310081  8316 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.309976  8869 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:29.362910  1719 raft_consensus.cc:2393] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 23: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:29.313326  9212 log.cc:927] Time spent T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.633s	user 0.003s	sys 0.003s
I20260709 13:04:29.360954  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:29.318980  9048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.008s	sys 0.017s
I20260709 13:04:29.309976  9019 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:29.373030  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65977us
I20260709 13:04:29.375977  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.379431  1648 raft_consensus.cc:1217] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Deduplicated request from leader. Original: 17.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:04:29.380033  1648 raft_consensus.cc:1240] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:29.328152  8913 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.387373  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:29.390244  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:29.391047  1721 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:29.394052  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:29.391242  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.403139  9058 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:29.406026  9058 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:29.408145  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 1.517s (SENT)
I20260709 13:04:29.408869  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.423367  9058 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:29.424914  1690 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:29.427951  9227 log.cc:927] Time spent T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:04:29.367060  8316 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:29.438339  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.026s (SENT)
W20260709 13:04:29.440202  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Timed out: RequestConsensusVote RPC to 127.0.107.193:45275 timed out after 2.038s (SENT)
I20260709 13:04:29.429584  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.426653  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:29.441691  2380 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:29.377701  9019 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.410686  9097 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:29.445386  1688 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.450403  9124 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.451251  9019 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.451342  9124 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.457887  9124 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.433514  9182 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:29.463722  1721 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 26.
I20260709 13:04:29.465094  1643 raft_consensus.cc:1240] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:29.470003  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:29.470896  1649 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:29.472864  1645 raft_consensus.cc:1217] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Deduplicated request from leader. Original: 19.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 13:04:29.473474  1645 raft_consensus.cc:1240] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:04:29.475011  1647 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:29.480183  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.491189  8913 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.460820  9007 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:29.450434  9177 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:29.494201  8316 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:29.495107  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.445040  8847 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:29.481110  1648 raft_consensus.cc:2468] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 19.
I20260709 13:04:29.447492  9097 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:29.506793  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66639.8us
I20260709 13:04:29.507002  1688 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:29.517812  1649 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:29.519304  2380 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 26.
I20260709 13:04:29.521801  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:29.524070  9177 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:04:29.525503  9171 raft_consensus.cc:2804] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:29.533390  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.533711  9171 raft_consensus.cc:697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:29.534958  9171 consensus_queue.cc:237] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.535513  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.536788  9177 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:29.538051  9177 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:29.539515  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.528725  9156 consensus_queue.cc:1059] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:04:29.540419  9177 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.541078  9156 raft_consensus.cc:3055] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:29.541525  9156 raft_consensus.cc:740] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:29.541558  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.542794  9177 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.542907  9156 consensus_queue.cc:260] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:29.543850  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.544972  9177 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:29.545270  9156 raft_consensus.cc:3060] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:29.546777  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:29.547689  9177 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:29.525045  9183 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:29.548389  9177 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.548915  9177 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:29.524524  9124 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:29.549922  9124 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.509420  9097 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:29.553045  9124 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.559580  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.559859  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.561285  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.561807  9029 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.562300  9144 consensus_queue.cc:1059] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:29.562492  9029 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.563459  9148 raft_consensus.cc:3055] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:29.563854  9148 raft_consensus.cc:740] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:29.564687  1723 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 23.
I20260709 13:04:29.564834  9148 consensus_queue.cc:260] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.566493  9148 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:29.561841  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.571532  1614 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:29.583027  9029 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:29.584643  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.586186  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.587543  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.589851  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:29.590473  9177 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.592015  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.595412  9144 raft_consensus.cc:3060] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.597657  9124 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:29.598362  9124 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 26 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:29.599545  2374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:29.599392  9124 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:29.600394  2374 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:29.605715  9144 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:29.606444  9148 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:29.608963  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.611670  9177 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:29.613709  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.615569  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:29.619191  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.621467  2374 raft_consensus.cc:2468] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:29.614463  9148 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:29.628226  9144 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:29.635766  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.639259  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:29.691062  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52116.6us
W20260709 13:04:29.630321  8836 consensus_meta.cc:287] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.003s
I20260709 13:04:29.678413  9097 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:29.698886  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66151.8us
I20260709 13:04:29.702354  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.707722  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:29.708461  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:29.713977  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:29.714831  1649 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:29.715073  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.716512  1647 raft_consensus.cc:2410] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:04:29.717658  1724 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:29.706586  1688 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:29.721109  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 103572us
I20260709 13:04:29.723037  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:29.698225  8316 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:04:29.725587  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:29.726025  8316 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.726395  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:29.706465  9043 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:29.726641  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.728981  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:29.729517  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.730543  1649 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 18.
W20260709 13:04:29.747295  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.796443  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:29.799414  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 76433.3us
I20260709 13:04:29.800882  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.759945  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.741766  1648 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:04:29.794677  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70080us
I20260709 13:04:29.812485  1649 raft_consensus.cc:1240] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:29.813436  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.730932  9224 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:29.821123  1540 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.726504  8316 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:29.837837  1690 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:29.839778  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:29.893134  1650 raft_consensus.cc:1275] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:29.894433  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:29.698976  8836 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:29.835925  9177 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:29.736841  9144 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:29.733633  9196 raft_consensus.cc:2804] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:29.915593  1646 raft_consensus.cc:1275] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:29.718343  9148 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:29.740546  8842 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:29.901463  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 996f59fd8f2340a49403b3e9ef4c8e86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader da3b57321031416cba9db75450ef302a (127.0.107.196:45309) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica da3b57321031416cba9db75450ef302a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:29.836556  9156 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:29.918457  1642 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Rejecting Update request from peer 58cdb682fbf54792a702a7a455f265a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:29.815301  9043 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:29.919304  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:29.914474  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.945634  1644 raft_consensus.cc:2368] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22.
I20260709 13:04:29.946380  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:29.950748  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.914549  9034 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:29.918987  9124 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:29.914667  9126 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:29.942258  9177 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:29.974620  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54674us
W20260709 13:04:29.980089  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 211) took 35137 ms (client timeout 15000 ms). Trace:
I20260709 13:04:29.981465  9106 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:30.032438  9126 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.035617  9126 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:30.004390  9239 log.cc:927] Time spent T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:04:30.036753  9144 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:30.033094  1466 rpcz_store.cc:269] 0709 13:03:54.842986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.384900 (+7541914us) service_pool.cc:224] Handling call
0709 13:04:29.979965 (+27595065us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:25.863325 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:25.889460 (+ 26135us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:25.889532 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:04:25.889550 (+    18us) tablet.cc:662] Decoding operations
0709 13:04:26.174334 (+284784us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:26.174439 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:26.174486 (+    47us) tablet.cc:685] Acquiring locks for 33 operations
0709 13:04:26.177684 (+  3198us) tablet.cc:701] Row locks acquired
0709 13:04:26.177736 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:04:26.177969 (+   233us) write_op.cc:270] Start()
0709 13:04:26.178183 (+   214us) write_op.cc:276] Timestamp: P: 1783602266177878 usec, L: 0
0709 13:04:26.178208 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:04:26.179182 (+   974us) log.cc:844] Serialized 12967 byte log entry
0709 13:04:26.573856 (+394674us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b18003e9aa0. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc839b49de1 00007fc839b565b1 00007fc830c813f3 00007fc830c837c1 00007fc830c83779 00007fc830c83709 00007fc830c836d1 00007fc830c829cd 000000000038f354 000000000038f199 00007fc830c6483a
0709 13:04:27.041805 (+467949us) op_driver.cc:464] REPLICATION: finished
0709 13:04:27.334100 (+292295us) write_op.cc:301] APPLY: starting
0709 13:04:27.377251 (+ 43151us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:28.201158 (+823907us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:28.202881 (+  1723us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:29.205269 (+1002388us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:29.206298 (+  1029us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=66,key_file_lookups=66,delta_file_lookups=0,mrs_lookups=0
0709 13:04:29.206358 (+    60us) write_op.cc:312] APPLY: finished
0709 13:04:29.207169 (+   811us) log.cc:844] Serialized 284 byte log entry
0709 13:04:29.342861 (+135692us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.375830 (+ 32969us) write_op.cc:454] Released schema lock
0709 13:04:29.397198 (+ 21368us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:29.482405 (+ 85207us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.500499 (+ 18094us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":227755,"apply.run_cpu_time_us":93197,"apply.run_wall_time_us":2144614,"cfile_cache_hit":134,"cfile_cache_hit_bytes":142265,"num_ops":33,"prepare.queue_time_us":321567,"prepare.run_cpu_time_us":22551,"prepare.run_wall_time_us":589857,"raft.queue_time_us":451652,"raft.run_cpu_time_us":46475,"raft.run_wall_time_us":145510,"replication_time_us":863018,"spinlock_wait_cycles":84111872,"thread_start_us":671234,"threads_started":5,"wal-append.queue_time_us":402385,"wal-append.run_cpu_time_us":9891,"wal-append.run_wall_time_us":1192264}]]}
I20260709 13:04:30.037400  9043 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:30.037415  9144 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.038457  9043 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.014351  9097 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:30.033392  1616 leader_election.cc:336] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:29.967805  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.941792  8842 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:30.041951  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 82219.8us
I20260709 13:04:30.042821  1616 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:29.994740  9034 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:30.046700  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 206960us
I20260709 13:04:30.064689  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:30.043959  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.049093  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:30.074940  1616 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.076879  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:30.077494  1616 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:30.094713  1616 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.095332  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:30.096372  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:30.099225  1616 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.045392  9144 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:30.116230  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 203997us
I20260709 13:04:30.097198  9019 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:30.086148  9106 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:30.093014  9177 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:30.072302  9196 raft_consensus.cc:697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:30.044057  9124 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.097601  9252 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:30.172746  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:30.175592  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 71972us
W20260709 13:04:30.175410  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 79487.2us
I20260709 13:04:30.130894  9144 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:30.105154  9162 consensus_queue.cc:1059] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:04:30.142912  9123 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:30.192589  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.191771  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:30.193756  1643 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.132685  9029 consensus_queue.cc:1048] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:04:30.201957  9124 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:30.214933  9254 log.cc:927] Time spent T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:04:30.180195  9177 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:30.243633  9161 consensus_queue.cc:1048] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:04:30.228932  9185 raft_consensus.cc:3055] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:30.260756  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 44173.3us
I20260709 13:04:30.219046  9144 consensus_queue.cc:1059] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:04:30.261744  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.270361  9248 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:30.272600  9248 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:30.273114  9248 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:30.260965  9247 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:30.275283  9247 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:30.263911  9246 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:30.289075   431 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 1601)
I20260709 13:04:30.294989  1643 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 20.
I20260709 13:04:30.295295  9246 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:30.264526  9177 consensus_queue.cc:1059] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:04:30.297133  9247 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:30.302639  9177 raft_consensus.cc:3055] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:30.303244  9177 raft_consensus.cc:740] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.304456  9177 consensus_queue.cc:260] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.297576  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.308528  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.309120  9148 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:30.310416  9148 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:30.310009  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:30.253014  9256 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:04:30.313475  9256 consensus_queue.cc:1243] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer 58cdb682fbf54792a702a7a455f265a2 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:30.314235  9256 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:30.315250  9256 consensus_queue.cc:1059] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:04:30.275866  9243 log.cc:927] Time spent T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:04:30.326789  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.332191  9043 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.332955  9246 consensus_queue.cc:1059] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:30.257897  9185 raft_consensus.cc:740] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:30.336856  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:30.341223  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:30.334635  9177 raft_consensus.cc:3060] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:30.346812  2376 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:04:30.349320  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:30.274147  9248 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.264568  9196 consensus_queue.cc:237] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:30.368104  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54557.3us
I20260709 13:04:30.369153  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:30.371554  1723 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:30.338771  9148 raft_consensus.cc:3055] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:30.373143  2372 raft_consensus.cc:1240] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:04:30.380522  9240 log.cc:927] Time spent T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.374s	user 0.004s	sys 0.003s
I20260709 13:04:30.383653  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:30.386493  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.386581  1645 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:30.387269  9144 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.387898  9144 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:30.389859  9156 raft_consensus.cc:3055] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:30.384992  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.390576  9156 raft_consensus.cc:740] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:04:30.374060  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.397981  1645 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 24.
W20260709 13:04:30.400959  1616 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 199 similar messages]
I20260709 13:04:30.352259  9043 raft_consensus.cc:3055] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:30.316291  9256 raft_consensus.cc:3055] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:30.405200  8836 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.408921  9185 consensus_queue.cc:260] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.417109  9262 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.375543  9260 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:30.437592  9034 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:30.402107  1616 leader_election.cc:336] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.390524  9144 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:30.445525  8316 consensus_meta.cc:287] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: Time spent flushing consensus metadata: real 0.549s	user 0.014s	sys 0.000s
W20260709 13:04:30.465555  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59463.9us
I20260709 13:04:30.444175  1688 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:30.395845  9148 raft_consensus.cc:740] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.393440  9156 consensus_queue.cc:260] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.451292  9248 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:30.404788  9246 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.435809  9197 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:04:30.445573  8842 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.658098  1723 raft_consensus.cc:2468] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
W20260709 13:04:30.509042  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 64993.6us
I20260709 13:04:30.474802  9256 raft_consensus.cc:740] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.503559  9262 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.663903  9262 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.505481  9260 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.506973  9144 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.475056  9043 raft_consensus.cc:740] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.668684  9144 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.669512  9043 consensus_queue.cc:260] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.671417  9043 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.674296  9260 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.640717  9185 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.504096  1616 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:30.643827  9265 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.675832  9265 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.676848  1468 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 21, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.679071  9265 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.660094  9246 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:30.680295  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 278176us
I20260709 13:04:30.682909  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:30.506920  9029 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:30.510035  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:30.693704  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 185904us
W20260709 13:04:30.695302  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:30.698355  1616 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.699944  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:30.701138  1689 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.647233  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:30.647972  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 136939us
I20260709 13:04:30.648870  9266 consensus_queue.cc:1059] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:30.656008  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:30.705287  1719 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:30.663911  1688 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.507578  8316 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.685552  8913 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:30.686882  9144 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.707769  9172 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.708768  8316 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.709913  1650 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 24: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:30.689559  9029 raft_consensus.cc:697] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:30.643898  9156 raft_consensus.cc:3060] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:30.644294  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 133713us
I20260709 13:04:30.715683  9172 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } attrs { replace: true } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:30.639649  9148 consensus_queue.cc:260] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:30.721604  9256 consensus_queue.cc:260] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.723616  9256 raft_consensus.cc:3060] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:30.724452  9263 consensus_queue.cc:1175] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3b57321031416cba9db75450ef302a" responder_term: 18 status { last_received { term: 16 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 17. Current term is 18. Ops: [17.14-17.14]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:30.705214  9265 raft_consensus.cc:2804] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:30.730988  9265 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:30.732033  9265 raft_consensus.cc:3060] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:30.736702  9034 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:30.737133  9144 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.737969  9144 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.747591  9172 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:30.749316  9148 raft_consensus.cc:3060] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:30.752326  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 54015.3us
W20260709 13:04:30.753312  1616 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.755148  9144 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:30.756564  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 109421us
I20260709 13:04:30.756950  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:30.679880  1468 catalog_manager.cc:5697] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 17. New cstate: current_term: 17 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:30.760773  9034 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:30.761473  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:30.762281  9246 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:30.765208  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.768955  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:30.757481  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:30.769948  1616 leader_election.cc:336] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.773218  1716 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 13:04:30.773545  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 77374.2us
I20260709 13:04:30.774026  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:30.774827  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.775401  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:30.775766  1690 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.760644  1468 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 20, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:30.773134  1616 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:30.777043  9034 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:30.777413  1468 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 12 to 15, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:30.778774  1616 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:30.779201  1719 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 24.
I20260709 13:04:30.785411  1539 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:30.804963  9029 consensus_queue.cc:237] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.787664  9177 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:30.806804  9263 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:30.807312  9177 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.807718  9263 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.808145  9246 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:30.808643  9177 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.808969  9263 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.814009  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:30.830120  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.831372  9265 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:30.787321  9144 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:30.860502  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 78702.5us
I20260709 13:04:30.862284  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.866668  9265 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:30.867743  9265 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:30.854282  1468 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 14 to 19, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.870743  1539 leader_election.cc:400] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22.
I20260709 13:04:30.871249  1539 leader_election.cc:403] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:30.877017  1539 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:30.882220  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:30.886674  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:30.887543  1722 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:30.897642  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.900305  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.913347  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.914974  1614 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:30.918843  1643 raft_consensus.cc:1275] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:30.921613  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.885030  9256 raft_consensus.cc:3060] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:30.885224  9144 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:30.922530  9081 raft_consensus.cc:2804] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:30.940656  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.941179  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:30.941710  1645 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 24: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:30.940997  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.942770  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:30.948963  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:30.951923  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 1acf73e529394004842219637cff7cfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:30.974225  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:30.922775  9265 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:30.934038  9156 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:30.984133  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.988605  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:30.980181  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.010406  9043 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.023916  9260 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.024799  9260 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.029393  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:30.925889  8842 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 13:04:31.027631  9260 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:30.979691  1690 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:31.051887  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 72213.9us
I20260709 13:04:30.988940  9081 raft_consensus.cc:697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
W20260709 13:04:31.053066  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 171439us
W20260709 13:04:31.052910  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 133246us
I20260709 13:04:30.980072  9144 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 24 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:30.993853  9266 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.067248  9081 consensus_queue.cc:237] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.073897  9266 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:30.986093  1468 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:31.097155  9144 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:31.106805  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45295.7us
W20260709 13:04:31.121179  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62007.7us
I20260709 13:04:31.138628  1722 raft_consensus.cc:2468] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 24.
I20260709 13:04:31.142829  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.143147  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.143693  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.145953  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:31.132902  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.082916  1468 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 19, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.131263  9148 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.151646  9172 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:31.165729  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.170397  9266 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:31.173367  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:31.187153  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.190093  9266 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.190819  9266 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.197856  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:31.198781  9260 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:31.166644  9148 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.199491  9260 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.202230  9148 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:31.157546  2344 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:31.204528  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50830.1us
W20260709 13:04:31.205376  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:31.154136  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45583.3us
W20260709 13:04:31.206544  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.207162  2344 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.160188  1723 raft_consensus.cc:1217] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Deduplicated request from leader. Original: 18.10->[20.11-20.11]   Dedup: 20.11->[]
I20260709 13:04:31.208427  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:31.208830  1723 raft_consensus.cc:1240] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:04:31.162647  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.211097  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.217058  1724 raft_consensus.cc:2393] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 24: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:31.156287  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 182149us
I20260709 13:04:31.226644  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.226684  1716 raft_consensus.cc:1240] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:04:31.227952  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.229578  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:31.231387  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.207724  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.250685  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43955.9us
I20260709 13:04:31.254878  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.202510  9156 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:31.202257  9260 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:31.252133  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.267565  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.205405  9266 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.270910  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.271955  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:31.273126  1541 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.220607  9272 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:31.277213  1614 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:31.287485  8848 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.288390  8848 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:31.290087   431 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 1601)
I20260709 13:04:31.292290  9247 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.293170  9247 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.295506  9247 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.297853  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.298336  9256 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) with higher term. Message: Invalid argument: T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e for earlier term 21. Current term is 22.
I20260709 13:04:31.299369  9256 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.300386  9256 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:31.259364  9043 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.304994  9043 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.204555  1468 catalog_manager.cc:5697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 19, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.311432  8848 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.311436  1468 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.315068  1468 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 16 to 23, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.317390  9197 raft_consensus.cc:2804] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:31.318202  9197 raft_consensus.cc:697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:31.317579  1468 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 10 to 16, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:31.319310  9197 consensus_queue.cc:237] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.300020  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.342725  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:31.324779  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 60332.9us
W20260709 13:04:31.345196  1616 leader_election.cc:336] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.348927  1616 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.347788  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.354387  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.357543  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.349327  9148 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:31.372630  8848 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:31.377249  8842 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:31.378387  1616 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.351505  9256 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:31.360858  9266 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.359462  9260 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.381685  8848 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.381294  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:31.381731  9156 raft_consensus.cc:3060] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:31.383230  1616 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.404673  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:31.403254  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 128386us
W20260709 13:04:31.410100  1616 leader_election.cc:336] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.410959  9256 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:31.417359  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.411134  1616 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.417359  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.400593  9148 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:31.439155  1616 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:31.436960  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 76730us
I20260709 13:04:31.426232  1468 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:31.437701  8842 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.418587  8316 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:31.471026  1616 leader_election.cc:336] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.474062  1616 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:31.476801  1616 leader_election.cc:336] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.477380  1616 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.481335  8848 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.466485  9043 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:31.495069  9277 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:31.523978  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 50325.5us
I20260709 13:04:31.533519  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.533156  9043 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.535761  9043 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.536248  9148 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.540978  9148 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.541637  9043 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.545032  8848 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.545674  8848 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.506263  9126 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:31.560772  8848 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.546674  1468 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 22, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.581401  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.568269  9148 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:31.587252  9156 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:31.601290  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.601410  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:31.602192  2344 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.608948  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.612705  1650 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 13:04:31.614719  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 79669.7us
I20260709 13:04:31.622061  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.622782  1722 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:04:31.630501  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:31.633343  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.640316  9263 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.641192  9263 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.612780  9287 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:31.571507  1468 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:31.654342  1542 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.655351  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:31.651391  1616 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.663108  9156 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:31.669072  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:31.681618  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.682595  1717 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 22.
W20260709 13:04:31.685470  1690 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.686112  1690 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:31.631234  9144 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.707217  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51977.1us
I20260709 13:04:31.711282  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.644137  9263 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:31.718988  1468 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 17, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.722419  9148 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:31.730283  9256 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:31.733353  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 48201.7us
I20260709 13:04:31.713965  9144 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.746616  9144 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:31.761361  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:31.762645  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:31.712757  9156 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:31.703451  9276 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.763262  9281 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.721457  1468 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:31.778910  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:31.771695  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 129391us
I20260709 13:04:31.780388  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:31.766086  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47928.9us
I20260709 13:04:31.772593  8842 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:31.788271  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:31.792171  8842 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.795436  8842 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:31.797854  1468 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 16, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:31.784204  9281 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.814595  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:31.830482  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:31.835150  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:31.835613  1616 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.835741  2344 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.838524  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:31.819339  9263 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:31.839452  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 71516.9us
I20260709 13:04:31.844144  1717 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:31.800271  1468 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.809059  9249 consensus_queue.cc:1048] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:04:31.854357  1650 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:04:31.836408  1616 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.864377  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.865695  9043 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.866400  9043 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:31.856817  9224 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.870776  2377 raft_consensus.cc:1275] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:31.871879  9197 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.873652  1468 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.879087  9281 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:31.836714  9156 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:31.838864  9144 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:31.885326  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49725us
I20260709 13:04:31.886300  9043 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:31.887583  1616 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.856856  9256 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:31.888620  1468 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 11 to 15, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.860283  9148 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.875640  8842 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.894090  9290 consensus_queue.cc:1048] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:31.897030  1616 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:31.906438  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:31.907275  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:31.906474  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:31.918354  1719 raft_consensus.cc:1275] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:04:31.934394  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.934938  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 100472us
W20260709 13:04:31.935645  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:31.936785  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:31.933320  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:31.939234  1648 raft_consensus.cc:1240] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Rejecting Update request from peer 4f737473248e4c059fa74af3b8e1f77e for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:04:31.900154  9256 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:31.910392  9295 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:31.958994  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51764.3us
W20260709 13:04:31.944124  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:31.960798  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:31.979671  1643 raft_consensus.cc:1275] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:31.980125  9156 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:31.989279  9156 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.989815  9156 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:31.997422  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:31.938958  9144 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:04:32.010239  9156 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.009936  1468 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:32.011298  8842 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:31.986014  9263 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:32.021085  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:31.981221  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.043373  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62199.8us
I20260709 13:04:31.920039  9148 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:32.070158  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 49177.3us
I20260709 13:04:32.077160  9156 consensus_queue.cc:1059] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:32.068688  1468 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.080487  9148 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.081338  9148 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.080798  1468 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:32.081893  9148 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.082922  1468 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 5 to 12, term changed from 15 to 17, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196), VOTER 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) evicted. New cstate: current_term: 17 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:32.089744  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 66865.9us
I20260709 13:04:32.062919  9263 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.119051  1468 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:31.969544  9256 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:32.126236  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.132896  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.110069  9144 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.138871  9148 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.115868  9126 consensus_queue.cc:1048] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:04:32.129014  9304 raft_consensus.cc:493] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.134513  9156 raft_consensus.cc:493] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.168653  1616 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.181598  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 79898.5us
I20260709 13:04:32.121270  1468 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 20, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.187009  1468 catalog_manager.cc:5697] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 17, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:32.137283  9172 consensus_queue.cc:1048] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:04:32.193092  1468 catalog_manager.cc:5697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 15, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.200001  1468 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 18 to 20, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:32.172456  9148 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.153839  9144 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.209177  1468 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 11 to 16, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.185842  9304 raft_consensus.cc:515] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.212309  1468 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.234413  1468 catalog_manager.cc:5697] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 18 to 19, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 19 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:32.185001  9156 raft_consensus.cc:515] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.236533  1468 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 12 to 13, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:32.237926  9300 log.cc:927] Time spent T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:04:32.220238  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:32.239413  9043 raft_consensus.cc:3055] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:32.239886  9043 raft_consensus.cc:740] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
W20260709 13:04:32.235436  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52499.3us
W20260709 13:04:32.242442  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:32.243687  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.246706  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:32.238868  1468 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 17, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.250092  9144 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.250500  1468 catalog_manager.cc:5697] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:32.261400  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:32.263824  1724 raft_consensus.cc:1275] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
W20260709 13:04:32.268441  1690 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:32.273090  1690 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.223606  9304 leader_election.cc:290] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:32.265762  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 108899us
I20260709 13:04:32.268796  9144 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.282181  9144 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.213150  9148 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:32.290699   431 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 1601)
W20260709 13:04:32.292013  9299 log.cc:927] Time spent T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:04:32.292683  9034 consensus_queue.cc:1048] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:32.290777  9156 leader_election.cc:290] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.303180  9148 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.312700  9144 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:32.324455  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50500.2us
I20260709 13:04:32.245710  9043 consensus_queue.cc:260] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:32.340391  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 225017us
I20260709 13:04:32.268982  9256 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:32.350628  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.351220  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:32.345038  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.348465  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.349401  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.355038  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.355883  1649 raft_consensus.cc:2468] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 20.
I20260709 13:04:32.359297  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.406399  1650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.406760  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.319594  9304 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.421206  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 81623.2us
W20260709 13:04:32.433681  1539 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.408383  9276 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.424751  9304 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.408663  9256 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.436194  9156 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:32.466953  9263 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.469637  9312 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.484789  9256 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.546160  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.550737  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.554239  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.559739  1719 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:04:32.562999  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.563160  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.564389  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da3ba11d664442d7b28bc849c61fdcdf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.566177  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9396e14142a4f6cb8733d8b08ac73b9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.566680  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:32.567700  9306 log.cc:927] Time spent T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 13:04:32.570290  9311 log.cc:927] Time spent T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:04:32.575776  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 122349us
I20260709 13:04:32.576259  1645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.522173  9263 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:32.523005  9312 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.606590  9148 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:32.622143  9304 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.648329  9312 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.650375  9263 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:32.658048  9256 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.640655  9148 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:32.671005  9312 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.673933  9312 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.668238  9263 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.677587  9263 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:32.682081  9043 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:32.687147  9312 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.680214  9263 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:32.695449  9148 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:32.706192  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:32.718438  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 366054us
I20260709 13:04:32.725668  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:32.726191  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:32.726979  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:32.726579  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:32.728515  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.729305  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.730301  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.730497  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.731525  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.732493  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.733902  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.734556  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.740065  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.740243  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.743305  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:32.740952  9148 raft_consensus.cc:2804] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.748023  9148 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.748673  9148 raft_consensus.cc:3060] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:32.747157  1539 leader_election.cc:304] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.750208  9304 raft_consensus.cc:2749] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:32.751240  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.752601  9304 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.754752  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.745230  9312 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:32.756693  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:32.758074  9256 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:32.756693  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.760192  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.767529  9312 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.753957  9144 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:32.815568  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 51364.9us
I20260709 13:04:32.823012  1539 leader_election.cc:304] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:32.807535  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.831830  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 49667.8us
W20260709 13:04:32.832588  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:32.833671  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:32.742693  9263 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:32.833177  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:32.828953  9144 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.837029  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:32.842574  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.829422  9256 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.849006  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 110022us
I20260709 13:04:32.831434  9304 raft_consensus.cc:2749] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:32.835698  9305 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:32.857365  9156 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:32.854655  9316 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:32.842882  9263 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.870137  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:32.900707  1616 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:32.903038  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:32.903399  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:32.842342  9144 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.918102  9263 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:32.862149  9256 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:32.904776  9304 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:32.958060  9256 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:32.885133  9305 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:32.974377  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71425.6us
I20260709 13:04:32.975636  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.977555  9256 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:32.977690  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:32.980129  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:32.980904  9338 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:32.997509  9312 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.997957  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:32.998261  9312 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.000871  9312 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.007401  9144 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.008833  9305 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:33.013187  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:33.023715  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.034025  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.038635  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.040532  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.043052  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.042879  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ef45e97cf24863a576b705ec8e07bd" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.023944  9256 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:33.026378  9305 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.045177  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:33.045599  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.109539  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:33.118160  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:33.049813  9305 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:33.198017  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87064.2us
W20260709 13:04:33.206526  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 88481.4us
I20260709 13:04:33.242000  9148 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:33.291440   431 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 1601)
I20260709 13:04:33.292101  9148 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.199802  9338 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.197587  9256 raft_consensus.cc:493] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.288751  9263 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:33.276201  9312 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:33.202714  9156 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:33.304669  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 996f59fd8f2340a49403b3e9ef4c8e86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader da3b57321031416cba9db75450ef302a (127.0.107.196:45309) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica da3b57321031416cba9db75450ef302a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:33.310422  1539 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:33.315970  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:33.306104  9334 log.cc:927] Time spent T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.200s	user 0.007s	sys 0.000s
W20260709 13:04:33.343976  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47516.8us
I20260709 13:04:33.308125  9148 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.311643  9304 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.341230  9312 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:33.309911  9256 raft_consensus.cc:515] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:33.343145  9294 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:33.309911  9263 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.357470  9156 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.357963  9336 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.311300  9338 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:33.420615  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 77439.5us
W20260709 13:04:33.422441  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65592.1us
I20260709 13:04:33.397046  9144 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:33.408025  9312 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:33.391186  9304 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:33.437563  9339 log.cc:927] Time spent T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:04:33.396690  9148 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:33.443403  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:33.444942  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:33.399389  9294 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.406731  9156 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:33.416368  9336 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:33.435492  9312 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.466285  9144 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.525564  9316 raft_consensus.cc:493] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:33.538496  9316 raft_consensus.cc:515] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:33.541936  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 215809us
I20260709 13:04:33.521034  9305 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.554785  9263 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:33.564934  9341 log.cc:927] Time spent T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:04:33.565253  9256 leader_election.cc:290] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:33.543061  9144 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:33.568156  9338 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:33.568858  9144 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:33.568997  9316 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.571893  9312 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.579808  9294 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:33.575042  9148 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.576171  9304 leader_election.cc:290] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.577250  9156 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:33.573722  9336 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:33.640759  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 324576us
I20260709 13:04:33.679677  1648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36598
I20260709 13:04:33.681140  1648 raft_consensus.cc:606] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Received request to transfer leadership
W20260709 13:04:33.683082  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 137272us
I20260709 13:04:33.683573  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.685205  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.680001  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:33.697885  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.698294  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:33.701046  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.701354  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.703480  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.711047  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8513b00817042619ae0bf0c8da58904" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:33.726596  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:33.727885  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.729399  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:33.727923  9304 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.731406  9304 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.734375  9336 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.735869  9336 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.737366  9304 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.739360  9336 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.720402  9156 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.743606  9156 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.744966  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.720503  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.761099  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:33.761183  9034 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:33.765030  9034 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.768604  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.773738  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.776619  1651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36598
I20260709 13:04:33.777359  1651 raft_consensus.cc:606] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Received request to transfer leadership
W20260709 13:04:33.785180  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 72262.6us
I20260709 13:04:33.793908  9156 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.794016  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:33.809330  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.810374  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.825076  9034 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.825711  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.829324  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.831827  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b34490a3844168645e5b940ba01a9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.842924  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86"
dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36598
I20260709 13:04:33.843765  1644 raft_consensus.cc:606] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Received request to transfer leadership
I20260709 13:04:33.854998  9156 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.856020  9156 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.858958  9156 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:33.896502  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 52601.7us
I20260709 13:04:33.916402  9336 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:33.917670  9148 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.925868  9148 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.917749  9336 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:33.939060  9148 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.941812  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.944643  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.955747  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.956804  1647 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 23.
I20260709 13:04:33.957710  9336 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:33.955788  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.970711  9034 raft_consensus.cc:993] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: : Instructing follower da3b57321031416cba9db75450ef302a to start an election
I20260709 13:04:33.971505  9034 raft_consensus.cc:1081] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Signalling peer da3b57321031416cba9db75450ef302a to start an election
I20260709 13:04:33.973218  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.974937  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.976992  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.977705  1722 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 22.
I20260709 13:04:33.984915  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.985708  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.986171  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.992830  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:33.994773  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:33.995621  1644 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 23: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:33.997268  2376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86"
dest_uuid: "da3b57321031416cba9db75450ef302a"
 from {username='slave'} at 127.0.0.1:41544
I20260709 13:04:33.997982  2376 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:33.998368  2376 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.013882  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.018395  9156 raft_consensus.cc:493] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.019285  9156 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.022486  9156 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:34.023937  1539 leader_election.cc:304] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.028417  2376 raft_consensus.cc:515] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.038344  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:34.028234  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 119) took 72202 ms (client timeout 15000 ms). Trace:
I20260709 13:04:34.032232  9156 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.037243  9336 raft_consensus.cc:2749] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:34.045061  1467 rpcz_store.cc:269] 0709 13:03:21.950205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:22.000842 (+ 50637us) service_pool.cc:224] Handling call
0709 13:04:34.027883 (+72027041us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62952,"reported_tablets":24,"spinlock_wait_cycles":2816}
I20260709 13:04:34.073055  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:34.073796  1644 raft_consensus.cc:3055] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:34.074100  1644 raft_consensus.cc:740] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:34.075202  1644 consensus_queue.cc:260] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.076195  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.078852  9156 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.086140  2376 leader_election.cc:290] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.089778  1644 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.067456  9148 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:34.095762  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:34.096958  1721 raft_consensus.cc:3060] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:34.102846  9336 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.118104  9156 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:34.119832  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.121403  1539 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.122371  9336 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:34.122987  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.123205  9336 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:34.124008  9148 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.136320  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 62836cb7c4a548b5a6c44c89782a958c with delete_type TABLET_DATA_TOMBSTONED (TS 58cdb682fbf54792a702a7a455f265a2 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:58764
I20260709 13:04:34.137740  1721 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
I20260709 13:04:34.147874  1644 raft_consensus.cc:2468] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 16.
W20260709 13:04:34.144683  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:34.148763  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 70178.8us
W20260709 13:04:34.149981  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:34.150300  1468 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 209) took 19287 ms (client timeout 15000 ms). Trace:
W20260709 13:04:34.151049  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.153326  1466 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.157285  2342 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:34.158555  9287 raft_consensus.cc:2804] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:34.151086  1468 rpcz_store.cc:269] 0709 13:04:14.862473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.618768 (+9756295us) service_pool.cc:224] Handling call
0709 13:04:34.149963 (+9531195us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:32.429946 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:32.430650 (+   704us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:32.430691 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:04:32.430712 (+    21us) tablet.cc:662] Decoding operations
0709 13:04:32.681911 (+251199us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:32.682011 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:32.682045 (+    34us) tablet.cc:685] Acquiring locks for 35 operations
0709 13:04:32.689291 (+  7246us) tablet.cc:701] Row locks acquired
0709 13:04:32.689329 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:04:32.689542 (+   213us) write_op.cc:270] Start()
0709 13:04:32.689689 (+   147us) write_op.cc:276] Timestamp: P: 1783602272689472 usec, L: 0
0709 13:04:32.689717 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:04:32.691581 (+  1864us) log.cc:844] Serialized 13767 byte log entry
0709 13:04:32.919905 (+228324us) op_driver.cc:464] REPLICATION: finished
0709 13:04:33.047926 (+128021us) write_op.cc:301] APPLY: starting
0709 13:04:33.048032 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:33.232448 (+184416us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:33.284504 (+ 52056us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:33.578323 (+293819us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:33.578839 (+   516us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=0,mrs_lookups=0
0709 13:04:33.578902 (+    63us) write_op.cc:312] APPLY: finished
0709 13:04:33.579571 (+   669us) log.cc:844] Serialized 300 byte log entry
0709 13:04:33.580668 (+  1097us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:33.580982 (+   314us) write_op.cc:454] Released schema lock
0709 13:04:33.581081 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:33.581583 (+   502us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:33.581615 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34263,"reported_tablets":52,"spinlock_wait_cycles":2304,"child_traces":[["op",{"apply.queue_time_us":127693,"apply.run_cpu_time_us":86752,"apply.run_wall_time_us":533603,"cfile_cache_hit":142,"cfile_cache_hit_bytes":150637,"num_ops":35,"prepare.queue_time_us":111445,"prepare.run_cpu_time_us":40542,"prepare.run_wall_time_us":314822,"raft.queue_time_us":83526,"raft.run_cpu_time_us":18472,"raft.run_wall_time_us":115992,"replication_time_us":229865,"thread_start_us":264654,"threads_started":4,"wal-append.queue_time_us":84870}]]}
W20260709 13:04:34.160435  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:34.161317  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.162443  9196 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:34.178488  1522 auto_rebalancer.cc:893] Removing replace marker failed: Replica cd6d377efe514bc58ae49c3f4a107232 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:34.179883  1522 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 996f59fd8f2340a49403b3e9ef4c8e86 already in progress
W20260709 13:04:34.180194  1522 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 996f59fd8f2340a49403b3e9ef4c8e86 already in progress
W20260709 13:04:34.180392  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 87637.2us
W20260709 13:04:34.188845  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 65935.9us
I20260709 13:04:34.190608  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.191877  9156 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.178695  9287 raft_consensus.cc:697] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:34.192749  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.191040  1466 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.193946  9156 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:34.194712  1466 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 16 to 20, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 20 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.197055  1466 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 16 to 19, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.199054  1466 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 20 to 21, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 21 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.193976  9287 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.201339  9357 tablet_replica.cc:333] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:04:34.194586  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.227788  9156 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.228197  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.233963  1466 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 17 to 19, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 19 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.217196  9196 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:34.244028  9156 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:34.264453  9357 raft_consensus.cc:2243] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:34.262006  1616 leader_election.cc:336] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:34.268826  1616 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:34.256531  1466 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 18 to 19, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 19 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.273156  1539 leader_election.cc:304] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.265630  9357 pending_rounds.cc:70] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Trying to abort 2 pending ops.
W20260709 13:04:34.272377  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.277920  1466 catalog_manager.cc:5697] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 16 to 17, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.277525  9156 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:34.279867  9034 raft_consensus.cc:2749] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:34.285774  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.276914  9357 pending_rounds.cc:77] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Aborting op as it isn't in flight: id { term: 15 index: 11 } timestamp: 7305634685454082048 op_type: NO_OP noop_request { }
I20260709 13:04:34.289944  9357 raft_consensus.cc:2889] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:04:34.292789   431 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 1601)
I20260709 13:04:34.294206  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
W20260709 13:04:34.293854  1616 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:34.295696  1466 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 14 to 15, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 15 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:34.303985  9156 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:34.304489  9156 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.304798  9156 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:34.304994  1466 catalog_manager.cc:5697] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 18 to 20, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 20 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:34.306730  1466 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 21 to 22, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 22 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.309517  1466 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 15 to 17, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 17 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.312127  1466 catalog_manager.cc:5697] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.313283  9156 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.314049  1466 catalog_manager.cc:5697] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.315819  9156 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.315872  1466 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 17 to 18, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 18 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.290642  9357 pending_rounds.cc:77] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Aborting op as it isn't in flight: id { term: 14 index: 10 } timestamp: 7305634670144184320 op_type: NO_OP noop_request { }
I20260709 13:04:34.317932  9357 raft_consensus.cc:2889] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:34.318482  9357 raft_consensus.cc:2272] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:34.318334  1466 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 15 to 17, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.320279  1466 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 16 to 20, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 20 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.322366  1466 catalog_manager.cc:5697] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.325878  1466 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e reported cstate change: term changed from 20 to 21, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:34.300577  1616 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:34.335806  9034 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:34.344538  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.347858  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:34.345674  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.350186  9357 ts_tablet_manager.cc:1916] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:04:34.354730  1690 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.369098  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.375994  1647 raft_consensus.cc:2393] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f737473248e4c059fa74af3b8e1f77e in current term 16: Already voted for candidate da3b57321031416cba9db75450ef302a in this term.
I20260709 13:04:34.381912  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:34.398172  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "996f59fd8f2340a49403b3e9ef4c8e86" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:34.403573  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:34.404031  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.404961  1647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:34.408802  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:34.409338  2377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c55c47030a294c17ad5561abe541b993" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:34.409502  1721 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:34.410118  1539 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.413789  9148 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.414137  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.409672  9156 raft_consensus.cc:493] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.409637  1647 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:34.416092  9156 raft_consensus.cc:515] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:34.426275  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.433385  9148 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:34.433029  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.440650  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.443224  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.447786  9336 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:34.444691  9148 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:34.464005  9352 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:34.466938  9336 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:34.467463  9352 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.479166  9148 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:34.480494  9148 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.454077  9156 leader_election.cc:290] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.508627  9352 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:34.517647  9148 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.526345  9357 ts_tablet_manager.cc:1929] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
W20260709 13:04:34.530054  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 86918.9us
I20260709 13:04:34.531179  1721 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 24.
I20260709 13:04:34.534802  9357 log.cc:1199] T 62836cb7c4a548b5a6c44c89782a958c P 58cdb682fbf54792a702a7a455f265a2: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-2-root/wals/62836cb7c4a548b5a6c44c89782a958c
I20260709 13:04:34.535151  9156 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.540827  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:34.553371  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.553666  9336 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.554457  9336 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.556120  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.556761  1539 leader_election.cc:304] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.557891  1647 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f737473248e4c059fa74af3b8e1f77e in term 24.
I20260709 13:04:34.558297  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.558854  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:34.535996  9156 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.571727  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20085fd5893f43378bf1c59b14c351c0" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:34.573578  9362 raft_consensus.cc:2749] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:34.600701  1690 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:34.607719  1690 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.628892  9029 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:34.627976  9362 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.651368  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 94690us
I20260709 13:04:34.653965  1539 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.650084  9336 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.654002  9362 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.674669  9362 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.676282  9336 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:34.668243  9148 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.687532  1539 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:34.692270  9336 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:34.693061  9336 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 24 LEADER]: Becoming Leader. State: Replica: 4f737473248e4c059fa74af3b8e1f77e, State: Running, Role: LEADER
I20260709 13:04:34.694581  9336 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.703990  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.710660  9362 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.711369  9362 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.719233  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.715902  1539 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.690045  9156 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.725679  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.729285  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.738080  9362 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:34.744578  9156 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:34.794817  9336 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.740694  9148 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.805929  1539 leader_election.cc:304] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:34.808759  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 62836cb7c4a548b5a6c44c89782a958c (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
W20260709 13:04:34.827061  1458 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 73974.6us
I20260709 13:04:34.827883  9336 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.810590  9156 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.832090  9148 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.836881  9156 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.822374  9362 raft_consensus.cc:2749] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:34.839811  9156 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:34.847330  9156 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.848685  9156 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.851605  9156 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:34.834439  9364 raft_consensus.cc:493] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.853338  9364 raft_consensus.cc:515] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:34.831686  9336 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:34.856150  9364 leader_election.cc:290] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:34.864183  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.866739  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:34.868216  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:34.869002  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.871073  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.876220  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.880545  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:34.891845  2376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:34.894309  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba20622bf78343c5bf6ac194af441026" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:34.894311  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3646214b1a14514955e904e05fadcf1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:34.956034  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45452.9us
I20260709 13:04:34.962190  1539 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.960317  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d12cc2bc7b4654acd41e6c45eb9753" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:34.964241  1539 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:34.961695  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:34.971253  2344 leader_election.cc:336] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:34.970839  9364 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:34.986428  9156 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.987733  9156 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:34.979172  9336 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:34.992822  1539 leader_election.cc:304] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:34.994499  9336 raft_consensus.cc:2749] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:34.979555  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.999580  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:35.002204  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:35.013155  9156 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:35.008270  9336 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.017715  9336 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:35.020735  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.025027  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.035677  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cb1d5977df435cac936d86c5bd1880" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.035568  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.040931  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.047361  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c9dfcb3286481dbbb396ccf74ada24" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.050453  9336 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:35.058157  1722 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:35.059001  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.059756  1649 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:04:35.062598  9368 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:35.089042  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:35.091517  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.098013  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51898.4us
I20260709 13:04:35.115835  9368 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:04:35.107247  1649 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:35.113008  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.124223  1539 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.126642  1539 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.128507  9362 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:35.130386  9362 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:35.133785  1539 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:35.135442  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.143738  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.151561  9362 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:35.156848  9370 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:35.157022  9364 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:35.161881  9336 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:35.160300  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:35.156394  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:35.179774  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.189404  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.190470  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:35.190533  9364 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:35.200232  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.202423  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:35.203730  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:35.205992  1646 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:04:35.212704  9364 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:35.212978  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.216197  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:35.218907  9364 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.219679  9364 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:35.224501  2372 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:04:35.242373  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 96302.7us
W20260709 13:04:35.247891  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:35.261884  9364 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:35.261915  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:35.264581  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8efe0ac5e5e45d8acfa1dfe55172859" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.269137  9383 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:04:35.281677  1716 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Refusing update from remote peer 4f737473248e4c059fa74af3b8e1f77e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:04:35.285454  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb755eb990bc40778167e79183a40b84" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:35.293547   431 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 1601)
I20260709 13:04:35.295125  1539 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.296923  9364 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:35.303725  1539 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:35.305085  9156 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:35.303262  9364 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:04:35.312960  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 67825.7us
W20260709 13:04:35.318462  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:35.336272  1716 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:35.348304  9383 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:35.400815  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.410135  1616 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 280 similar messages]
W20260709 13:04:35.496248  9390 log.cc:927] Time spent T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:04:35.528108  8848 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 3 to 10, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:35.532078  9370 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:35.560148  9370 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:35.659879  1466 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 224) took 9525 ms. Trace:
W20260709 13:04:35.670425  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 75033.6us
I20260709 13:04:35.681002  9362 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.680191  1722 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 3 to 10, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:35.681808  9362 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:35.692672  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:35.663157  1466 rpcz_store.cc:276] 0709 13:04:26.134594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.038854 (+3904260us) service_pool.cc:224] Handling call
0709 13:04:35.659743 (+5620889us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:34.398167 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:34.399005 (+   838us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:34.399061 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:04:34.399083 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:34.417748 (+ 18665us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:34.417870 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:34.417899 (+    29us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:04:34.419617 (+  1718us) tablet.cc:701] Row locks acquired
0709 13:04:34.419666 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:04:34.419882 (+   216us) write_op.cc:270] Start()
0709 13:04:34.420056 (+   174us) write_op.cc:276] Timestamp: P: 1783602274419809 usec, L: 0
0709 13:04:34.420074 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:04:34.457090 (+ 37016us) log.cc:844] Serialized 7507 byte log entry
0709 13:04:34.692323 (+235233us) op_driver.cc:464] REPLICATION: finished
0709 13:04:34.750373 (+ 58050us) write_op.cc:301] APPLY: starting
0709 13:04:34.750556 (+   183us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:34.896124 (+145568us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:34.956980 (+ 60856us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:35.162978 (+205998us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:35.163569 (+   591us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 13:04:35.163638 (+    69us) write_op.cc:312] APPLY: finished
0709 13:04:35.189969 (+ 26331us) log.cc:844] Serialized 172 byte log entry
0709 13:04:35.191087 (+  1118us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:35.191362 (+   275us) write_op.cc:454] Released schema lock
0709 13:04:35.191455 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:35.192066 (+   611us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:35.192098 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28774,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":57529,"apply.run_cpu_time_us":40320,"apply.run_wall_time_us":441699,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83661,"num_ops":19,"prepare.queue_time_us":44756,"prepare.run_cpu_time_us":7632,"prepare.run_wall_time_us":98741,"replication_time_us":270639,"thread_start_us":83435,"threads_started":2}]]}
W20260709 13:04:35.704284  9389 log.cc:927] Time spent T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 13:04:35.731981  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 60650.6us
I20260709 13:04:35.674754  9364 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:35.712299  2380 raft_consensus.cc:2955] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 3 to 10, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:35.740202  9364 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:35.743302  9364 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:35.746479  9156 raft_consensus.cc:493] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.747284  9156 raft_consensus.cc:515] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:35.754565  9392 log.cc:927] Time spent T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:04:35.788167  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 59787.4us
W20260709 13:04:35.790608  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:35.792707  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 84407.1us
I20260709 13:04:35.780251  9362 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:35.803336  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.806262  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:35.814544  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 245790us
I20260709 13:04:35.825527  9156 leader_election.cc:290] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:35.836864  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.838361  1539 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:35.820165  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:35.846208  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30619dafd05d4e36a03ff8f500c3a38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:35.851471  9156 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:35.896521  1646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:35.898152  1539 leader_election.cc:304] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:35.908563  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 54632us
I20260709 13:04:35.895570  9362 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.920529  9403 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:35.921727  9370 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:35.932773  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:35.936422  1539 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.937574  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:35.952842   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:35.953949  9406 raft_consensus.cc:2749] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:35.957652  1539 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:35.954713  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1acf73e529394004842219637cff7cfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:04:35.942428  9362 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:35.966462  9362 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:35.967590  9362 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:35.955664  9403 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:35.949409  9364 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:35.975975  9403 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:35.981468   431 tablet_replica.cc:333] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:35.983476   431 raft_consensus.cc:2243] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:35.990017  9407 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.990883  9407 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:35.995971  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.003422  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.005174  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.010546  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.003674  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac0a563dc584b4fbe82f01f362fa966" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.013790  9407 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.015409   431 raft_consensus.cc:2272] T 5fa278fc63cf4f6e8ca5de6992849e6e P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.019910   431 tablet_replica.cc:333] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:36.029315  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:36.030277  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.032773  9362 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.033895  9362 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:36.037632  9403 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.038435  9403 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:36.039016  9362 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:36.035879  9370 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.040199  9370 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.041247  9403 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:36.042754  9370 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.036090  9407 raft_consensus.cc:493] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.043860  9407 raft_consensus.cc:515] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:36.044574  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 59804.8us
I20260709 13:04:36.046653  9407 leader_election.cc:290] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:36.067927  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.076931  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.079174  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.079481  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.087018  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.094063  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4391f9a46a9542a7bf726b05393b7609" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.095592  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.098654  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.097543  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c501b3d8f86404e991cec1222bd0825" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.116540  9417 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.117393  9417 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.122007  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c1afc8c26047b0b3d8c006f9df5a30" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.122035  9403 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.123533  9403 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.087095  9407 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.137004  1539 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:36.140228  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 225419us
I20260709 13:04:36.143405   431 raft_consensus.cc:2243] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.138098  9417 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.153228  9403 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.153692  9407 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.162978  9336 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:36.166029  1539 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.174961   431 pending_rounds.cc:70] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:36.175586   431 pending_rounds.cc:77] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 17 index: 11 } timestamp: 7305634735097442304 op_type: NO_OP noop_request { }
I20260709 13:04:36.177465   431 raft_consensus.cc:2889] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:36.178102   431 raft_consensus.cc:2272] T 93ef45e97cf24863a576b705ec8e07bd P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.179138  9407 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.173812  9403 raft_consensus.cc:2749] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:36.186573  1469 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 220) took 93345 ms (client timeout 15000 ms). Trace:
W20260709 13:04:36.187217  1469 rpcz_store.cc:269] 0709 13:03:02.841323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.272394 (+431071us) service_pool.cc:224] Handling call
0709 13:04:36.186263 (+92913869us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":53,"spinlock_wait_cycles":1488512}
I20260709 13:04:36.192387  1539 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.197417  1539 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
W20260709 13:04:36.197692  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:36.199579  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:36.201565  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.204624  9336 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:36.205850  1539 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:36.203094  9407 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:36.218504  9407 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:36.219631   431 tablet_replica.cc:333] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.223060  1539 leader_election.cc:304] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:36.207582  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.225178  1716 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:04:36.234510  9417 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.248847   431 raft_consensus.cc:2243] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.248593  9417 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.250059   431 raft_consensus.cc:2272] T fb755eb990bc40778167e79183a40b84 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.205744  9403 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:36.262257  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.246145  9336 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.262156  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:36.245129  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:36.266371  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.278331  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.293426  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c47fb1f3610c496ea3c5542634768fa6" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.269220  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:36.301666  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 56582.4us
I20260709 13:04:36.267431  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.306901  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.286888   431 tablet_replica.cc:333] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.277868  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.265650  9403 raft_consensus.cc:2749] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:36.272720  9336 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.281540  9417 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.319651   431 raft_consensus.cc:2243] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:36.324760  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.312475  9407 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:36.328091  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.336314   431 raft_consensus.cc:2272] T f9396e14142a4f6cb8733d8b08ac73b9 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.343792   431 tablet_replica.cc:333] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.344129  9336 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.345172  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.347745   431 raft_consensus.cc:2243] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.349071   431 raft_consensus.cc:2272] T da3ba11d664442d7b28bc849c61fdcdf P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.350646  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.364506  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bd1d3ea2c984a999cdd42423ebec16f" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.367925  9336 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:36.369118  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.374403   431 tablet_replica.cc:333] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.393383   431 raft_consensus.cc:2243] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.367467  1539 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.395870  9407 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:36.396235   431 raft_consensus.cc:2272] T f3646214b1a14514955e904e05fadcf1 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.400367   431 tablet_replica.cc:333] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.401623   431 raft_consensus.cc:2243] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.402678   431 raft_consensus.cc:2272] T de2b34490a3844168645e5b940ba01a9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:36.405804  1688 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dde38eb3bd9341aa90d41a6f21bb29a1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.408582  1539 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:36.415505  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.418545  1539 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.419400  9417 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:36.419834  9407 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.423437  9407 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:36.423698  9370 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:36.424582  9403 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.426628  9407 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:36.427109  9403 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:36.421820  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.432291  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.421525   431 tablet_replica.cc:333] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.436728   431 raft_consensus.cc:2243] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:36.437637  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.441857  9403 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:36.431345  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.446199  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.449517  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:36.451815  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.458720   431 raft_consensus.cc:2272] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.459213  1649 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:04:36.460086  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81bce6c3bf1547e4b8d33586cafe5f6c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.463507  1466 catalog_manager.cc:5697] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.460871  1539 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.467610  1466 catalog_manager.cc:5697] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 21 to 22, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.470084  1466 catalog_manager.cc:5697] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 18 to 19, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.473528  1539 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:36.475618  1466 catalog_manager.cc:5697] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.473450  9403 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:36.479560  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.477480   431 tablet_replica.cc:333] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.479110  9275 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:04:36.477941  9370 raft_consensus.cc:2749] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:36.491146   431 raft_consensus.cc:2243] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.494138  9407 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:36.494518  1466 catalog_manager.cc:5697] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.495442  9407 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:36.503647  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.506117   431 raft_consensus.cc:2272] T b8513b00817042619ae0bf0c8da58904 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.506690  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:36.511515   431 tablet_replica.cc:333] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.500937  9407 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:36.513080  1539 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:36.514739  9370 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:36.519740  1466 catalog_manager.cc:5697] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.521757  1466 catalog_manager.cc:5697] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 20, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.523521  1466 catalog_manager.cc:5697] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.532619   431 raft_consensus.cc:2243] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.533656   431 pending_rounds.cc:70] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:36.534185   431 pending_rounds.cc:77] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305634831384596480 op_type: NO_OP noop_request { }
I20260709 13:04:36.534686   431 raft_consensus.cc:2889] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:36.540738  9407 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.552861  9407 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.547187  9371 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:04:36.552757  1643 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:04:36.555685  9407 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.532429  1466 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.557492  1466 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 16 to 18, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.552698   431 raft_consensus.cc:2272] T c47fb1f3610c496ea3c5542634768fa6 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.572078   431 tablet_replica.cc:333] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.575304  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.579217  9407 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.574769  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 62836cb7c4a548b5a6c44c89782a958c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:58764
I20260709 13:04:36.580928  9407 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:36.582422   431 raft_consensus.cc:2243] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.591172   431 raft_consensus.cc:2272] T d7e1643d2c2f4a96a57a8bae29e7f19a P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.576265  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d22dc39228b4001961709e74ef6bc9c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.591367  9407 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.559955  1466 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 22, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.593533  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.598395  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.599450  9370 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.574735  1470 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 207) took 137524 ms (client timeout 15000 ms). Trace:
I20260709 13:04:36.604276  9370 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:36.604730  1470 rpcz_store.cc:269] 0709 13:02:19.050620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.050929 (+   309us) service_pool.cc:224] Handling call
0709 13:04:36.574617 (+137523688us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":280323,"reported_tablets":52}
I20260709 13:04:36.607576  9370 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:36.610848   431 tablet_replica.cc:333] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.619411   431 raft_consensus.cc:2243] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.616505  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:36.642333  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 62836cb7c4a548b5a6c44c89782a958c (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:04:36.651805   431 pending_rounds.cc:70] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:36.652302   431 pending_rounds.cc:77] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305634847194009600 op_type: NO_OP noop_request { }
I20260709 13:04:36.653220   431 raft_consensus.cc:2889] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:36.653776   431 raft_consensus.cc:2272] T 30619dafd05d4e36a03ff8f500c3a38c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.658645  1723 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
W20260709 13:04:36.659375  1690 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.623387  1539 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.625277  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:36.661619  9370 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:36.613567  1466 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.664393  1466 catalog_manager.cc:5697] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 17, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.666702  1539 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.667196  1466 catalog_manager.cc:5697] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 20, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.669836  1466 catalog_manager.cc:5697] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 20, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.672760  9370 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:36.688995  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer da3b57321031416cba9db75450ef302a (127.0.107.196:45309): Couldn't send request to peer da3b57321031416cba9db75450ef302a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 740ac262ab184918992607d66446b8f3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.690587   431 tablet_replica.cc:333] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.692039   431 raft_consensus.cc:2243] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:36.697395  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.711447   431 raft_consensus.cc:2272] T 1acf73e529394004842219637cff7cfc P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:36.711925  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:36.714181  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:36.715022  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.715881   431 tablet_replica.cc:333] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:36.698216  1539 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:36.709252  1651 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:36.672096  1466 catalog_manager.cc:5697] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 6 to 23, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:04:36.719710  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.721148  9370 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:36.720136  1466 catalog_manager.cc:5697] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 21, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.722774  1466 catalog_manager.cc:5697] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.725471  9291 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:36.728685  1466 catalog_manager.cc:5697] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.734993  1466 catalog_manager.cc:5697] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.737186  1466 catalog_manager.cc:5697] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 19 to 20, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.739046  1466 catalog_manager.cc:5697] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 17, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.741006  1466 catalog_manager.cc:5697] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 19, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 19 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:36.743049  1466 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:04:36.745712  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.749936  1466 catalog_manager.cc:5697] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 21, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.753865  1466 catalog_manager.cc:5697] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 16 to 21, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.766803  1466 catalog_manager.cc:5697] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:36.771171   431 raft_consensus.cc:2243] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.770399  1466 catalog_manager.cc:5697] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 21 to 22, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.773466  1466 catalog_manager.cc:5697] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:36.782869  1466 catalog_manager.cc:5697] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.840708  9249 raft_consensus.cc:2955] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:04:36.903772   431 raft_consensus.cc:2272] T 20085fd5893f43378bf1c59b14c351c0 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.916285  1651 raft_consensus.cc:2955] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:04:36.937693   431 tablet_replica.cc:333] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:36.954154  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:36.956604   431 raft_consensus.cc:2243] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:36.957558   431 raft_consensus.cc:2272] T 996f59fd8f2340a49403b3e9ef4c8e86 P 4f737473248e4c059fa74af3b8e1f77e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:36.980067  1466 catalog_manager.cc:5697] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 18 to 23, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.985590  1466 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:37.012310  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:37.048591  1522 auto_rebalancer.cc:629] Failed to schedule move for tablet 7011d2943a2040ccb1a34122c0fd3ac3: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:37.059135  1522 auto_rebalancer.cc:629] Failed to schedule move for tablet e8efe0ac5e5e45d8acfa1dfe55172859: Illegal state: Replica 58cdb682fbf54792a702a7a455f265a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:37.061079   431 tablet_replica.cc:333] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.062588   431 raft_consensus.cc:2243] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.063671   431 pending_rounds.cc:70] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 2 pending ops.
W20260709 13:04:37.093273  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.098320  9425 ts_tablet_manager.cc:933] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.064015   431 pending_rounds.cc:77] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305634892110659584 op_type: NO_OP noop_request { }
I20260709 13:04:37.144870   431 raft_consensus.cc:2889] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:04:37.156492  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.145445   431 pending_rounds.cc:77] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 18 index: 10 } timestamp: 7305634842362945536 op_type: NO_OP noop_request { }
I20260709 13:04:37.157359   431 raft_consensus.cc:2889] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:37.157800   431 raft_consensus.cc:2272] T 8014e1ef10d84a9f9c4d676b56c5105d P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.164103  9425 tablet_copy_client.cc:323] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:04:37.176883   431 tablet_replica.cc:333] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.178179   431 raft_consensus.cc:2243] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:04:37.253846  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet dde38eb3bd9341aa90d41a6f21bb29a1 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:04:37.254704  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet dde38eb3bd9341aa90d41a6f21bb29a1 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-dde38eb3bd9341aa90d41a6f21bb29a1
I20260709 13:04:37.264396  1735 tablet_copy_source_session.cc:215] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:37.272447  9407 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.273200  9407 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:37.278115  9425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde38eb3bd9341aa90d41a6f21bb29a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:37.280462  9407 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:37.285096  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:37.326202   431 raft_consensus.cc:2272] T 1be748f6ba29434fbee8b14cedc1d192 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.336035  9409 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:37.336884  9409 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.341142  9409 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.336143  9425 tablet_copy_client.cc:806] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:37.348538  9425 tablet_copy_client.cc:670] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:37.373339  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:37.377385  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:37.378095   431 tablet_replica.cc:333] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.379251   431 raft_consensus.cc:2243] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.380252   431 raft_consensus.cc:2272] T 78d12cc2bc7b4654acd41e6c45eb9753 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.407085   431 tablet_replica.cc:333] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.408278   431 raft_consensus.cc:2243] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.424633  1539 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:37.426123  9407 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:37.438704  1458 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 996f59fd8f2340a49403b3e9ef4c8e86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd6d377efe514bc58ae49c3f4a107232 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:37.453413   431 raft_consensus.cc:2272] T d5ac82307bd14706ab03f881a279f3e5 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:37.459349  1616 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:37.460860   431 tablet_replica.cc:333] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.467036  9436 ts_tablet_manager.cc:933] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: Initiating tablet copy from peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.468609  9425 tablet_copy_client.cc:538] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:04:37.470419  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:37.460070  1616 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:37.461934   431 raft_consensus.cc:2243] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.509438   431 raft_consensus.cc:2272] T 38bef56447d743db91bd1f59274db995 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.497287  9409 raft_consensus.cc:2749] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:37.518873  9375 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:37.519642  9375 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:37.522636  1458 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 1acf73e529394004842219637cff7cfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 10
W20260709 13:04:37.529099  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.538105  1466 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 210) took 5712 ms. Trace:
I20260709 13:04:37.542198  1466 rpcz_store.cc:276] 0709 13:04:31.823036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.779916 (+3956880us) service_pool.cc:224] Handling call
0709 13:04:37.534903 (+1754987us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:37.087092 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:37.088157 (+  1065us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:37.088227 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:04:37.088252 (+    25us) tablet.cc:662] Decoding operations
0709 13:04:37.091474 (+  3222us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:37.146955 (+ 55481us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:37.158757 (+ 11802us) tablet.cc:685] Acquiring locks for 32 operations
0709 13:04:37.165445 (+  6688us) tablet.cc:701] Row locks acquired
0709 13:04:37.165485 (+    40us) write_op.cc:260] PREPARE: finished
0709 13:04:37.165707 (+   222us) write_op.cc:270] Start()
0709 13:04:37.165903 (+   196us) write_op.cc:276] Timestamp: P: 1783602277165629 usec, L: 0
0709 13:04:37.165951 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:04:37.170086 (+  4135us) log.cc:844] Serialized 12569 byte log entry
0709 13:04:37.180267 (+ 10181us) op_driver.cc:464] REPLICATION: finished
0709 13:04:37.182199 (+  1932us) write_op.cc:301] APPLY: starting
0709 13:04:37.182315 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:37.216670 (+ 34355us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:37.216851 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:37.271713 (+ 54862us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:37.272221 (+   508us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=0,mrs_lookups=0
0709 13:04:37.272273 (+    52us) write_op.cc:312] APPLY: finished
0709 13:04:37.273113 (+   840us) log.cc:844] Serialized 276 byte log entry
0709 13:04:37.274187 (+  1074us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:37.274493 (+   306us) write_op.cc:454] Released schema lock
0709 13:04:37.274579 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:37.387321 (+112742us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d69e28 00007fc8377a4581 00007fc8377a3d58 00007fc8377a89f1 00007fc8377a89a9 00007fc8377a8939 00007fc8377a8901 00007fc8377a7b2d 000000000038f354 000000000038f199
0709 13:04:37.387573 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:37.387602 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":120421,"reported_tablets":52,"spinlock_wait_cycles":2944,"child_traces":[["op",{"apply.queue_time_us":1301,"apply.run_cpu_time_us":52498,"apply.run_wall_time_us":92872,"cfile_cache_hit":130,"cfile_cache_hit_bytes":138079,"num_ops":32,"prepare.queue_time_us":14463,"prepare.run_cpu_time_us":16540,"prepare.run_wall_time_us":300468,"raft.queue_time_us":3517,"raft.run_cpu_time_us":1674,"raft.run_wall_time_us":1674,"replication_time_us":14101,"spinlock_wait_cycles":5912704,"thread_start_us":32693,"threads_started":4,"wal-append.queue_time_us":2440}]]}
I20260709 13:04:37.555797   431 tablet_replica.cc:333] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.570389  9375 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:37.572129   431 raft_consensus.cc:2243] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.573225   431 pending_rounds.cc:70] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:37.573931   431 pending_rounds.cc:77] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 17 index: 14 } timestamp: 7305634885842038784 op_type: NO_OP noop_request { }
I20260709 13:04:37.574373   431 raft_consensus.cc:2889] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:37.574800   431 raft_consensus.cc:2272] T 08c9dfcb3286481dbbb396ccf74ada24 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:37.580308  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.580452  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:37.581279  1644 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 24.
I20260709 13:04:37.583386  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:37.585561  9436 tablet_copy_client.cc:323] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:44281
I20260709 13:04:37.588534  1735 tablet_copy_service.cc:140] P 58cdb682fbf54792a702a7a455f265a2: Received BeginTabletCopySession request for tablet 740ac262ab184918992607d66446b8f3 from peer da3b57321031416cba9db75450ef302a ({username='slave'} at 127.0.0.1:60218)
I20260709 13:04:37.589651  1735 tablet_copy_service.cc:161] P 58cdb682fbf54792a702a7a455f265a2: Beginning new tablet copy session on tablet 740ac262ab184918992607d66446b8f3 from peer da3b57321031416cba9db75450ef302a at {username='slave'} at 127.0.0.1:60218: session id = da3b57321031416cba9db75450ef302a-740ac262ab184918992607d66446b8f3
I20260709 13:04:37.586652  9375 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:37.604727  9375 raft_consensus.cc:493] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:37.605259  9375 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:37.608996   431 tablet_replica.cc:333] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.612649   431 raft_consensus.cc:2243] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.613811   431 pending_rounds.cc:70] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:37.614153   431 pending_rounds.cc:77] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 21 index: 10 } timestamp: 7305634858249777152 op_type: NO_OP noop_request { }
I20260709 13:04:37.614735   431 raft_consensus.cc:2889] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:37.615163   431 raft_consensus.cc:2272] T 7011d2943a2040ccb1a34122c0fd3ac3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.641705  1735 tablet_copy_source_session.cc:215] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:37.642829   431 tablet_replica.cc:333] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.653409   431 raft_consensus.cc:2243] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.654855   431 raft_consensus.cc:2272] T 26917aa1f55f48918fedd4960446f38c P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:37.656633  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:37.658775  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.661158  9375 raft_consensus.cc:515] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.647223  1465 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.665548  9436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740ac262ab184918992607d66446b8f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:37.645941  2344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 47541.2us
I20260709 13:04:37.665935  9425 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:04:37.675685   431 tablet_replica.cc:333] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:37.695724  1690 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:37.699339   431 raft_consensus.cc:2243] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.708608  9375 leader_election.cc:290] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:37.709038  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa278fc63cf4f6e8ca5de6992849e6e" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:37.709913  1651 raft_consensus.cc:3060] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:37.710490  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:37.711127  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:37.714102  9407 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.714962  9407 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.718964  9407 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.721182  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:37.729089  9407 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.730343  9407 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.733204  9407 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.734498  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:37.739960  1651 raft_consensus.cc:2468] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 25.
I20260709 13:04:37.743611  1688 leader_election.cc:304] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:37.747632  9375 raft_consensus.cc:2804] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:37.748446  9375 raft_consensus.cc:697] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 25 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:37.749732  9375 consensus_queue.cc:237] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:37.751407  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:37.761025  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.719182  1465 catalog_manager.cc:5697] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 15 to 18, leader changed from da3b57321031416cba9db75450ef302a (127.0.107.196) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:37.772038  1690 leader_election.cc:336] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:37.783597   431 raft_consensus.cc:2272] T 740ac262ab184918992607d66446b8f3 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.816722  1465 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:37.818264  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.829873  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:37.835997  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 51465us
I20260709 13:04:37.849943  9407 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.850894  9407 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.851365  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69630519d3c48eca350b04183abca32" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:37.852453  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:37.855214  9407 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.857021  1465 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.858456  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:37.868618  9407 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.870019  9407 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:37.839720   431 tablet_replica.cc:333] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.876197  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:37.882369  1539 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:37.892084  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:37.892470  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f048890019da43548110ab19fce7a878" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
W20260709 13:04:37.895539  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.899744  9407 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:37.876237   431 raft_consensus.cc:2243] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.913242  9407 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:37.917193  9407 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:37.917935  9407 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.922124  1539 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:37.924067  1539 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:37.934504   431 raft_consensus.cc:2272] T fd4d3a0e164248a5be4b2d19407551af P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.938318  9407 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:37.939095  9407 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:37.950518  9436 tablet_copy_client.cc:806] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:37.952597   431 tablet_replica.cc:333] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:37.947335  9407 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:37.953784   431 raft_consensus.cc:2243] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:37.970273   431 pending_rounds.cc:70] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 2 pending ops.
I20260709 13:04:37.979382   431 pending_rounds.cc:77] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 23 index: 12 } timestamp: 7305634847566057472 op_type: NO_OP noop_request { }
I20260709 13:04:37.979965   431 raft_consensus.cc:2889] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:37.980324   431 pending_rounds.cc:77] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305634807058997248 op_type: NO_OP noop_request { }
I20260709 13:04:37.980822   431 raft_consensus.cc:2889] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:37.981264   431 raft_consensus.cc:2272] T 48efc21b583b4e7fa81ec24726402281 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.968700  9407 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.983917  9436 tablet_copy_client.cc:670] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:37.984318  9407 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:37.988911  9407 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:38.016595  9436 tablet_copy_client.cc:538] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:38.020494   431 tablet_replica.cc:333] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.022049   431 raft_consensus.cc:2243] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.023322   431 raft_consensus.cc:2272] T e8efe0ac5e5e45d8acfa1dfe55172859 P 4f737473248e4c059fa74af3b8e1f77e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:37.996851  9407 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.040923  1539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 116915us
I20260709 13:04:38.042447  1539 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:38.046190  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:38.046885  9445 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:38.041661  9444 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:38.048331  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e7f73e79344731aa22e748b700ffd9" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:38.049623  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:38.050860  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:38.049296  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.055177  9444 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:38.058794  9444 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:38.064671   431 tablet_replica.cc:333] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.066020   431 raft_consensus.cc:2243] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.077217  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:38.078351   431 raft_consensus.cc:2272] T 42b29e0e5e884027ab75497ae8e020bf P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.080005  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87baee349cdd48d88642bf37176b1b26" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:38.083770  9436 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: Bootstrap starting.
I20260709 13:04:38.084909  1539 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:38.083513   431 tablet_replica.cc:333] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.043104  9407 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:38.089196  9444 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:38.090163  9407 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:38.091794   431 raft_consensus.cc:2243] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:38.096207  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 43775.9us
I20260709 13:04:38.093222  1539 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:38.096801   431 pending_rounds.cc:70] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 2 pending ops.
I20260709 13:04:38.106659   431 pending_rounds.cc:77] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305634858700083200 op_type: NO_OP noop_request { }
I20260709 13:04:38.110710   431 raft_consensus.cc:2889] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:38.098827  9407 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:38.097723  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.117472   431 pending_rounds.cc:77] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305634817123577856 op_type: NO_OP noop_request { }
I20260709 13:04:38.120373   431 raft_consensus.cc:2889] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:38.123790   431 raft_consensus.cc:2272] T 54db7417c3014ba3948e6ce733254ba3 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.098588  1539 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:38.131453  9407 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:38.139892  9425 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:38.141340  9425 tablet_bootstrap.cc:492] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:04:38.145279  1651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:38.146633  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:38.149334  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.167546  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08002e8620 after lost signal to thread 434
W20260709 13:04:38.169080  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08002efb40 after lost signal to thread 1508
W20260709 13:04:38.169982  1597 debug-util.cc:398] Leaking SignalData structure 0x7b080031f560 after lost signal to thread 1600
W20260709 13:04:38.177480  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08000ade40 after lost signal to thread 1675
W20260709 13:04:38.179296  1597 debug-util.cc:398] Leaking SignalData structure 0x7b080017e300 after lost signal to thread 1749
I20260709 13:04:38.179540   431 tablet_replica.cc:333] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:38.179718  1597 debug-util.cc:398] Leaking SignalData structure 0x7b080017e3e0 after lost signal to thread 1750
W20260709 13:04:38.181617  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08001aa600 after lost signal to thread 2404
W20260709 13:04:38.181988  1597 debug-util.cc:398] Leaking SignalData structure 0x7b08001eaa40 after lost signal to thread 2405
I20260709 13:04:38.148123  9425 ts_tablet_manager.cc:1403] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.483s	user 0.056s	sys 0.032s
I20260709 13:04:38.192951  9425 raft_consensus.cc:359] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:04:38.194376  9425 raft_consensus.cc:740] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:04:38.195142  9425 consensus_queue.cc:260] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:04:38.201846  1539 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:38.207635   431 raft_consensus.cc:2243] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.208642  9425 ts_tablet_manager.cc:1434] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.004s
I20260709 13:04:38.215900  9407 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:38.222503  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-dde38eb3bd9341aa90d41a6f21bb29a1 received from {username='slave'} at 127.0.0.1:60218
I20260709 13:04:38.223197  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-dde38eb3bd9341aa90d41a6f21bb29a1 on tablet dde38eb3bd9341aa90d41a6f21bb29a1 with peer da3b57321031416cba9db75450ef302a
I20260709 13:04:38.227516  1651 raft_consensus.cc:1275] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:38.233006  9249 consensus_queue.cc:1048] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:38.234390   431 raft_consensus.cc:2272] T 9c347beda1704867ba2582a7a34d255b P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.250319  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:38.292685   431 tablet_replica.cc:333] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.293597  9407 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.294346   431 raft_consensus.cc:2243] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.294232  9407 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:38.298521  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:38.300073  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:38.300514  9407 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:38.301752  1539 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:38.310111  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:38.310772  9407 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:38.317271  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.342295   431 raft_consensus.cc:2272] T f048890019da43548110ab19fce7a878 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.356819  9407 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.357529  9407 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:38.359289  9436 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:38.361377  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:38.363754  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5938eec8e53f4d08b26ec3011f1cdbfc" candidate_uuid: "4f737473248e4c059fa74af3b8e1f77e" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:38.366662  1539 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f737473248e4c059fa74af3b8e1f77e; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:38.368379  9444 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:38.368381  9407 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:38.376017  9436 tablet_bootstrap.cc:492] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: Bootstrap complete.
I20260709 13:04:38.382064  9436 ts_tablet_manager.cc:1403] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: Time spent bootstrapping tablet: real 0.299s	user 0.061s	sys 0.010s
I20260709 13:04:38.411872  1466 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 3 to 10, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 15 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.419821   431 tablet_replica.cc:333] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.414314  1466 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 21 to 24, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 24 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:38.421205   431 raft_consensus.cc:2243] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.422394   431 raft_consensus.cc:2272] T ba20622bf78343c5bf6ac194af441026 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.422425  1466 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 18, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 18 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:38.467650   431 tablet_replica.cc:333] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.473783   431 raft_consensus.cc:2243] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.474845   431 raft_consensus.cc:2272] T 27e7f73e79344731aa22e748b700ffd9 P 4f737473248e4c059fa74af3b8e1f77e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.435802  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.507241  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 115958us
W20260709 13:04:38.514055  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.478366  1466 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 13, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:04:38.428211  9436 raft_consensus.cc:359] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:04:38.518500  9436 raft_consensus.cc:740] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: da3b57321031416cba9db75450ef302a, State: Initialized, Role: LEARNER
I20260709 13:04:38.519469  9436 consensus_queue.cc:260] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } }
I20260709 13:04:38.523744  9436 ts_tablet_manager.cc:1434] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: Time spent starting tablet: real 0.141s	user 0.002s	sys 0.008s
I20260709 13:04:38.528209  1735 tablet_copy_service.cc:342] P 58cdb682fbf54792a702a7a455f265a2: Request end of tablet copy session da3b57321031416cba9db75450ef302a-740ac262ab184918992607d66446b8f3 received from {username='slave'} at 127.0.0.1:60218
I20260709 13:04:38.529095  1735 tablet_copy_service.cc:434] P 58cdb682fbf54792a702a7a455f265a2: ending tablet copy session da3b57321031416cba9db75450ef302a-740ac262ab184918992607d66446b8f3 on tablet 740ac262ab184918992607d66446b8f3 with peer da3b57321031416cba9db75450ef302a
W20260709 13:04:38.521149  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.532052  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.533805  1466 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 21 to 24, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193). New cstate: current_term: 24 leader_uuid: "4f737473248e4c059fa74af3b8e1f77e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:38.535643  1466 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index -1 to 11, NON_VOTER da3b57321031416cba9db75450ef302a (127.0.107.196) added. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: true } } }
I20260709 13:04:38.541370  1466 catalog_manager.cc:5697] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 22 to 24, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 24 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.543874  1466 catalog_manager.cc:5697] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 20, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:38.549455  1466 catalog_manager.cc:5697] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 17 to 18, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:38.554318  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 62836cb7c4a548b5a6c44c89782a958c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:58764
I20260709 13:04:38.555830   431 tablet_replica.cc:333] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:38.557898  1465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 212) took 27644 ms (client timeout 15000 ms). Trace:
I20260709 13:04:38.551059  1466 catalog_manager.cc:5697] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 18, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
I20260709 13:04:38.566346  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 62836cb7c4a548b5a6c44c89782a958c (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:04:38.573151  2372 raft_consensus.cc:1217] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 LEARNER]: Deduplicated request from leader. Original: 19.10->[19.11-19.11]   Dedup: 19.11->[]
I20260709 13:04:38.562359  1466 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 15 to 16, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:38.558444  1465 rpcz_store.cc:269] 0709 13:04:10.913379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.455228 (+13541849us) service_pool.cc:224] Handling call
0709 13:04:38.557799 (+14102571us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:38.066244 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:38.067356 (+  1112us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:38.067402 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:04:38.067424 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:38.068185 (+   761us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:38.068287 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:38.068341 (+    54us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:04:38.094070 (+ 25729us) tablet.cc:701] Row locks acquired
0709 13:04:38.094094 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:04:38.094281 (+   187us) write_op.cc:270] Start()
0709 13:04:38.094557 (+   276us) write_op.cc:276] Timestamp: P: 1783602278094224 usec, L: 0
0709 13:04:38.094589 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:04:38.095399 (+   810us) log.cc:844] Serialized 1691 byte log entry
0709 13:04:38.145566 (+ 50167us) op_driver.cc:464] REPLICATION: finished
0709 13:04:38.185889 (+ 40323us) write_op.cc:301] APPLY: starting
0709 13:04:38.185999 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:38.320690 (+134691us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:38.320824 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:38.323745 (+  2921us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:38.324036 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:04:38.324097 (+    61us) write_op.cc:312] APPLY: finished
0709 13:04:38.376485 (+ 52388us) log.cc:844] Serialized 49 byte log entry
0709 13:04:38.377448 (+   963us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:38.377571 (+   123us) write_op.cc:454] Released schema lock
0709 13:04:38.377623 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:38.378186 (+   563us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:38.378220 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":40018,"apply.run_cpu_time_us":9572,"apply.run_wall_time_us":192247,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20871,"num_ops":4,"prepare.queue_time_us":128026,"prepare.run_cpu_time_us":4203,"prepare.run_wall_time_us":29624,"replication_time_us":50735,"thread_start_us":140473,"threads_started":2}]]}
I20260709 13:04:38.596854   431 raft_consensus.cc:2243] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.603098   431 raft_consensus.cc:2272] T 1d22dc39228b4001961709e74ef6bc9c P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.607336   431 tablet_replica.cc:333] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.608301   431 raft_consensus.cc:2243] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.609309   431 raft_consensus.cc:2272] T 6bd1d3ea2c984a999cdd42423ebec16f P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.621490   431 tablet_replica.cc:333] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.622687   431 raft_consensus.cc:2243] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.623698   431 pending_rounds.cc:70] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:38.624010   431 pending_rounds.cc:77] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305634862673039360 op_type: NO_OP noop_request { }
I20260709 13:04:38.624507   431 raft_consensus.cc:2889] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:38.625206   431 raft_consensus.cc:2272] T e69630519d3c48eca350b04183abca32 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.634886  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.640108   431 tablet_replica.cc:333] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:38.641189  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.641400   431 raft_consensus.cc:2243] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:38.642145  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:38.642823  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.642351   431 raft_consensus.cc:2272] T 81bce6c3bf1547e4b8d33586cafe5f6c P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.658890   431 tablet_replica.cc:333] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.659946   431 raft_consensus.cc:2243] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.660771   431 raft_consensus.cc:2272] T 9273554b0f40499d928621ee1bccc9d8 P 4f737473248e4c059fa74af3b8e1f77e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.664767   431 tablet_replica.cc:333] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.665807   431 raft_consensus.cc:2243] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.666548   431 raft_consensus.cc:2272] T 87baee349cdd48d88642bf37176b1b26 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.686622  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:38.694218   431 tablet_replica.cc:333] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:38.694991  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.707342  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.708544  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.716686   431 raft_consensus.cc:2243] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.717818   431 raft_consensus.cc:2272] T 44cb1d5977df435cac936d86c5bd1880 P 4f737473248e4c059fa74af3b8e1f77e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.721658   431 tablet_replica.cc:333] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.722909   431 raft_consensus.cc:2243] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.723765   431 raft_consensus.cc:2272] T 5938eec8e53f4d08b26ec3011f1cdbfc P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.726413  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.753108  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:38.754267  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.769613   431 tablet_replica.cc:333] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.782866   431 raft_consensus.cc:2243] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.783746   431 pending_rounds.cc:70] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:38.784021   431 pending_rounds.cc:77] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 17 index: 12 } timestamp: 7305634858066030592 op_type: NO_OP noop_request { }
I20260709 13:04:38.784545   431 raft_consensus.cc:2889] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:38.784940   431 raft_consensus.cc:2272] T 54bb54b92d594aa7a2d8f8402e20c0ca P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.794479   431 tablet_replica.cc:333] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.821484   431 raft_consensus.cc:2243] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.822800   431 raft_consensus.cc:2272] T 62836cb7c4a548b5a6c44c89782a958c P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.836760   431 tablet_replica.cc:333] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.837792   431 raft_consensus.cc:2243] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.838729   431 pending_rounds.cc:70] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:38.839015   431 pending_rounds.cc:77] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 19 index: 15 } timestamp: 7305634859597688832 op_type: NO_OP noop_request { }
I20260709 13:04:38.841322  1466 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41288 (request call id 211) took 1024 ms. Trace:
I20260709 13:04:38.841826  1466 rpcz_store.cc:276] 0709 13:04:37.817351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.817535 (+   184us) service_pool.cc:224] Handling call
0709 13:04:38.841227 (+1023692us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:38.677970 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:38.678607 (+   637us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:38.678645 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:04:38.678667 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:38.679767 (+  1100us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:38.679862 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:38.679895 (+    33us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:04:38.681175 (+  1280us) tablet.cc:701] Row locks acquired
0709 13:04:38.681198 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:04:38.681377 (+   179us) write_op.cc:270] Start()
0709 13:04:38.681528 (+   151us) write_op.cc:276] Timestamp: P: 1783602278681324 usec, L: 0
0709 13:04:38.681552 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:04:38.691210 (+  9658us) log.cc:844] Serialized 4587 byte log entry
0709 13:04:38.724942 (+ 33732us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd7571 00007fc833d69e28 00007fc8377a4581 00007fc8377a3d58 00007fc8377a89f1 00007fc8377a89a9 00007fc8377a8939 00007fc8377a8901 00007fc8377a7b2d 000000000038f354 000000000038f199
0709 13:04:38.743113 (+ 18171us) op_driver.cc:464] REPLICATION: finished
0709 13:04:38.743929 (+   816us) write_op.cc:301] APPLY: starting
0709 13:04:38.744038 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:38.763676 (+ 19638us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:38.763743 (+    67us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:38.814561 (+ 50818us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:38.814852 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:04:38.814915 (+    63us) write_op.cc:312] APPLY: finished
0709 13:04:38.815295 (+   380us) log.cc:844] Serialized 105 byte log entry
0709 13:04:38.815524 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:38.815687 (+   163us) write_op.cc:454] Released schema lock
0709 13:04:38.815753 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:38.816191 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:38.816231 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66173,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":489,"apply.run_cpu_time_us":20158,"apply.run_wall_time_us":72227,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46046,"num_ops":11,"prepare.queue_time_us":59127,"prepare.run_cpu_time_us":7920,"prepare.run_wall_time_us":47425,"replication_time_us":61369,"spinlock_wait_cycles":8853888,"thread_start_us":55177,"threads_started":1}]]}
I20260709 13:04:38.870170  9249 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:38.871142  9249 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:38.876780  9249 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:38.839419   431 raft_consensus.cc:2889] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:38.877663   431 raft_consensus.cc:2272] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.878309  9407 raft_consensus.cc:670] T 4391f9a46a9542a7bf726b05393b7609 P 4f737473248e4c059fa74af3b8e1f77e: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 13:04:38.883158  9434 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:38.883953  9434 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:38.886359  9434 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:38.891423   431 tablet_replica.cc:333] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.904142  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:38.905285  1716 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 24.
I20260709 13:04:38.905522   431 raft_consensus.cc:2243] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.922746  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:38.923619  1644 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 24.
W20260709 13:04:38.923831  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:38.926651  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:38.928495   431 pending_rounds.cc:70] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:38.928615  9442 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:38.929147   431 pending_rounds.cc:77] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305634859548540928 op_type: NO_OP noop_request { }
I20260709 13:04:38.929307  9442 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:38.929678   431 raft_consensus.cc:2889] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:38.929936  9442 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:38.930189   431 raft_consensus.cc:2272] T 57592deb11fc4830921f01c2a3c32c4a P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.936794  2376 raft_consensus.cc:1217] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 LEARNER]: Deduplicated request from leader. Original: 18.12->[18.13-18.13]   Dedup: 18.13->[]
I20260709 13:04:38.939138  1466 catalog_manager.cc:5697] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 24 to 25, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 25 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } }
W20260709 13:04:38.945761  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.962236   431 tablet_replica.cc:333] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:38.969018   431 raft_consensus.cc:2243] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.970106   431 raft_consensus.cc:2272] T dde38eb3bd9341aa90d41a6f21bb29a1 P 4f737473248e4c059fa74af3b8e1f77e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:38.969004  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:38.974860  9375 raft_consensus.cc:1064] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:04:38.979097   431 tablet_replica.cc:333] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:38.983121  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:38.983558  9375 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 9, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:38.984907   431 raft_consensus.cc:2243] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:38.985724   431 raft_consensus.cc:2272] T 9bb9de7d85a2414bb920f63ec26f8686 P 4f737473248e4c059fa74af3b8e1f77e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:38.996449   431 tablet_replica.cc:333] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
W20260709 13:04:38.998003  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1acf73e529394004842219637cff7cfc: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:04:38.998332   431 raft_consensus.cc:2243] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:38.999927  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:38.999917   431 raft_consensus.cc:2272] T 7778fc5cf1c746c4a4a974883a271ce0 P 4f737473248e4c059fa74af3b8e1f77e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:39.018383   431 tablet_replica.cc:333] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:39.021638  9457 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:39.022220  9457 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:39.022650  9457 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:39.028223  1474 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 120) took 55867 ms (client timeout 15000 ms). Trace:
W20260709 13:04:39.029461  1474 rpcz_store.cc:269] 0709 13:03:43.160461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.871564 (+6711103us) service_pool.cc:224] Handling call
0709 13:04:39.027839 (+49156275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26610,"reported_tablets":24}
I20260709 13:04:39.039021  9442 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:39.004897  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.058928  2372 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:04:39.062206  1690 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:39.019649   431 raft_consensus.cc:2243] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:39.074262   431 pending_rounds.cc:70] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:39.074597   431 pending_rounds.cc:77] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 18 index: 14 } timestamp: 7305634827922722816 op_type: NO_OP noop_request { }
I20260709 13:04:39.075078   431 raft_consensus.cc:2889] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:39.075793   431 raft_consensus.cc:2272] T 01c1afc8c26047b0b3d8c006f9df5a30 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:39.072111  9375 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:39.093313  9457 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:39.098541  1471 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 219) took 119675 ms (client timeout 15000 ms). Trace:
W20260709 13:04:39.099165  1471 rpcz_store.cc:269] 0709 13:02:39.423459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.478894 (+ 55435us) service_pool.cc:224] Handling call
0709 13:04:39.098181 (+119619287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23285,"reported_tablets":53}
W20260709 13:04:39.102664  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:39.103091  1473 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 118) took 100392 ms (client timeout 15000 ms). Trace:
W20260709 13:04:39.103622  1473 rpcz_store.cc:269] 0709 13:02:58.767238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.808502 (+ 41264us) service_pool.cc:224] Handling call
0709 13:04:39.102835 (+100294333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20884,"reported_tablets":24}
I20260709 13:04:39.128492  9457 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:39.132668  1649 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:39.133471  9442 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:39.135263   431 tablet_replica.cc:333] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:39.139663  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:39.144497  1723 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 25: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:39.136183   431 raft_consensus.cc:2243] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:39.153330   431 pending_rounds.cc:70] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:39.153609   431 pending_rounds.cc:77] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 15 index: 14 } timestamp: 7305634828441489408 op_type: NO_OP noop_request { }
I20260709 13:04:39.153944   431 raft_consensus.cc:2889] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:39.154294   431 raft_consensus.cc:2272] T fac0a563dc584b4fbe82f01f362fa966 P 4f737473248e4c059fa74af3b8e1f77e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:39.156898  1649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:39.158962  1649 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 25: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
W20260709 13:04:39.163734  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:39.182397   431 tablet_replica.cc:333] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:39.182250  9375 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:39.183535   431 raft_consensus.cc:2243] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:39.188548   431 raft_consensus.cc:2272] T c55c47030a294c17ad5561abe541b993 P 4f737473248e4c059fa74af3b8e1f77e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:39.195394  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:39.195660  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.196900  9249 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:04:39.205791  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.212328   431 tablet_replica.cc:333] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: stopping tablet replica
I20260709 13:04:39.217453   431 raft_consensus.cc:2243] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:39.218396   431 pending_rounds.cc:70] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Trying to abort 1 pending ops.
I20260709 13:04:39.218746   431 pending_rounds.cc:77] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e: Aborting op as it isn't in flight: id { term: 22 index: 11 } timestamp: 7305634857484333056 op_type: NO_OP noop_request { }
I20260709 13:04:39.219137   431 raft_consensus.cc:2889] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:39.219513   431 raft_consensus.cc:2272] T 2c501b3d8f86404e991cec1222bd0825 P 4f737473248e4c059fa74af3b8e1f77e [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:39.218292  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:39.222193  1616 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:39.224685  9441 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:39.248744  9375 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index 11 to 12, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:39.264292  1468 catalog_manager.cc:5697] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.269287  1651 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:39.297411  2372 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:39.302141  1472 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41284 (request call id 223) took 32414 ms (client timeout 15000 ms). Trace:
W20260709 13:04:39.302847  1472 rpcz_store.cc:269] 0709 13:04:06.887542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.685499 (+8797957us) service_pool.cc:224] Handling call
0709 13:04:16.035366 (+349867us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54003395d4. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833d76f3a 00007fc839b2d1cc 00007fc839c2b37e 00007fc8380ec2a4 00007fc8380ec232 00007fc8380ec161 00007fc8380ec0dc 00007fc8380eb392 00007fc8320a948c 00007fc8320a88c6
0709 13:04:39.301860 (+23266494us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":53,"spinlock_wait_cycles":416657792}
W20260709 13:04:39.303673  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:39.325546  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.311384  1468 catalog_manager.cc:5697] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 17 to 19, leader changed from 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 19 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:39.333699  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.337853  1468 catalog_manager.cc:5697] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.346923  9249 raft_consensus.cc:1064] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: attempting to promote NON_VOTER da3b57321031416cba9db75450ef302a to VOTER
I20260709 13:04:39.348552  1468 catalog_manager.cc:5697] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.350596  9249 consensus_queue.cc:237] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:39.391294  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.403477  1473 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 11 to 12, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } attrs { replace: true } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:39.406054  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.447566  2372 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 LEARNER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:04:39.456954  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.457507  1648 raft_consensus.cc:1275] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:39.459237  9249 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:04:39.462080  9375 consensus_queue.cc:1048] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:04:39.468631  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37380 (request call id 122) took 11564 ms (client timeout 15000 ms). Trace:
W20260709 13:04:39.474051  1467 rpcz_store.cc:269] 0709 13:04:27.904638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.119245 (+6214607us) service_pool.cc:224] Handling call
0709 13:04:39.468540 (+5349295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135,"reported_tablets":24}
W20260709 13:04:39.495814  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.498401  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.499238  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:39.512693  1714 tablet_service.cc:1558] Processing DeleteTablet for tablet 62836cb7c4a548b5a6c44c89782a958c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:58764
I20260709 13:04:39.518806  1468 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41296 (request call id 213) took 10160 ms. Trace:
I20260709 13:04:39.519528  1468 rpcz_store.cc:276] 0709 13:04:29.358822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.172180 (+4813358us) service_pool.cc:224] Handling call
0709 13:04:39.518734 (+5346554us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:39.362434 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:39.363070 (+   636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:39.363116 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:04:39.363132 (+    16us) tablet.cc:662] Decoding operations
0709 13:04:39.363824 (+   692us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:39.363900 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:39.363939 (+    39us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:04:39.364383 (+   444us) tablet.cc:701] Row locks acquired
0709 13:04:39.364398 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:04:39.364591 (+   193us) write_op.cc:270] Start()
0709 13:04:39.364738 (+   147us) write_op.cc:276] Timestamp: P: 1783602279364539 usec, L: 0
0709 13:04:39.364771 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:04:39.365704 (+   933us) log.cc:844] Serialized 1686 byte log entry
0709 13:04:39.381060 (+ 15356us) op_driver.cc:464] REPLICATION: finished
0709 13:04:39.383222 (+  2162us) write_op.cc:301] APPLY: starting
0709 13:04:39.383327 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:39.387013 (+  3686us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:39.387130 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:39.391081 (+  3951us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:39.391322 (+   241us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:04:39.391373 (+    51us) write_op.cc:312] APPLY: finished
0709 13:04:39.391675 (+   302us) log.cc:844] Serialized 49 byte log entry
0709 13:04:39.392669 (+   994us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:39.392802 (+   133us) write_op.cc:454] Released schema lock
0709 13:04:39.392867 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:39.393306 (+   439us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:39.393340 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":40238,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":1096,"apply.run_cpu_time_us":8611,"apply.run_wall_time_us":10060,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16744,"num_ops":4,"prepare.queue_time_us":180,"prepare.run_cpu_time_us":3642,"prepare.run_wall_time_us":15873,"replication_time_us":16093}]]}
I20260709 13:04:39.518754  9372 raft_consensus.cc:2955] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 13 to 14, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:39.556318  1648 raft_consensus.cc:2955] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:39.593679  1458 catalog_manager.cc:5028] TS 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): tablet 62836cb7c4a548b5a6c44c89782a958c (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
I20260709 13:04:39.607652  2372 raft_consensus.cc:2955] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:39.682919  1468 catalog_manager.cc:5697] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:39.724498  1467 catalog_manager.cc:5697] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: config changed from index 13 to 14, da3b57321031416cba9db75450ef302a (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:39.729548  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.791286  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.805181  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.810495  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.888756  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.899279  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:39.963244  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:39.979594  1723 consensus_queue.cc:237] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:39.984818  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.994426  2372 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:39.997164  9457 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:39.997846  1648 raft_consensus.cc:1275] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:40.000013  9275 consensus_queue.cc:1048] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:40.024721  9375 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:40.030275  1644 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:40.038506  2372 raft_consensus.cc:2955] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:40.013702  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:40.091583  9434 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:40.092371  9434 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:40.093806  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:40.095739  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dde38eb3bd9341aa90d41a6f21bb29a1 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:40.098882  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:40.101509  9434 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:40.091033  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:40.120698  1467 catalog_manager.cc:5697] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 12 to 13, VOTER 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) evicted. New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:40.143317  1616 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:40.144081  1616 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:40.145136  9434 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:40.198078  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:40.233294  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.283612  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:40.292739  9457 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:40.293589  9457 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:40.298357  9457 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:40.303088  1690 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:40.299746  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dde38eb3bd9341aa90d41a6f21bb29a1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:40.325910  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:40.329201  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:40.330180  1648 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:40.370981  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:40.372228  9372 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:40.372709  9372 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4f737473248e4c059fa74af3b8e1f77e)
I20260709 13:04:40.373131  9372 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:40.403452  9372 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:40.416922  9372 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:40.418632  1690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111) [suppressed 478 similar messages]
W20260709 13:04:40.422319  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.423074  1690 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:40.426739  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:40.427862  1648 raft_consensus.cc:3060] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:40.442408  1648 raft_consensus.cc:2468] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
I20260709 13:04:40.454331  1688 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:40.455876  9372 raft_consensus.cc:2804] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:40.456590  9372 raft_consensus.cc:697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:40.457674  9372 consensus_queue.cc:237] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:40.516250  1473 catalog_manager.cc:5697] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 18 to 19, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:40.522550  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.578747  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.641414  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:40.665498  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.672839  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:40.762984  9375 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.763577  9375 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:40.766433  9375 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:40.776464  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 52038.7us
W20260709 13:04:40.778167  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:40.785909  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:40.810087  9434 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.810803  9434 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:40.815121  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:40.815110  9434 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:40.827895  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:40.828876  1723 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 25.
I20260709 13:04:40.841626  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232" is_pre_election: true
I20260709 13:04:40.842429  1644 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 25.
I20260709 13:04:40.841077  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:40.845034  9411 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:40.845527  9411 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.845954  9411 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:40.858609  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e
I20260709 13:04:40.859994  9375 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:40.860535  9375 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.860941  9375 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:40.871464  9411 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:40.874799  9411 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:40.876366  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
W20260709 13:04:40.882342  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:40.913868  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:40.915311  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:40.917594  9375 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:40.919056  1723 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd6d377efe514bc58ae49c3f4a107232 in current term 26: Already voted for candidate 58cdb682fbf54792a702a7a455f265a2 in this term.
I20260709 13:04:40.922533  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:40.925010  9411 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:40.951850  9375 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:40.957633  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.958911  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.963186  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:40.963987  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:40.968971  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:40.971232  1648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cd6d377efe514bc58ae49c3f4a107232"
I20260709 13:04:40.972319  1648 raft_consensus.cc:2393] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 58cdb682fbf54792a702a7a455f265a2 in current term 26: Already voted for candidate cd6d377efe514bc58ae49c3f4a107232 in this term.
I20260709 13:04:40.979035  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:40.980522  9375 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:40.999660  1648 raft_consensus.cc:1275] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:41.003165  9372 consensus_queue.cc:1048] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:41.029233  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.046013  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.110148   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:41.123335  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.228384  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.231070  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.262966  1614 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:45309, user_credentials={real_user=slave}} blocked reactor thread for 71446.1us
W20260709 13:04:41.309819  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.329109  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.342236  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.351261  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.355366  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1acf73e529394004842219637cff7cfc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:41.452873  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.620827  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.734977  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.744395  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:41.763994  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.936983  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.983131  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.983891  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.035833  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.086467  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.072624  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.102034  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.110805   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:42.133291  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.146528  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.192679  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.246868  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.312983  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.436195  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.483120  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.569360  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:42.577099  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.611042  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.706372  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 58264.8us
W20260709 13:04:42.709342  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 57219.6us
W20260709 13:04:42.715468  1460 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 112678us
W20260709 13:04:42.718308  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dde38eb3bd9341aa90d41a6f21bb29a1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:42.821714  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.898562  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 50030.2us
W20260709 13:04:42.901253  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.952656  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.002811  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.010809  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.044080  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.050904  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.051658  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.114643  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.120550  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.121526   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:43.202083  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:43.210858  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.338079  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:43.402379  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:43.409857  9491 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.410945  9491 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:43.415576  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:43.416483  1720 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 26.
I20260709 13:04:43.418180  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:43.421672  9490 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:43.422216  9490 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.422598  9490 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:43.433586  9491 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:43.454169  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.455273  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:43.493526  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:43.557142  9490 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:43.576997  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 68274.9us
W20260709 13:04:43.585369  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.601162  1460 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 48464.4us
I20260709 13:04:43.604866  9490 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 27 election: Requested vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:43.608065  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1be748f6ba29434fbee8b14cedc1d192" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:43.608961  1720 raft_consensus.cc:3060] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:43.618788  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:43.651058  1720 raft_consensus.cc:2468] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd6d377efe514bc58ae49c3f4a107232 in term 27.
W20260709 13:04:43.652999  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:43.654114  1614 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232; no voters: 
I20260709 13:04:43.656548  9490 raft_consensus.cc:2804] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:04:43.656999  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:43.657668  9490 raft_consensus.cc:697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 27 LEADER]: Becoming Leader. State: Replica: cd6d377efe514bc58ae49c3f4a107232, State: Running, Role: LEADER
I20260709 13:04:43.659116  9490 consensus_queue.cc:237] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:43.677681  1616 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:43.682123  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.687968  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.702936  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:43.755652  1473 catalog_manager.cc:5697] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 reported cstate change: term changed from 24 to 27, leader changed from 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193) to cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "cd6d377efe514bc58ae49c3f4a107232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:44.128993   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:44.133646  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.147892  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:44.156155  1616 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:44.193787  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.241772  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.242609  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.244628  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.250214  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:44.270239  1720 raft_consensus.cc:1275] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Refusing update from remote peer cd6d377efe514bc58ae49c3f4a107232: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 13:04:44.273893  9442 consensus_queue.cc:1048] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:44.315467  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.396068  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.399969  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.444526  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:44.509668  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.516479  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.526957  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.597992  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.645074  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.651196  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:44.684633  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.688126  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.688880  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.788221  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.909003  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dde38eb3bd9341aa90d41a6f21bb29a1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:44.918221  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.960759  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.963477  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:44.994349  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:45.043722  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.129784   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:45.236809  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.351613  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.420279  1690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111) [suppressed 494 similar messages]
W20260709 13:04:45.444797  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.536618  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.567019  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.633313  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:45.653581  2372 consensus_queue.cc:237] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:45.681005  1720 raft_consensus.cc:1275] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:04:45.692205  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:45.692960  9480 consensus_queue.cc:1048] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:04:45.693317  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.727406  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:45.732407  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.746929  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:45.789976  9480 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index 3 to 10, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:45.794183  1720 raft_consensus.cc:2955] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 3 to 10, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
W20260709 13:04:45.808899  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:45.864439  1473 catalog_manager.cc:5697] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: config changed from index 3 to 10, VOTER cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) evicted. New cstate: current_term: 16 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } } }
I20260709 13:04:45.867690  1458 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 996f59fd8f2340a49403b3e9ef4c8e86 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
W20260709 13:04:45.886976  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.904014  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.922254  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:46.001869  1632 tablet_service.cc:1558] Processing DeleteTablet for tablet 996f59fd8f2340a49403b3e9ef4c8e86 with delete_type TABLET_DATA_TOMBSTONED (TS cd6d377efe514bc58ae49c3f4a107232 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:36598
I20260709 13:04:46.032307  9525 tablet_replica.cc:333] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:46.104329  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.130527   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:46.226019  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:46.265316  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.350432  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:46.355757  9525 raft_consensus.cc:2243] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:46.359071  9525 raft_consensus.cc:2272] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:46.364248  9525 ts_tablet_manager.cc:1916] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:04:46.355899  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:46.424798  9525 ts_tablet_manager.cc:1929] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.9
I20260709 13:04:46.425631  9525 log.cc:1199] T 996f59fd8f2340a49403b3e9ef4c8e86 P cd6d377efe514bc58ae49c3f4a107232: Deleting WAL directory at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0/minicluster-data/ts-1-root/wals/996f59fd8f2340a49403b3e9ef4c8e86
I20260709 13:04:46.428786  1458 catalog_manager.cc:5028] TS cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): tablet 996f59fd8f2340a49403b3e9ef4c8e86 (table test-workload [id=04c6d1d1265d41f9b6684e1faf227f31]) successfully deleted
W20260709 13:04:46.462007  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.470816  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.472677  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.482828  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.549418  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.561738  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:46.662775  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62018.1us
W20260709 13:04:46.852007  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 88446.3us
W20260709 13:04:47.025298  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.058123  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:47.058904  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.061861  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:47.087198  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.102053  1616 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:47.118929  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.119799  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.131099   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:47.145696  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.152678  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.153548  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.210362  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.242173  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.308692  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 66577.7us
W20260709 13:04:47.371102  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.410923  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.418859  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.431288  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.449174  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 71167.1us
W20260709 13:04:47.454644  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.450670  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.498610  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.506490  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.548956  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:47.571889  1463 maintenance_manager.cc:419] P d61e0c3f41244926bf144dca0f9acf61: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033356
W20260709 13:04:47.689749  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.892575  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 151248us
W20260709 13:04:47.895061  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.900595  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:47.901404  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:04:47.912652  1462 maintenance_manager.cc:643] P d61e0c3f41244926bf144dca0f9acf61: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.338s	user 0.048s	sys 0.011s Metrics: {"bytes_written":7527,"delete_count":0,"lbm_write_time_us":99601,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":14343,"threads_started":1,"update_count":302,"wal-append.queue_time_us":13243}
I20260709 13:04:47.930169  1463 maintenance_manager.cc:419] P d61e0c3f41244926bf144dca0f9acf61: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.322715
W20260709 13:04:47.946612  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.099748  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1acf73e529394004842219637cff7cfc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:48.131692   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:48.148629  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.150051  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.151470  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.169395  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.340188  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.364565  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.401157  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:48.443082  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.521207  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:48.553735  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:48.556638  1462 maintenance_manager.cc:643] P d61e0c3f41244926bf144dca0f9acf61: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.619s	user 0.101s	sys 0.015s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":28293,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":42898,"dirs.run_cpu_time_us":534,"dirs.run_wall_time_us":534,"lbm_read_time_us":10311,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":22,"lbm_write_time_us":2967,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":11,"peak_mem_usage":308641,"reinsert_count":0,"thread_start_us":47489,"threads_started":1,"update_count":302}
W20260709 13:04:48.592252  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.593045  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.679662  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.716964  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.726836  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.784055  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.815070  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.864713  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.935125  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.943089  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.957710  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac7a0 after lost signal to thread 434
W20260709 13:04:48.959472  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc140 after lost signal to thread 1508
W20260709 13:04:48.961362  1505 debug-util.cc:398] Leaking SignalData structure 0x7b0800038e20 after lost signal to thread 1675
W20260709 13:04:48.962867  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08000f6020 after lost signal to thread 1749
W20260709 13:04:48.971236  1505 debug-util.cc:398] Leaking SignalData structure 0x7b080026e080 after lost signal to thread 2404
W20260709 13:04:48.972215  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1ca0 after lost signal to thread 9533
W20260709 13:04:48.972658  1505 debug-util.cc:398] Leaking SignalData structure 0x7b08002389a0 after lost signal to thread 9536
W20260709 13:04:48.990288  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.052810  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:49.132239   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:49.164145  1460 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 90640.2us
W20260709 13:04:49.164738  1616 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:49.217025  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dde38eb3bd9341aa90d41a6f21bb29a1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:49.228083  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.257733  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.305527  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:49.372872  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.464080  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.500594  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.526115  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.607813  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.608764  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:49.610903  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.616514  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.639575  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.730367  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.743083  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.775461  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.782181  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.782959  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.798662  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.884013  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.948693  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:49.969715  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.057099  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.133016   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 1597)
W20260709 13:04:50.136848  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.279613  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.343547  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.434464  1690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111) [suppressed 486 similar messages]
W20260709 13:04:50.472731  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.504980  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:50.539726  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.549641  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.567675  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.579356  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.588744  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.685372  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.781549  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.831342  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.008164  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.130563  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.192366  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:51.277448  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.292105  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.301399  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:51.302069  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.393215  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.532778  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.538868  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.558836  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.670955  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.699955  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.858407  1616 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:51.888386  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.943611  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.961953  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.962772  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.969655  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.019299  1616 consensus_peers.cc:597] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:52.089558  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:52.134690   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:04:52.158082  1616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 65633.3us
W20260709 13:04:52.168064  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.190177  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.191184  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.196002  1616 consensus_peers.cc:597] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.205376  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.220908  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.228742  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.239749  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.272938  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:52.317835  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.367043  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.426162  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.459092  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.483173  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.499053  1616 consensus_peers.cc:597] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.542351  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.610991  1616 consensus_peers.cc:597] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.781184  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:52.828802  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:52.850365  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:52.933743  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:52.974632  1616 consensus_peers.cc:597] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:53.021648  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 62518.8us
W20260709 13:04:53.049887  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.083221  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.097436   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:53.101912  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:53.192790  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:53.194677  1616 consensus_peers.cc:597] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:53.201604  1688 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.207656  1688 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.218084  1688 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.219684  1688 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.262786  1688 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.264281  1688 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.268599  1688 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.276964  1688 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.289244  1688 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.295629  1688 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.338208  1688 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.346750  2342 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.351238  1688 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.364658  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.369920  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.370272  1688 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.403017  1688 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.416821  1688 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.425064  1688 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.431185  1688 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.470990  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:53.508906  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.522104  1688 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:53.549959   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:53.567440   431 tablet_replica.cc:333] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.571811   431 raft_consensus.cc:2243] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:53.577739  1688 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.582314  1688 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.581041  2342 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.584161  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:53.585364   431 raft_consensus.cc:2272] T f9396e14142a4f6cb8733d8b08ac73b9 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:53.594252  1688 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:53.597790   431 tablet_replica.cc:333] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.599138   431 raft_consensus.cc:2243] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:53.610790   431 raft_consensus.cc:2272] T 93ef45e97cf24863a576b705ec8e07bd P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:53.613811  1688 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.635938  1616 consensus_peers.cc:597] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.636886   431 tablet_replica.cc:333] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.642189   431 raft_consensus.cc:2243] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:53.656209   431 raft_consensus.cc:2272] T 5fa278fc63cf4f6e8ca5de6992849e6e P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:53.664856   431 tablet_replica.cc:333] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:53.663987  2342 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.674078  1688 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:53.693915  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:53.693950   431 raft_consensus.cc:2243] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:53.703404   431 raft_consensus.cc:2272] T 363173567dd940e99e459f02148a42e0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:53.728992   431 tablet_replica.cc:333] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.745420   431 raft_consensus.cc:2243] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:53.746497  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:53.752825   431 raft_consensus.cc:2272] T da3ba11d664442d7b28bc849c61fdcdf P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:53.753283  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:53.794876  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.761535   431 tablet_replica.cc:333] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.816181   431 raft_consensus.cc:2243] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:53.818166   431 raft_consensus.cc:2272] T f3646214b1a14514955e904e05fadcf1 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:53.831017  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.841528   431 tablet_replica.cc:333] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.847226   431 raft_consensus.cc:2243] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:53.850785   431 raft_consensus.cc:2272] T de2b34490a3844168645e5b940ba01a9 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:53.849524  1616 consensus_peers.cc:597] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.874276   431 tablet_replica.cc:333] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:53.923833   431 raft_consensus.cc:2243] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:53.925737   431 raft_consensus.cc:2272] T e8efe0ac5e5e45d8acfa1dfe55172859 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:53.933106  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 45411us
W20260709 13:04:54.017109  1616 consensus_peers.cc:597] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:54.060233   431 tablet_replica.cc:333] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.065068   431 raft_consensus.cc:2243] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.067457   431 raft_consensus.cc:2272] T 20085fd5893f43378bf1c59b14c351c0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.076001   431 tablet_replica.cc:333] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.078642   431 raft_consensus.cc:2243] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:54.090404   431 raft_consensus.cc:2272] T c47fb1f3610c496ea3c5542634768fa6 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.147542  9530 raft_consensus.cc:493] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.159902  9530 raft_consensus.cc:515] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:54.187940  1616 consensus_peers.cc:597] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.189193   431 tablet_replica.cc:333] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:54.203356  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.204031  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 45136.2us
I20260709 13:04:54.210244  9530 leader_election.cc:290] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.192080   431 raft_consensus.cc:2243] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.220775  9569 raft_consensus.cc:493] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.222234  9569 raft_consensus.cc:515] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:54.223932  1616 leader_election.cc:336] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.229813  9530 raft_consensus.cc:493] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.231052  9530 raft_consensus.cc:515] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.240104  9569 leader_election.cc:290] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.243501   431 raft_consensus.cc:2272] T 30619dafd05d4e36a03ff8f500c3a38c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.235252  9530 leader_election.cc:290] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.270125  9585 raft_consensus.cc:493] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:54.283926  9591 raft_consensus.cc:493] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.283828  9574 raft_consensus.cc:493] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.289012  9574 raft_consensus.cc:515] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.288913  9591 raft_consensus.cc:515] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.295004  9574 leader_election.cc:290] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:54.308080  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.308573  9579 raft_consensus.cc:493] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.283569  9585 raft_consensus.cc:515] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:54.309796  9579 raft_consensus.cc:515] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.283185   431 tablet_replica.cc:333] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.323344  9585 leader_election.cc:290] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:54.325941  9579 leader_election.cc:290] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.330780  9574 raft_consensus.cc:493] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.331733  9574 raft_consensus.cc:515] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.332319  9591 leader_election.cc:290] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.333902   431 raft_consensus.cc:2243] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:04:54.334614  9574 leader_election.cc:290] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:54.334777  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.335151   431 raft_consensus.cc:2272] T 1acf73e529394004842219637cff7cfc P cd6d377efe514bc58ae49c3f4a107232 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.335484  1690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:45275, user_credentials={real_user=slave}} blocked reactor thread for 53297us
I20260709 13:04:54.360924  9574 raft_consensus.cc:493] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.362106  9574 raft_consensus.cc:515] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.368367  9574 leader_election.cc:290] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.370044  9591 raft_consensus.cc:493] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.387678  9591 raft_consensus.cc:515] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.389189  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bef56447d743db91bd1f59274db995" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.391209  9591 leader_election.cc:290] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.370867  9569 raft_consensus.cc:493] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.392629  1722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ac82307bd14706ab03f881a279f3e5" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.392604  9569 raft_consensus.cc:515] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.396514  9569 leader_election.cc:290] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.403266  9591 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.404162  9591 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:54.376677   431 tablet_replica.cc:333] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.406217   431 raft_consensus.cc:2243] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:54.406741  1723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c347beda1704867ba2582a7a34d255b" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.408916  1720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48efc21b583b4e7fa81ec24726402281" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.409916  9591 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:54.409965  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7011d2943a2040ccb1a34122c0fd3ac3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.410563  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb9de7d85a2414bb920f63ec26f8686" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.411777  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42b29e0e5e884027ab75497ae8e020bf" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.413741  9569 raft_consensus.cc:493] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.421125  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd4d3a0e164248a5be4b2d19407551af" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.440546  9569 raft_consensus.cc:515] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.426874  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e1643d2c2f4a96a57a8bae29e7f19a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.427692  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.429674  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9273554b0f40499d928621ee1bccc9d8" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.433245  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26917aa1f55f48918fedd4960446f38c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:54.443421  9569 leader_election.cc:290] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.447124  9569 raft_consensus.cc:493] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:54.447968  9569 raft_consensus.cc:515] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:54.451671  9569 leader_election.cc:290] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:54.458663  1616 leader_election.cc:336] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:54.459331  1616 leader_election.cc:336] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:54.459993  1616 consensus_peers.cc:597] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.460794  1616 leader_election.cc:336] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:54.461412  1616 consensus_peers.cc:597] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.461930  1616 leader_election.cc:336] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.462396  1616 leader_election.cc:304] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:04:54.463471  1616 leader_election.cc:336] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.463776  9579 raft_consensus.cc:2749] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:54.467412   431 raft_consensus.cc:2272] T 1be748f6ba29434fbee8b14cedc1d192 P cd6d377efe514bc58ae49c3f4a107232 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.468523  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde38eb3bd9341aa90d41a6f21bb29a1" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:54.490504  1616 leader_election.cc:336] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:54.491458  1616 leader_election.cc:336] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.492439  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:54.478443  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:04:54.485448  9579 raft_consensus.cc:493] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:54.497833  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.498124  1616 leader_election.cc:336] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:54.498795  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.498895  1616 consensus_peers.cc:597] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.507557  1616 leader_election.cc:336] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:54.508163  1616 leader_election.cc:336] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.507148  1614 leader_election.cc:304] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.503808  9579 raft_consensus.cc:515] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.513147  9585 raft_consensus.cc:2749] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.514775  1614 leader_election.cc:304] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.515121  9579 leader_election.cc:290] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.515667  9585 raft_consensus.cc:2749] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:54.517168  1616 leader_election.cc:336] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.503289   431 tablet_replica.cc:333] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.521453   431 raft_consensus.cc:2243] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.522857   431 raft_consensus.cc:2272] T 78d12cc2bc7b4654acd41e6c45eb9753 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.534690  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54db7417c3014ba3948e6ce733254ba3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.537132  1721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c9e58595214defa013274c5193de1a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.542898   431 tablet_replica.cc:333] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:54.545935  1616 leader_election.cc:336] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.547703  9585 raft_consensus.cc:493] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.548523  9585 raft_consensus.cc:515] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.551082  9585 leader_election.cc:290] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.552764  1614 leader_election.cc:304] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:54.554630  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.561443  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:54.564615  9569 raft_consensus.cc:2749] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:54.566424  1614 leader_election.cc:304] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.567548  9569 raft_consensus.cc:2749] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:54.568713   431 raft_consensus.cc:2243] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.569456  1614 leader_election.cc:304] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.569828   431 raft_consensus.cc:2272] T d5ac82307bd14706ab03f881a279f3e5 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.570580  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.571640  9569 raft_consensus.cc:2749] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:54.570972  1614 leader_election.cc:304] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.576731  9569 raft_consensus.cc:2749] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:54.576143  1614 leader_election.cc:304] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.581487   431 tablet_replica.cc:333] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.581874  9569 raft_consensus.cc:2749] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:54.585417  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.585536   431 raft_consensus.cc:2243] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.586931   431 raft_consensus.cc:2272] T 38bef56447d743db91bd1f59274db995 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.587945  1614 leader_election.cc:304] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.590783  9569 raft_consensus.cc:493] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.591900  9569 raft_consensus.cc:515] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.592370  9585 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:54.596624  1616 leader_election.cc:336] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.596774  9585 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:54.603935  9586 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:54.605002  9586 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.605325  9585 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:54.604141  9569 leader_election.cc:290] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.604003  9579 raft_consensus.cc:493] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:04:54.608579  9579 raft_consensus.cc:515] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.609696  9586 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:54.610891  9579 leader_election.cc:290] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.607409  9574 raft_consensus.cc:2749] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:54.612746  1614 leader_election.cc:304] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.615554  9569 raft_consensus.cc:2749] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.618767  1614 leader_election.cc:304] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.619488   431 tablet_replica.cc:333] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.621199   431 raft_consensus.cc:2243] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.621762  9569 raft_consensus.cc:2749] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:54.623165  1690 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.624629   431 raft_consensus.cc:2272] T fd4d3a0e164248a5be4b2d19407551af P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.628688   431 tablet_replica.cc:333] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.629878   431 raft_consensus.cc:2243] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.630851   431 raft_consensus.cc:2272] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.621994  1616 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.622924  1614 leader_election.cc:304] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:04:54.636276  1616 leader_election.cc:336] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.640100  9569 raft_consensus.cc:2749] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:54.642658  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d5536a0ca5d48e4b98e0084e0aefb70" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.644307  1724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8014e1ef10d84a9f9c4d676b56c5105d" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.647677  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7778fc5cf1c746c4a4a974883a271ce0" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:54.649966  1616 leader_election.cc:336] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.651871  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:54.653177  1616 consensus_peers.cc:597] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.651979  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "740ac262ab184918992607d66446b8f3" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:54.646413  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.662218  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57592deb11fc4830921f01c2a3c32c4a" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.664486  9579 raft_consensus.cc:493] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
W20260709 13:04:54.664773  1688 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:54.665309  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:54.665398  9579 raft_consensus.cc:515] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:54.666726  9586 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:54.668016  9579 leader_election.cc:290] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:54.670128  9569 raft_consensus.cc:493] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da3b57321031416cba9db75450ef302a)
I20260709 13:04:54.672837  9569 raft_consensus.cc:515] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:54.677021  1614 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
W20260709 13:04:54.679581  1616 consensus_peers.cc:597] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:54.680398  9574 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:54.680982  1614 leader_election.cc:304] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a
I20260709 13:04:54.681948  9574 raft_consensus.cc:2749] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.634776   431 tablet_replica.cc:333] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:54.687359  1616 leader_election.cc:336] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.679440  9569 leader_election.cc:290] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:54.684119   431 raft_consensus.cc:2243] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.705158   431 raft_consensus.cc:2272] T d7e1643d2c2f4a96a57a8bae29e7f19a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.692134  1614 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
W20260709 13:04:54.706665  1616 leader_election.cc:336] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.707249  9569 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.712391   431 tablet_replica.cc:333] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.716580  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bb54b92d594aa7a2d8f8402e20c0ca" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:54.718832  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62836cb7c4a548b5a6c44c89782a958c" candidate_uuid: "cd6d377efe514bc58ae49c3f4a107232" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:54.721715  1614 leader_election.cc:304] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.722661  9569 raft_consensus.cc:2749] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:54.722924   431 raft_consensus.cc:2243] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.724737  1614 leader_election.cc:304] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.725412   431 raft_consensus.cc:2272] T 740ac262ab184918992607d66446b8f3 P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.725703  1614 raft_consensus.cc:2686] T 8014e1ef10d84a9f9c4d676b56c5105d P cd6d377efe514bc58ae49c3f4a107232: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:04:54.730669   431 tablet_replica.cc:333] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.734607   431 raft_consensus.cc:2243] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.735836   431 raft_consensus.cc:2272] T 7011d2943a2040ccb1a34122c0fd3ac3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.736838  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.741851  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.731403  1614 leader_election.cc:304] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.743191  9569 raft_consensus.cc:2749] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:54.745343  1614 leader_election.cc:304] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.756671  9569 raft_consensus.cc:2749] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:54.759560  1614 leader_election.cc:304] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:04:54.760529  9569 raft_consensus.cc:2749] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:54.765923  1614 leader_election.cc:304] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd6d377efe514bc58ae49c3f4a107232; no voters: 4f737473248e4c059fa74af3b8e1f77e, da3b57321031416cba9db75450ef302a
I20260709 13:04:54.766985  9569 raft_consensus.cc:2749] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:54.771176   431 tablet_replica.cc:333] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.774372   431 raft_consensus.cc:2243] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.775589   431 raft_consensus.cc:2272] T 26917aa1f55f48918fedd4960446f38c P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.787830   431 tablet_replica.cc:333] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.789263   431 raft_consensus.cc:2243] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:54.791416   431 raft_consensus.cc:2272] T 6834f8da7764493a94dc534e4471b3dc P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.791833  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.796362   431 tablet_replica.cc:333] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.847275   431 raft_consensus.cc:2243] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.874498   431 raft_consensus.cc:2272] T 48efc21b583b4e7fa81ec24726402281 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.882099   431 tablet_replica.cc:333] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.902108   431 raft_consensus.cc:2243] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:04:54.908959  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.918669   431 raft_consensus.cc:2272] T 2c501b3d8f86404e991cec1222bd0825 P cd6d377efe514bc58ae49c3f4a107232 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:54.923115   431 tablet_replica.cc:333] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.924729  9586 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:54.931773  9586 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:54.934104  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:54.934921  9586 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:54.937599   431 raft_consensus.cc:2243] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:54.948899   431 raft_consensus.cc:2272] T 42b29e0e5e884027ab75497ae8e020bf P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:54.943149  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:54.961795   431 tablet_replica.cc:333] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:54.972760   431 raft_consensus.cc:2243] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:54.964287  1690 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:54.973712  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:54.976416  9586 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:55.030853   431 raft_consensus.cc:2272] T 54db7417c3014ba3948e6ce733254ba3 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.036756   431 tablet_replica.cc:333] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.037890   431 raft_consensus.cc:2243] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.039258   431 raft_consensus.cc:2272] T 9c347beda1704867ba2582a7a34d255b P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.048394   431 tablet_replica.cc:333] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.049727   431 raft_consensus.cc:2243] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:04:55.052539  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:55.053390  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:55.051983   431 raft_consensus.cc:2272] T f048890019da43548110ab19fce7a878 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.080785   431 tablet_replica.cc:333] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.082144   431 raft_consensus.cc:2243] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.083386   431 raft_consensus.cc:2272] T 4d5536a0ca5d48e4b98e0084e0aefb70 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.103394   431 tablet_replica.cc:333] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.104561   431 raft_consensus.cc:2243] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.113758   431 raft_consensus.cc:2272] T e69630519d3c48eca350b04183abca32 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:55.117923  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:55.127056  9584 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.127851  9584 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:55.130594  9584 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:55.125892   431 tablet_replica.cc:333] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:55.135478  2342 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:55.138092   431 raft_consensus.cc:2243] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.142674   431 raft_consensus.cc:2272] T 87baee349cdd48d88642bf37176b1b26 P cd6d377efe514bc58ae49c3f4a107232 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:55.145875  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:55.159175  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:55.160188  9584 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:55.158346  1616 consensus_peers.cc:597] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:55.164000   431 tablet_replica.cc:333] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.173378   431 raft_consensus.cc:2243] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.177829   431 raft_consensus.cc:2272] T 44cb1d5977df435cac936d86c5bd1880 P cd6d377efe514bc58ae49c3f4a107232 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.188951   431 tablet_replica.cc:333] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
W20260709 13:04:55.200009  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:55.198323   431 raft_consensus.cc:2243] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.208273   431 raft_consensus.cc:2272] T 6bd1d3ea2c984a999cdd42423ebec16f P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.221544   431 tablet_replica.cc:333] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.222987   431 raft_consensus.cc:2243] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.224997   431 raft_consensus.cc:2272] T ba20622bf78343c5bf6ac194af441026 P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.229074   431 tablet_replica.cc:333] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.231882   431 raft_consensus.cc:2243] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.234352   431 raft_consensus.cc:2272] T 27e7f73e79344731aa22e748b700ffd9 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.238778   431 tablet_replica.cc:333] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.240815   431 raft_consensus.cc:2243] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.242369   431 raft_consensus.cc:2272] T 1d22dc39228b4001961709e74ef6bc9c P cd6d377efe514bc58ae49c3f4a107232 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.246363   431 tablet_replica.cc:333] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.247428   431 raft_consensus.cc:2243] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.249133   431 raft_consensus.cc:2272] T fb93f4d4caa2474bb5d42f9940f0dcdc P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.253305   431 tablet_replica.cc:333] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.254642   431 raft_consensus.cc:2243] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.256382   431 raft_consensus.cc:2272] T f6d77c909fe243b197ef61ea6b271908 P cd6d377efe514bc58ae49c3f4a107232 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.267546   431 tablet_replica.cc:333] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.269277   431 raft_consensus.cc:2243] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.270730   431 raft_consensus.cc:2272] T 54bb54b92d594aa7a2d8f8402e20c0ca P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.279099   431 tablet_replica.cc:333] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.282480   431 raft_consensus.cc:2243] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.283733   431 raft_consensus.cc:2272] T 62836cb7c4a548b5a6c44c89782a958c P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.289995   431 tablet_replica.cc:333] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.291237   431 raft_consensus.cc:2243] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.293071   431 raft_consensus.cc:2272] T b837a5ec9a0d4f2a8635dedf45d51aa2 P cd6d377efe514bc58ae49c3f4a107232 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.298386   431 tablet_replica.cc:333] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.299996   431 raft_consensus.cc:2243] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.302834   431 raft_consensus.cc:2272] T 4391f9a46a9542a7bf726b05393b7609 P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.307413   431 tablet_replica.cc:333] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.308576   431 raft_consensus.cc:2243] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.309554   431 raft_consensus.cc:2272] T 57592deb11fc4830921f01c2a3c32c4a P cd6d377efe514bc58ae49c3f4a107232 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.314473   431 tablet_replica.cc:333] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.315631   431 raft_consensus.cc:2243] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.317396   431 raft_consensus.cc:2272] T 414132da4f974e5c912e5d5dfa077654 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.321084   431 tablet_replica.cc:333] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.336670   431 raft_consensus.cc:2243] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.339358   431 raft_consensus.cc:2272] T dde38eb3bd9341aa90d41a6f21bb29a1 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.344930   431 tablet_replica.cc:333] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.361106   431 raft_consensus.cc:2243] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.362247   431 raft_consensus.cc:2272] T 53c9e58595214defa013274c5193de1a P cd6d377efe514bc58ae49c3f4a107232 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.365860   431 tablet_replica.cc:333] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.371996   431 raft_consensus.cc:2243] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:55.369275  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.381610   431 raft_consensus.cc:2272] T 9bb9de7d85a2414bb920f63ec26f8686 P cd6d377efe514bc58ae49c3f4a107232 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.385560   431 tablet_replica.cc:333] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.397411   431 raft_consensus.cc:2243] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.398483   431 raft_consensus.cc:2272] T 7778fc5cf1c746c4a4a974883a271ce0 P cd6d377efe514bc58ae49c3f4a107232 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.407934   431 tablet_replica.cc:333] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.418283   431 raft_consensus.cc:2243] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:55.419322   431 raft_consensus.cc:2272] T 9273554b0f40499d928621ee1bccc9d8 P cd6d377efe514bc58ae49c3f4a107232 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:55.423460   431 tablet_replica.cc:333] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232: stopping tablet replica
I20260709 13:04:55.424499   431 raft_consensus.cc:2243] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:04:55.437635   431 raft_consensus.cc:2272] T fac0a563dc584b4fbe82f01f362fa966 P cd6d377efe514bc58ae49c3f4a107232 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:55.444757  1688 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 626 similar messages]
W20260709 13:04:55.463165  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.489723  9594 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.490651  9594 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:55.495963  9586 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.497779  9586 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:55.500465  9586 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:55.507867  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:55.512912  9594 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:55.515327  1688 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:55.520454  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:55.521119  1690 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:55.521575  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:55.523159  9594 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:55.528236  1688 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:55.528879  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:55.530745  9586 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:55.572227  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:55.581847  1688 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.618820  1688 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.676966  1688 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.712615  1688 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.721203  1688 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.724629  1688 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.744891  1688 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.783268  1688 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.821255  9595 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.822049  9595 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:55.824952  9595 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:55.828112  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:55.834702  2342 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:55.835563  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:55.836766  9595 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:55.851187  9595 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.852033  9595 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:55.855556  9595 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:55.856786  1688 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.857632  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:55.860075  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:55.862107  2342 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:55.863565  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:55.865691  9584 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:55.870568  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.886617  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:55.905510  1688 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:55.919605  9594 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.920480  9594 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:55.923556  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:55.939137  1688 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:55.945007  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:55.947407  1616 connection.cc:466] server connection from 127.0.0.1:49766 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49766 (request call id 4025) could send its response
I20260709 13:04:55.945871  2380 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 15.
I20260709 13:04:55.953480  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:55.942811  9594 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:55.957860  9586 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:55.958737  9586 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:55.959228  9586 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:55.961853  1688 connection.cc:573] client connection to 127.0.107.194:43325 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:04:55.963027  1688 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.973502  1688 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.977684  1688 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:55.989269  1688 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.000813  1688 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.022409  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:56.015662  9586 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.028882  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1acf73e529394004842219637cff7cfc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:56.029682  2380 raft_consensus.cc:3060] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:56.036654  1688 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.037432  1688 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.056041  2380 raft_consensus.cc:2468] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 16.
W20260709 13:04:56.058177  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:56.056696  9586 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:56.059492  1688 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.060717  9594 raft_consensus.cc:2804] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:56.062409  9594 raft_consensus.cc:697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:56.063674  9594 consensus_queue.cc:237] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.091102  1473 catalog_manager.cc:5697] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 15 to 16, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:56.096987  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.107623  2342 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.120203  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:56.137010  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:56.146521  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.150991  1688 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.152395  1688 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.153292  1688 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.197496  2342 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.200799  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:56.211266  9586 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.212520  9595 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.212365  9586 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.213267  9595 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.216096  9595 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:56.216768  9594 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.217415  9594 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.219897  9594 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:56.220153  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:56.216061  9586 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:56.228475  2342 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.229182  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.230623  9584 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:56.235317  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:56.236193  2380 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
W20260709 13:04:56.229121  1690 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.252203  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:56.262305  9594 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:56.262857  9594 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.263257  9594 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:56.272917  1688 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:56.281589  9586 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.282321  9586 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:56.283552  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:56.284833  9586 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.288622  1688 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:56.289752  1688 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.290210  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.292305  9584 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.297909  9584 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.299719  9586 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.301040  9584 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:56.301371  9594 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.302954  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
W20260709 13:04:56.308076  2342 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.308794  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.309859  9584 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.319545  9594 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:56.314352  1690 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.333276  9586 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.334820  9596 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.336071  9596 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.344285  9596 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:56.347344  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834f8da7764493a94dc534e4471b3dc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:56.348363  2380 raft_consensus.cc:3060] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:56.334043  9586 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.361372  2380 raft_consensus.cc:2468] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 23.
W20260709 13:04:56.365806  1688 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:56.371086  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1acf73e529394004842219637cff7cfc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.385547  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.371025  9594 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.386525  9594 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.389408  9594 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:56.390074  9596 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.364604  9586 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:56.394811  9584 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.396014  9584 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.399915  9584 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.402245  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.406526  1688 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
W20260709 13:04:56.405920  1690 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:56.408222  1688 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:56.408682  9586 raft_consensus.cc:2804] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:04:56.408954  1688 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:56.410081  1688 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.412374  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.415373  9586 raft_consensus.cc:697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:56.417526  9586 consensus_queue.cc:237] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.431020  9595 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.431998  9595 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.436275  9595 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:04:56.429327  1688 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.441259  9594 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.441679  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.441970  9594 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.446506  9596 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:56.446528  9594 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:56.447938  1688 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.453562  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:56.453699  2342 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.454109  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:56.454511  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.456321  9586 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:56.457032  9584 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:56.456581  1688 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:56.459867  1688 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.461036  1473 catalog_manager.cc:5697] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 22 to 23, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:56.463747  1690 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.464324  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.465209  9586 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:56.479749  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
W20260709 13:04:56.483201  2342 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.484021  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.484856  9584 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:56.487550  9586 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.488234  9586 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:56.488703  1688 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.490739  9586 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:56.493404  1688 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: cd6d377efe514bc58ae49c3f4a107232, da3b57321031416cba9db75450ef302a
I20260709 13:04:56.494306  9586 raft_consensus.cc:2749] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:04:56.501892  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:56.502841  2372 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 21.
I20260709 13:04:56.504029  9596 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.504716  9596 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.507150  9596 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.516538  1690 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.518625  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:56.529176  9596 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:56.529742  9596 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.530179  9596 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:56.538904  1688 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.547057  9584 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.547879  9584 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:56.562608  1688 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.565974  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.567359  9594 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:56.551425  9596 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.570958  9584 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.563287  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.582649  9596 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
W20260709 13:04:56.601872  2342 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.602761  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.603746  9596 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.603842  9584 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:56.604604  9596 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.609328  9594 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.610116  9594 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.613252  9594 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:56.613538  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414132da4f974e5c912e5d5dfa077654" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:56.614310  2372 raft_consensus.cc:3060] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:56.619787  1690 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.622753  9596 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:56.625051  2380 raft_consensus.cc:1275] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:56.633383  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:56.633684  9594 consensus_queue.cc:1048] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:56.636375  1688 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:56.634830  2380 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 25.
W20260709 13:04:56.637162  1688 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.640633  2372 raft_consensus.cc:2468] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 22.
I20260709 13:04:56.645987  9596 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.662168  9596 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:56.673933  1690 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.675319  9596 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:56.676534  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:56.678882  9594 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:56.679486  9594 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.679920  9594 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:56.681401  2342 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:56.702888  9594 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.704865  1688 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.710572  9586 raft_consensus.cc:2804] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:56.714967  9584 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.715790  9584 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.716707  9595 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.717418  9595 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.720175  9595 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.726670  1688 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.727408  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.729041  9596 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:56.732437  2342 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.738128  9584 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.739336  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:56.740729  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:56.741250  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:56.744521  1688 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.753116  9586 raft_consensus.cc:697] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:56.751824  9584 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:56.765650  9586 consensus_queue.cc:237] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.783075  9615 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.783299  9614 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.785244  9612 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.791548  9614 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.791549  9612 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.795392  9612 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:56.795491  9614 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:56.805838  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2" is_pre_election: true
I20260709 13:04:56.806727  1725 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 26.
I20260709 13:04:56.817772  9594 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:56.822183  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:56.830834  9584 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:56.837086  9584 raft_consensus.cc:493] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.837595  9584 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:56.828202  9615 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:56.847720  9615 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.850135  2342 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.860267  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:56.863289  2342 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.868584  9612 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:56.874522  2380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a" is_pre_election: true
I20260709 13:04:56.875399  2380 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 18.
I20260709 13:04:56.875835  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6d77c909fe243b197ef61ea6b271908" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
W20260709 13:04:56.874030  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.876801  2372 raft_consensus.cc:3060] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:56.868124  9619 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.883960  9619 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.887545  9584 raft_consensus.cc:515] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.876911  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:56.904376  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:56.910199  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:56.911512  9618 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:56.916723  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:56.917465  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:56.919617  2372 raft_consensus.cc:2468] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 26.
I20260709 13:04:56.926779  9619 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:56.927151  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:56.929222  2342 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:56.928433  1473 catalog_manager.cc:5697] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 21 to 22, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.930649  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:56.932862  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:56.935563  9615 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:56.936004  9619 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:56.936093  9615 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:56.936836  9615 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:56.937696  9584 leader_election.cc:290] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:04:56.942018  1725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b837a5ec9a0d4f2a8635dedf45d51aa2" candidate_uuid: "da3b57321031416cba9db75450ef302a" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "58cdb682fbf54792a702a7a455f265a2"
I20260709 13:04:56.943046  1725 raft_consensus.cc:3060] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:56.955307  2342 leader_election.cc:336] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:56.963390  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:56.975244  1688 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: 
I20260709 13:04:56.978578  9620 raft_consensus.cc:2804] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:04:56.979872  9620 raft_consensus.cc:697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 26 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
W20260709 13:04:56.980605   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1672)
I20260709 13:04:56.981226  9620 consensus_queue.cc:237] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:56.996873  1725 raft_consensus.cc:2468] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3b57321031416cba9db75450ef302a in term 27.
W20260709 13:04:56.997764  1688 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:56.998615  1690 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:57.000542  2342 leader_election.cc:304] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:57.006187  1688 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:57.012452  9584 raft_consensus.cc:2804] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:04:57.012894  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.014990  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.013461  9584 raft_consensus.cc:697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 27 LEADER]: Becoming Leader. State: Replica: da3b57321031416cba9db75450ef302a, State: Running, Role: LEADER
I20260709 13:04:57.016935  9584 consensus_queue.cc:237] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:57.021744  1688 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:57.022397  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:57.026170  9620 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:57.027213  1473 catalog_manager.cc:5697] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 25 to 26, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 26 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.009953  9615 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:57.025542  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:57.034940  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.041633  2372 raft_consensus.cc:1275] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:04:57.042426  9615 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Requested vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), da3b57321031416cba9db75450ef302a (127.0.107.196:45309)
I20260709 13:04:57.067860  1467 catalog_manager.cc:5697] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a reported cstate change: term changed from 26 to 27, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to da3b57321031416cba9db75450ef302a (127.0.107.196). New cstate: current_term: 27 leader_uuid: "da3b57321031416cba9db75450ef302a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:57.081751  1688 consensus_peers.cc:597] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.085812  9620 consensus_queue.cc:1048] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:04:57.089769  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.090992  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:04:57.103825  2372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb93f4d4caa2474bb5d42f9940f0dcdc" candidate_uuid: "58cdb682fbf54792a702a7a455f265a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da3b57321031416cba9db75450ef302a"
I20260709 13:04:57.104673  2372 raft_consensus.cc:3060] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:57.146924  2372 raft_consensus.cc:2468] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58cdb682fbf54792a702a7a455f265a2 in term 19.
W20260709 13:04:57.149103  1688 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:57.165560  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.166316  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.168658  1688 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2, da3b57321031416cba9db75450ef302a; no voters: cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:57.170832  9620 raft_consensus.cc:2804] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:57.171891  9620 raft_consensus.cc:697] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Becoming Leader. State: Replica: 58cdb682fbf54792a702a7a455f265a2, State: Running, Role: LEADER
I20260709 13:04:57.173312  9620 consensus_queue.cc:237] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:57.207815  1688 consensus_peers.cc:597] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.222381  1467 catalog_manager.cc:5697] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 reported cstate change: term changed from 18 to 19, leader changed from cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194) to 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "58cdb682fbf54792a702a7a455f265a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:57.281597  2372 raft_consensus.cc:1275] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:04:57.283378  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.296422  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.297891  9620 consensus_queue.cc:1048] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:04:57.458811  1688 consensus_peers.cc:597] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.470993  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dde38eb3bd9341aa90d41a6f21bb29a1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:57.482589  1725 raft_consensus.cc:1275] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Refusing update from remote peer da3b57321031416cba9db75450ef302a: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:04:57.484422  9619 consensus_queue.cc:1048] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [LEADER]: Connected to new peer: Peer: permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:57.513732  2372 raft_consensus.cc:1275] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:04:57.521001  9594 consensus_queue.cc:1048] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:04:57.523257  2342 consensus_peers.cc:597] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.524531  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.562824  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.604590  9620 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:57.605353  9620 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:57.608038  9620 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:57.628779  1688 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:57.685199  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:57.685913  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:57.686383  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:57.695122  9594 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:57.696484  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.702114  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:57.705415  2372 raft_consensus.cc:1275] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Refusing update from remote peer 58cdb682fbf54792a702a7a455f265a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:57.713768  9594 consensus_queue.cc:1048] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:57.731084  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.737187  1688 consensus_peers.cc:597] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.747591  9594 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:57.748211  9594 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:57.753691  9594 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:57.771351  1690 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:57.782560  1688 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:57.783165  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:57.785249  9620 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:57.868239  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.975664  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.981169   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1672)
W20260709 13:04:58.111877  1688 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.112880  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:58.126199  1688 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.146229  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:04:58.200486  9620 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.201293  9620 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:58.206063  9620 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:58.235303  9594 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.236006  9594 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:58.239702  9594 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.241899  1690 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.261973  1688 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.262573  1688 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.272969  1688 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:58.274595  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:58.275378  9620 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:58.276916  1688 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:58.277559  1690 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.277667  1688 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.278075  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:58.278998  9620 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:58.278205  1688 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.339536  1688 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.366238  1688 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.384930  9620 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.385670  9620 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:58.395987  9584 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.396834  9584 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:58.399591  9584 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.404004  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.406119  9620 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.407361  1688 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.415300  1690 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.416916  2342 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:58.418723  1688 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.419960  9620 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.420776  9620 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:04:58.436067  1690 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.437569  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:58.444774  9620 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.442029  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:58.457984  2344 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:58.459301  9584 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:58.497982  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 58178us
W20260709 13:04:58.504699  1688 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:58.505296  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:58.508607  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.509343  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:58.511719  9620 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:58.513550  9620 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.514462  9620 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:58.527228  9620 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.533193  2342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:44281, user_credentials={real_user=slave}} blocked reactor thread for 60902.3us
I20260709 13:04:58.558365  9633 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:58.560261  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.587740  2342 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.559053  9633 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:58.594623  9633 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.601747  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.609344  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.610035  2342 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:58.616676  1688 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:58.617770  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:58.617811  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:58.620723  9633 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:58.628512  9620 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:58.630759  1688 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.631634  9586 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
W20260709 13:04:58.633152  1688 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.633744  1688 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.640962  1688 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.636662  9586 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:58.645709  9586 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.668673  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:58.689186  1690 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:58.691798  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.693446  1688 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.695156  1688 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:58.695894  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:04:58.709329  1688 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.709326  9620 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:58.710640  1688 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.720062  1688 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:58.720799  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:58.721899  9620 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:58.721165  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:58.755944  1688 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.774883  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.786187  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:58.818574  9620 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.819248  9620 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:58.822068  9620 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:58.840104  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:58.841097  1688 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:58.870321  1688 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.893108  1688 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.897745  1688 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:58.981791   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1672)
W20260709 13:04:58.992246  1688 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:58.995800  9633 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:58.996419  9633 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:59.007121  9633 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:59.008498  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:04:59.010567  2342 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:59.013983  2342 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:04:59.014473  2342 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:59.015301  9633 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:59.039579  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:59.040273  2342 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:59.052875  9633 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:59.053658  9633 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:04:59.067845  2342 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:59.069555  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:59.069979  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:59.072712  9633 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:04:59.073323  9634 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:59.199836  1688 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:59.252876  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.255712  1688 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:59.264536  9620 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:59.265097  9620 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:59.267436  9620 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:59.275970  1688 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:59.349781  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:04:59.406194  9633 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:59.406791  9633 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:04:59.409102  9633 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:59.418215  2342 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:59.419288  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:59.419909  2344 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:59.422933  9633 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:59.445482  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.536828  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 57331.3us
W20260709 13:04:59.587931  1688 consensus_peers.cc:597] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:59.627256  1688 consensus_peers.cc:597] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:59.725135  9620 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:04:59.725771  9620 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:04:59.729460  9620 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:04:59.732553  1688 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:04:59.825052  1688 consensus_peers.cc:597] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:59.860412  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.861739  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:59.862242  1690 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:59.863109  9620 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:04:59.863646  1690 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:59.864117  1690 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:59.865842  9620 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:59.867532  1690 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:04:59.867954  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:04:59.868733  9620 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:59.880795  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.910118  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.982357   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1672)
I20260709 13:05:00.019646  9620 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.020534  9620 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:00.022691  9620 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.027657  1690 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:00.040470  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:00.040990  1688 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:00.041738  9620 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:00.043385  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.043828  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:05:00.085636  9586 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.086161  9586 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:00.089969  9586 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.092527  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.096719  1690 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:00.107787  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.108331  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.113399  1688 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:00.113982  1688 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:00.119334  9620 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:00.141698  1688 consensus_peers.cc:597] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.177891  2342 consensus_peers.cc:597] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:00.183420  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:00.187091  9586 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.187649  9586 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:00.190064  9586 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.195253  1688 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:00.201771  1690 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:00.202237  1690 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:00.202951  9586 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:00.341307  9586 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.342001  9586 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:00.349599  9586 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.356578  1688 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:00.360253  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.362758  1690 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:00.363279  1690 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:05:00.363896  1688 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:00.364982  9586 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:00.461813  2342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111) [suppressed 719 similar messages]
W20260709 13:05:00.479312  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:00.527904  1746 debug-util.cc:398] Leaking SignalData structure 0x7b0800395700 after lost signal to thread 434
W20260709 13:05:00.529071  1746 debug-util.cc:398] Leaking SignalData structure 0x7b080035b620 after lost signal to thread 1508
W20260709 13:05:00.530308  1746 debug-util.cc:398] Leaking SignalData structure 0x7b080035b460 after lost signal to thread 1749
W20260709 13:05:00.531456  1746 debug-util.cc:398] Leaking SignalData structure 0x7b0800128820 after lost signal to thread 2404
W20260709 13:05:00.555505  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.556154  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.586580  1688 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.588527  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.592995  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:00.651799  9620 raft_consensus.cc:493] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.652388  9620 raft_consensus.cc:515] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:00.656770  9620 leader_election.cc:290] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.686102  1690 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:00.720981  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.737198  1688 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.737783  1688 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.738273  1688 leader_election.cc:336] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:00.738656  1688 leader_election.cc:304] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:00.742427  9633 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.743147  9633 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:05:00.752455  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:00.753978  9633 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.760191  2342 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:00.760697  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:00.762176  9633 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:00.765017  9620 raft_consensus.cc:2749] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:00.799191  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:00.830680  9633 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:00.831369  9633 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:00.833667  9633 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:00.850149  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:00.855629  2342 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:00.856717  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:00.857599  9633 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:00.861014  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.876330  1688 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.907478  1688 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.914639  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:00.928244  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.973744  1688 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:00.982944   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1672)
W20260709 13:05:01.006433  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:01.017704  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 44013.5us
W20260709 13:05:01.065004  1688 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.086115  2342 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.153664  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.154634  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.192190  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.213016  1688 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.213723  1688 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.224176  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.241611  1688 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.242188  1688 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.246156  1688 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.285079  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:05:01.335482  1688 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.339530  1688 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.367715  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:01.385483  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:01.397900  1688 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.398669  1688 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.399330  1688 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.422401  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:01.456955  9639 raft_consensus.cc:493] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:01.457607  9639 raft_consensus.cc:515] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:01.460204  9639 leader_election.cc:290] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:01.504691  2342 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.528162  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:01.531718  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.548555  2342 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.554714  1690 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:01.555229  1688 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.555490  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.562052  1688 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.562773  1688 leader_election.cc:336] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:01.563305  1688 leader_election.cc:304] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:01.564567  9620 raft_consensus.cc:2749] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:01.567019  1688 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:01.637451  9611 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:01.638083  9611 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
W20260709 13:05:01.650415  1690 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:01.651816  9611 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:01.658049  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.694408  1688 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:01.695092  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:01.696043  9639 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:01.700991  1688 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.701862  1688 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:01.711410  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.715756  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:01.724498  9639 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:01.725070  9639 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:01.727164  9639 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:01.735178  1688 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:01.740288  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:01.762980  1688 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:01.765581  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:01.768100  9639 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:01.798166  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:01.827020  9611 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:01.827689  9611 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:01.830199  9611 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:01.855628  1688 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:01.901525  1690 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:01.905699  1690 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:01.907073  9611 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:01.967535  9620 raft_consensus.cc:493] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:01.968151  9620 raft_consensus.cc:515] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:01.970602  9620 leader_election.cc:290] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:01.976532  1690 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:01.988775  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.007915  1688 leader_election.cc:336] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:02.008772  1688 leader_election.cc:304] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:02.009598  9620 raft_consensus.cc:2749] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:02.010016   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1672)
W20260709 13:05:02.085579  1688 consensus_peers.cc:597] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:02.108579  9638 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:02.109318  9638 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:02.111851  9638 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:02.123694  2342 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:02.132490  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:02.133265  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:02.134483  9638 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:02.249498  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.340242  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:02.356957  9638 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:02.357628  9638 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:02.364075  9638 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:02.372229  2342 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:02.374315  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:02.374763  2344 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:02.376253  9638 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:02.396839  1688 consensus_peers.cc:597] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:02.397590  1688 consensus_peers.cc:597] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:02.417919  9620 raft_consensus.cc:493] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:02.418542  9620 raft_consensus.cc:515] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:02.420459  9620 leader_election.cc:290] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:02.428102  1690 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:02.463419  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:02.470497  1688 leader_election.cc:336] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:02.473389  1688 leader_election.cc:304] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:02.492163  9639 raft_consensus.cc:2749] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:02.494745  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:05:02.532612  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.588181  2342 consensus_peers.cc:597] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:02.582399  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.628763  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:02.676594  9639 raft_consensus.cc:493] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:02.677536  9639 raft_consensus.cc:515] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:02.679886  9639 leader_election.cc:290] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:02.699496  1688 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:02.700598  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.714174  1690 leader_election.cc:336] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:02.717990  1690 leader_election.cc:304] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:02.718806  9639 raft_consensus.cc:2749] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:02.727936  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.747440  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:02.779733  1688 consensus_peers.cc:597] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:02.887046  1688 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:02.993211  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.010660   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1672)
W20260709 13:05:03.048215  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.159049  1690 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:03.177403  9642 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:03.178273  9642 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:03.187956  9642 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:03.225481  1688 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43325, user_credentials={real_user=slave}} blocked reactor thread for 49892.3us
W20260709 13:05:03.243808  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.248852  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:03.273566  1688 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.285426  1688 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.291122  2342 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:03.295748  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:03.302213  9643 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:03.361114  1690 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.363863  1688 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.371064  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.399466  1690 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:03.408618  9611 raft_consensus.cc:493] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:03.409312  9611 raft_consensus.cc:515] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:03.417763  9611 leader_election.cc:290] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:03.429834  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.444024  1688 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.449610  1690 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:03.452318  1690 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:03.521582  1688 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.524757  1688 leader_election.cc:336] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:03.525290  1688 leader_election.cc:304] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:03.526165  9620 raft_consensus.cc:2749] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:03.529703  1690 consensus_peers.cc:597] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.532083  1688 consensus_peers.cc:597] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.558719  1690 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:03.585189  1690 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.586256  1688 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.589131  1688 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.589658  1688 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.592053  1690 consensus_peers.cc:597] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:03.597234  1688 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.607923  1688 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:03.611210  9620 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:03.611753  9620 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:03.613579  9620 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:03.629559  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:03.641923  1688 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:03.647697  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:03.648419  9620 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:03.651485  2342 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.654855  1688 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.679129  1688 consensus_peers.cc:597] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.682845  1688 consensus_peers.cc:597] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.692726  1688 consensus_peers.cc:597] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.742457  1688 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.752669  1690 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:03.852029  9643 raft_consensus.cc:493] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:03.852748  9643 raft_consensus.cc:515] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:05:03.860773  1690 consensus_peers.cc:597] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:03.867789  2344 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:03.868398  9643 leader_election.cc:290] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:03.907868  2342 leader_election.cc:336] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:03.908792  2342 leader_election.cc:304] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:03.909610  9643 raft_consensus.cc:2749] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:03.916525  1688 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.940567  2342 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.960989  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:03.969167  1688 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:03.999238  1690 consensus_peers.cc:597] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:03.999904  1690 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:04.011677   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1672)
W20260709 13:05:04.028661  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:04.046233  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:04.061714  1688 consensus_peers.cc:597] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:04.097846  2342 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:04.148547  1690 consensus_peers.cc:597] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:04.259497  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:05:04.272469  1690 consensus_peers.cc:597] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:04.273830  1690 consensus_peers.cc:597] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:04.273257  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:04.281414  9643 raft_consensus.cc:493] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:04.283196  9643 raft_consensus.cc:515] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:05:04.294945  2342 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:04.302524  2344 leader_election.cc:336] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:04.306475  2344 leader_election.cc:304] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:04.308434  9643 leader_election.cc:290] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:05:04.311406  9642 raft_consensus.cc:2749] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:04.330188  1690 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:04.343230  1688 consensus_peers.cc:597] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:04.355926  1688 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:04.375073  1688 consensus_peers.cc:597] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:04.404767  1688 consensus_peers.cc:597] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:04.479532  9649 raft_consensus.cc:493] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:04.480410  9649 raft_consensus.cc:515] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:04.491446  9649 leader_election.cc:290] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:04.566905  1690 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:04.640821  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:04.673904  1688 leader_election.cc:336] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:04.674437  1688 leader_election.cc:304] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:04.675391  9650 raft_consensus.cc:2749] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:05:04.683379  1688 consensus_peers.cc:597] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:04.715503  1688 consensus_peers.cc:597] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:04.721642  1690 consensus_peers.cc:597] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:04.742375  1688 consensus_peers.cc:597] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:04.780371  1690 consensus_peers.cc:597] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:05:04.820569  1690 consensus_peers.cc:597] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:04.838650  9653 raft_consensus.cc:493] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:04.839340  9653 raft_consensus.cc:515] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:04.841857  9653 leader_election.cc:290] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:04.852065  1688 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:04.859052  1690 leader_election.cc:336] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:04.859619  1690 leader_election.cc:304] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:04.861256  9650 raft_consensus.cc:2749] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:05:04.862515  1690 consensus_peers.cc:597] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:05:04.874142  9611 raft_consensus.cc:493] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:04.874771  9611 raft_consensus.cc:515] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:04.876796  9611 leader_election.cc:290] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:04.881989  1690 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:04.882548  1688 leader_election.cc:336] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:04.882993  1688 leader_election.cc:304] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:04.883822  9611 raft_consensus.cc:2749] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:05:04.920877  9651 raft_consensus.cc:493] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:04.921609  9651 raft_consensus.cc:515] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:04.926357  9651 leader_election.cc:290] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:04.928797  2344 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:04.931871  2342 leader_election.cc:336] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:04.932400  2342 leader_election.cc:304] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:04.933254  9651 raft_consensus.cc:2749] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:04.957481  1688 consensus_peers.cc:597] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:04.982296  1690 consensus_peers.cc:597] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:05.009935  1690 consensus_peers.cc:597] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:05.088306  1690 consensus_peers.cc:597] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:05.106233  1690 consensus_peers.cc:597] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:05.123364  2342 consensus_peers.cc:597] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:05.244060  9611 raft_consensus.cc:493] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:05.244683  9611 raft_consensus.cc:515] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:05.244969   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:05:05.246603  9611 leader_election.cc:290] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:05.261765  1690 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:05.264395  1688 leader_election.cc:336] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:05.265007  1688 leader_election.cc:304] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
W20260709 13:05:05.265223  1690 consensus_peers.cc:597] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:05.265957  9611 raft_consensus.cc:2749] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:05.285694  1690 consensus_peers.cc:597] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:05.369822  9650 raft_consensus.cc:493] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:05.370466  9650 raft_consensus.cc:515] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } }
I20260709 13:05:05.373152  9650 leader_election.cc:290] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
W20260709 13:05:05.379084  1690 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:05.379866  1688 leader_election.cc:336] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:05.380317  1688 leader_election.cc:304] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58cdb682fbf54792a702a7a455f265a2; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:05.381176  9650 raft_consensus.cc:2749] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:05:05.406654  1688 consensus_peers.cc:597] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:05.412736   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:05:05.467883  1690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111) [suppressed 731 similar messages]
W20260709 13:05:05.471144  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.480270   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:05.481351   431 tablet_replica.cc:333] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.482141   431 raft_consensus.cc:2243] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.483464   431 raft_consensus.cc:2272] T f9396e14142a4f6cb8733d8b08ac73b9 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.486506   431 tablet_replica.cc:333] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.487331   431 raft_consensus.cc:2243] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.488912   431 raft_consensus.cc:2272] T 363173567dd940e99e459f02148a42e0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.492455   431 tablet_replica.cc:333] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.493754   431 raft_consensus.cc:2243] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.494766   431 raft_consensus.cc:2272] T 93ef45e97cf24863a576b705ec8e07bd P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.495705  1688 consensus_peers.cc:597] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:05.498207   431 tablet_replica.cc:333] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.499070   431 raft_consensus.cc:2243] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:05:05.500133  2342 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.500901   431 raft_consensus.cc:2272] T fb755eb990bc40778167e79183a40b84 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.503408   431 tablet_replica.cc:333] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.504181   431 raft_consensus.cc:2243] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.505452   431 raft_consensus.cc:2272] T da3ba11d664442d7b28bc849c61fdcdf P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.508031   431 tablet_replica.cc:333] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.508703   431 raft_consensus.cc:2243] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.509740   431 raft_consensus.cc:2272] T 5fa278fc63cf4f6e8ca5de6992849e6e P 58cdb682fbf54792a702a7a455f265a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.511771   431 tablet_replica.cc:333] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.512406   431 raft_consensus.cc:2243] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.512993   431 raft_consensus.cc:2272] T c47fb1f3610c496ea3c5542634768fa6 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.515221   431 tablet_replica.cc:333] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.515846   431 raft_consensus.cc:2243] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.517462   431 raft_consensus.cc:2272] T 996f59fd8f2340a49403b3e9ef4c8e86 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.520471   431 tablet_replica.cc:333] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.521366   431 raft_consensus.cc:2243] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.522641   431 raft_consensus.cc:2272] T de2b34490a3844168645e5b940ba01a9 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.525655   431 tablet_replica.cc:333] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.526567   431 raft_consensus.cc:2243] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.527802   431 raft_consensus.cc:2272] T 20085fd5893f43378bf1c59b14c351c0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.530179  1690 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:05.530632   431 tablet_replica.cc:333] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.531553   431 raft_consensus.cc:2243] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.532228   431 raft_consensus.cc:2272] T b8513b00817042619ae0bf0c8da58904 P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.534793  1688 consensus_peers.cc:597] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:05.535066   431 tablet_replica.cc:333] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.536088   431 raft_consensus.cc:2243] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.538064   431 raft_consensus.cc:2272] T 740ac262ab184918992607d66446b8f3 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.540990   431 tablet_replica.cc:333] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.541839   431 raft_consensus.cc:2243] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.542461   431 raft_consensus.cc:2272] T 1be748f6ba29434fbee8b14cedc1d192 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.545269   431 tablet_replica.cc:333] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.546034   431 raft_consensus.cc:2243] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:05:05.545922  2342 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.547221   431 raft_consensus.cc:2272] T 30619dafd05d4e36a03ff8f500c3a38c P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.549539   431 tablet_replica.cc:333] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.550203   431 raft_consensus.cc:2243] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.551327   431 raft_consensus.cc:2272] T 1acf73e529394004842219637cff7cfc P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.553757   431 tablet_replica.cc:333] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.554595   431 raft_consensus.cc:2243] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.555917   431 raft_consensus.cc:2272] T 8014e1ef10d84a9f9c4d676b56c5105d P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.558558   431 tablet_replica.cc:333] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.559158   431 raft_consensus.cc:2243] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.560287   431 raft_consensus.cc:2272] T 7778fc5cf1c746c4a4a974883a271ce0 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.562371   431 tablet_replica.cc:333] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.562940   431 raft_consensus.cc:2243] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.563851   431 raft_consensus.cc:2272] T 9bb9de7d85a2414bb920f63ec26f8686 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.566320   431 tablet_replica.cc:333] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.567039   431 raft_consensus.cc:2243] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.568303   431 raft_consensus.cc:2272] T d7e1643d2c2f4a96a57a8bae29e7f19a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.570143  1688 consensus_peers.cc:597] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:05.570971   431 tablet_replica.cc:333] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.571621   431 raft_consensus.cc:2243] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.572736   431 raft_consensus.cc:2272] T d5ac82307bd14706ab03f881a279f3e5 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.574944   431 tablet_replica.cc:333] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.575763   431 raft_consensus.cc:2243] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.576885   431 raft_consensus.cc:2272] T 38bef56447d743db91bd1f59274db995 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.579028   431 tablet_replica.cc:333] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.579600   431 raft_consensus.cc:2243] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.580622   431 raft_consensus.cc:2272] T 08c9dfcb3286481dbbb396ccf74ada24 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.582829   431 tablet_replica.cc:333] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.583426   431 raft_consensus.cc:2243] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.584525   431 raft_consensus.cc:2272] T 7011d2943a2040ccb1a34122c0fd3ac3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.586654   431 tablet_replica.cc:333] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.587417   431 raft_consensus.cc:2243] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.588564   431 raft_consensus.cc:2272] T fd4d3a0e164248a5be4b2d19407551af P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.591420   431 tablet_replica.cc:333] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.592221   431 raft_consensus.cc:2243] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.593006   431 raft_consensus.cc:2272] T e8efe0ac5e5e45d8acfa1dfe55172859 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.595569   431 tablet_replica.cc:333] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.596454   431 raft_consensus.cc:2243] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.597098   431 raft_consensus.cc:2272] T 2c501b3d8f86404e991cec1222bd0825 P 58cdb682fbf54792a702a7a455f265a2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.600726   431 tablet_replica.cc:333] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.601482   431 raft_consensus.cc:2243] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.602674   431 raft_consensus.cc:2272] T c55c47030a294c17ad5561abe541b993 P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.605321   431 tablet_replica.cc:333] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.605887   431 raft_consensus.cc:2243] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.606351   431 raft_consensus.cc:2272] T 87baee349cdd48d88642bf37176b1b26 P 58cdb682fbf54792a702a7a455f265a2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.608520   431 tablet_replica.cc:333] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.609416   431 raft_consensus.cc:2243] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.610167   431 raft_consensus.cc:2272] T 44cb1d5977df435cac936d86c5bd1880 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.613121   431 tablet_replica.cc:333] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
W20260709 13:05:05.613765  1688 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:05.614030   431 raft_consensus.cc:2243] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.615197   431 raft_consensus.cc:2272] T 9c347beda1704867ba2582a7a34d255b P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.618052   431 tablet_replica.cc:333] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.618816   431 raft_consensus.cc:2243] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.619482   431 raft_consensus.cc:2272] T 27e7f73e79344731aa22e748b700ffd9 P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.622227   431 tablet_replica.cc:333] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
W20260709 13:05:05.622232  2342 consensus_peers.cc:597] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:05:05.623190   431 raft_consensus.cc:2243] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.624135   431 raft_consensus.cc:2272] T 1d22dc39228b4001961709e74ef6bc9c P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.626565   431 tablet_replica.cc:333] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.627167   431 raft_consensus.cc:2243] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.628242   431 raft_consensus.cc:2272] T 6834f8da7764493a94dc534e4471b3dc P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.630565   431 tablet_replica.cc:333] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.631151   431 raft_consensus.cc:2243] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.632679   431 raft_consensus.cc:2272] T 48efc21b583b4e7fa81ec24726402281 P 58cdb682fbf54792a702a7a455f265a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.635032   431 tablet_replica.cc:333] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.635849   431 raft_consensus.cc:2243] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.637198   431 raft_consensus.cc:2272] T 4d5536a0ca5d48e4b98e0084e0aefb70 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.637755  2401 debug-util.cc:398] Leaking SignalData structure 0x7b08001b15c0 after lost signal to thread 434
W20260709 13:05:05.638931  2401 debug-util.cc:398] Leaking SignalData structure 0x7b080027c520 after lost signal to thread 1508
I20260709 13:05:05.640084   431 tablet_replica.cc:333] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
W20260709 13:05:05.640331  2401 debug-util.cc:398] Leaking SignalData structure 0x7b0800055520 after lost signal to thread 1749
I20260709 13:05:05.640913   431 raft_consensus.cc:2243] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:05.641640  2401 debug-util.cc:398] Leaking SignalData structure 0x7b0800284940 after lost signal to thread 2404
I20260709 13:05:05.641839   431 raft_consensus.cc:2272] T e69630519d3c48eca350b04183abca32 P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.644753   431 tablet_replica.cc:333] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.645555   431 raft_consensus.cc:2243] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.646920   431 raft_consensus.cc:2272] T 81bce6c3bf1547e4b8d33586cafe5f6c P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.650059   431 tablet_replica.cc:333] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.650929   431 raft_consensus.cc:2243] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.651800   431 raft_consensus.cc:2272] T 6bd1d3ea2c984a999cdd42423ebec16f P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.654486   431 tablet_replica.cc:333] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.655260   431 raft_consensus.cc:2243] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.656630   431 raft_consensus.cc:2272] T dde38eb3bd9341aa90d41a6f21bb29a1 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.657441  1690 consensus_peers.cc:597] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:05.659348   431 tablet_replica.cc:333] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.659991   431 raft_consensus.cc:2243] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.661392   431 raft_consensus.cc:2272] T 414132da4f974e5c912e5d5dfa077654 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.663636   431 tablet_replica.cc:333] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.664229   431 raft_consensus.cc:2243] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.665467   431 raft_consensus.cc:2272] T 9273554b0f40499d928621ee1bccc9d8 P 58cdb682fbf54792a702a7a455f265a2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.667608   431 tablet_replica.cc:333] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.668154   431 raft_consensus.cc:2243] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.669219   431 raft_consensus.cc:2272] T 5938eec8e53f4d08b26ec3011f1cdbfc P 58cdb682fbf54792a702a7a455f265a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.671434   431 tablet_replica.cc:333] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.671981   431 raft_consensus.cc:2243] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.672465   431 raft_consensus.cc:2272] T b837a5ec9a0d4f2a8635dedf45d51aa2 P 58cdb682fbf54792a702a7a455f265a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.674590   431 tablet_replica.cc:333] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.675079   431 raft_consensus.cc:2243] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.676195   431 raft_consensus.cc:2272] T fb93f4d4caa2474bb5d42f9940f0dcdc P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.678413   431 tablet_replica.cc:333] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.679024   431 raft_consensus.cc:2243] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.680759   431 raft_consensus.cc:2272] T f6d77c909fe243b197ef61ea6b271908 P 58cdb682fbf54792a702a7a455f265a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.683034   431 tablet_replica.cc:333] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.683612   431 raft_consensus.cc:2243] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.684803   431 raft_consensus.cc:2272] T 54bb54b92d594aa7a2d8f8402e20c0ca P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.687028   431 tablet_replica.cc:333] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.687569   431 raft_consensus.cc:2243] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.688671   431 raft_consensus.cc:2272] T 57592deb11fc4830921f01c2a3c32c4a P 58cdb682fbf54792a702a7a455f265a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.689759  9642 raft_consensus.cc:493] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:05.690367  9642 raft_consensus.cc:515] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:05.691407   431 tablet_replica.cc:333] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.692312   431 raft_consensus.cc:2243] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.693709   431 raft_consensus.cc:2272] T 54db7417c3014ba3948e6ce733254ba3 P 58cdb682fbf54792a702a7a455f265a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.696681   431 tablet_replica.cc:333] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.697388   431 raft_consensus.cc:2243] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:05.698676   431 raft_consensus.cc:2272] T 42b29e0e5e884027ab75497ae8e020bf P 58cdb682fbf54792a702a7a455f265a2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.701725   431 tablet_replica.cc:333] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
I20260709 13:05:05.702487   431 raft_consensus.cc:2243] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.703578   431 raft_consensus.cc:2272] T 53c9e58595214defa013274c5193de1a P 58cdb682fbf54792a702a7a455f265a2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:05.704269  9642 leader_election.cc:290] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:05:05.706537   431 tablet_replica.cc:333] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2: stopping tablet replica
W20260709 13:05:05.704573  2344 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:05.707297   431 raft_consensus.cc:2243] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:05.708000   431 raft_consensus.cc:2272] T 01c1afc8c26047b0b3d8c006f9df5a30 P 58cdb682fbf54792a702a7a455f265a2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:05.710307  2342 leader_election.cc:336] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:05.710829  2342 leader_election.cc:304] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:05.712237  9642 raft_consensus.cc:2749] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:05.737708  2342 consensus_peers.cc:597] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:05.787120  2344 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:05.830546  2342 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:05.918258  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Couldn't send request to peer 58cdb682fbf54792a702a7a455f265a2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:06.289502  2342 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:06.292742  2344 consensus_peers.cc:597] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:05:06.299028  2342 consensus_peers.cc:597] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:06.314601  2342 consensus_peers.cc:597] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:06.504594  2344 consensus_peers.cc:597] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:05:06.618284   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:06.627377  9651 raft_consensus.cc:493] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.628240  9651 raft_consensus.cc:515] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.634155  9651 leader_election.cc:290] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.659087  2344 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:06.668395  9651 raft_consensus.cc:493] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.669348  9651 raft_consensus.cc:515] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.672312  9651 leader_election.cc:290] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:05:06.673763  9655 raft_consensus.cc:493] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.675279  9655 raft_consensus.cc:515] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.680471  9655 leader_election.cc:290] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.689690  2344 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:06.713176  2342 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Couldn't send request to peer cd6d377efe514bc58ae49c3f4a107232. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:05:06.716080  2342 leader_election.cc:336] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.717119  2342 leader_election.cc:304] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:05:06.718643  9651 raft_consensus.cc:2749] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:06.721284  9651 raft_consensus.cc:493] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.722229  9651 raft_consensus.cc:515] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:05:06.737093  2344 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:06.738626  9651 leader_election.cc:290] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:05:06.744480  9655 raft_consensus.cc:493] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.745206  9655 raft_consensus.cc:515] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.747931  9655 leader_election.cc:290] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:05:06.743646  9651 raft_consensus.cc:493] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.751554  9651 raft_consensus.cc:515] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.756489  9655 raft_consensus.cc:493] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.770746  9655 raft_consensus.cc:515] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.774487  9655 leader_election.cc:290] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.776310  2344 consensus_peers.cc:597] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:06.778326  9651 leader_election.cc:290] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.787283  2342 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:06.789687  2342 leader_election.cc:336] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.790411  2342 leader_election.cc:304] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.791427  9651 raft_consensus.cc:2749] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:06.793169  2342 leader_election.cc:336] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.794152  2342 leader_election.cc:304] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:05:06.795442  9651 raft_consensus.cc:2749] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:06.812930  2342 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:06.825543   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
W20260709 13:05:06.828711  2342 leader_election.cc:336] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.829541  2342 leader_election.cc:304] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:05:06.830678  9651 raft_consensus.cc:2749] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:06.832485  2342 leader_election.cc:336] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.833211  2342 leader_election.cc:304] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.834828  9651 raft_consensus.cc:2749] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:06.848292  2342 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
W20260709 13:05:06.849207  2342 leader_election.cc:336] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
W20260709 13:05:06.851498  2342 leader_election.cc:336] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:06.855793  2342 leader_election.cc:304] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.857831  9651 raft_consensus.cc:493] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.859401  9655 raft_consensus.cc:2749] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:06.859002  9651 raft_consensus.cc:515] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
W20260709 13:05:06.865003  2342 leader_election.cc:336] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:06.865639  2342 leader_election.cc:304] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.868944  9655 raft_consensus.cc:2749] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:06.871364  9651 leader_election.cc:290] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.879011  2344 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:06.883546  2344 consensus_peers.cc:597] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:05:06.881145  9651 raft_consensus.cc:493] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.886041  9655 raft_consensus.cc:493] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd6d377efe514bc58ae49c3f4a107232)
I20260709 13:05:06.888713  9655 raft_consensus.cc:515] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.888067  9651 raft_consensus.cc:515] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } attrs { replace: true } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.890769  9655 leader_election.cc:290] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325)
I20260709 13:05:06.893394  9651 leader_election.cc:290] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.898684  2344 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:06.899801  2344 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:06.902319  2342 leader_election.cc:336] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.903106  2342 leader_election.cc:304] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:05:06.904196  9651 raft_consensus.cc:2749] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:05:06.907930  2342 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.908592  2342 leader_election.cc:304] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
I20260709 13:05:06.909456  9651 raft_consensus.cc:2749] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:06.910933  2342 leader_election.cc:336] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:06.911679  2342 leader_election.cc:304] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.912846  9651 raft_consensus.cc:2749] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:05:06.913717  2342 leader_election.cc:336] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
I20260709 13:05:06.920110  9651 raft_consensus.cc:493] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.920908  9651 raft_consensus.cc:515] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.924774  9655 raft_consensus.cc:493] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.925657  9655 raft_consensus.cc:515] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd6d377efe514bc58ae49c3f4a107232" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43325 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:06.926962  9651 leader_election.cc:290] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:05:06.928131  9655 leader_election.cc:290] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
W20260709 13:05:06.969650  2342 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:06.970392  2342 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd6d377efe514bc58ae49c3f4a107232 (127.0.107.194:43325): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43325: connect: Connection refused (error 111)
W20260709 13:05:06.971169  2342 leader_election.cc:336] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.971544  2342 leader_election.cc:304] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.972329  9651 raft_consensus.cc:2749] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:05:06.972636  2342 leader_election.cc:336] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:06.973119  2342 leader_election.cc:304] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 58cdb682fbf54792a702a7a455f265a2, cd6d377efe514bc58ae49c3f4a107232
I20260709 13:05:06.973958  9651 raft_consensus.cc:2749] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:06.989681   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:06.991180   431 tablet_replica.cc:333] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:06.992686   431 raft_consensus.cc:2243] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:06.995172   431 raft_consensus.cc:2272] T 740ac262ab184918992607d66446b8f3 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:06.996172  9651 raft_consensus.cc:493] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 58cdb682fbf54792a702a7a455f265a2)
I20260709 13:05:06.997588  9651 raft_consensus.cc:515] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4f737473248e4c059fa74af3b8e1f77e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45275 } } peers { permanent_uuid: "58cdb682fbf54792a702a7a455f265a2" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44281 } } peers { permanent_uuid: "da3b57321031416cba9db75450ef302a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 45309 } attrs { promote: false } }
I20260709 13:05:07.007295   431 tablet_replica.cc:333] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.008777   431 raft_consensus.cc:2243] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:05:07.009503  2344 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:07.010607   431 raft_consensus.cc:2272] T 996f59fd8f2340a49403b3e9ef4c8e86 P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.011202  9651 leader_election.cc:290] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275), 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281)
I20260709 13:05:07.015111   431 tablet_replica.cc:333] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.016252   431 raft_consensus.cc:2243] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.017789   431 raft_consensus.cc:2272] T ba20622bf78343c5bf6ac194af441026 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.021008   431 tablet_replica.cc:333] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.022037   431 raft_consensus.cc:2243] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.023785   431 raft_consensus.cc:2272] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:07.023962  2342 leader_election.cc:336] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 58cdb682fbf54792a702a7a455f265a2 (127.0.107.195:44281): Network error: Client connection negotiation failed: client connection to 127.0.107.195:44281: connect: Connection refused (error 111)
I20260709 13:05:07.024928  2342 leader_election.cc:304] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3b57321031416cba9db75450ef302a; no voters: 4f737473248e4c059fa74af3b8e1f77e, 58cdb682fbf54792a702a7a455f265a2
W20260709 13:05:07.025923  2342 raft_consensus.cc:2686] T 81bce6c3bf1547e4b8d33586cafe5f6c P da3b57321031416cba9db75450ef302a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:05:07.027949   431 tablet_replica.cc:333] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.029313   431 raft_consensus.cc:2243] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.030608   431 raft_consensus.cc:2272] T f048890019da43548110ab19fce7a878 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.034901   431 tablet_replica.cc:333] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.036141   431 raft_consensus.cc:2243] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.037832   431 raft_consensus.cc:2272] T dde38eb3bd9341aa90d41a6f21bb29a1 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.042397   431 tablet_replica.cc:333] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.043686   431 raft_consensus.cc:2243] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.044715   431 raft_consensus.cc:2272] T 414132da4f974e5c912e5d5dfa077654 P da3b57321031416cba9db75450ef302a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.048636   431 tablet_replica.cc:333] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.049666   431 raft_consensus.cc:2243] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.050990   431 raft_consensus.cc:2272] T 4391f9a46a9542a7bf726b05393b7609 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.055392   431 tablet_replica.cc:333] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.056315   431 raft_consensus.cc:2243] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.057416   431 raft_consensus.cc:2272] T 53c9e58595214defa013274c5193de1a P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.062139   431 tablet_replica.cc:333] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.063675   431 raft_consensus.cc:2243] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.065081   431 raft_consensus.cc:2272] T f3646214b1a14514955e904e05fadcf1 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.068831   431 tablet_replica.cc:333] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.069913   431 raft_consensus.cc:2243] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.071034   431 raft_consensus.cc:2272] T 1acf73e529394004842219637cff7cfc P da3b57321031416cba9db75450ef302a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.076635   431 tablet_replica.cc:333] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.078591   431 raft_consensus.cc:2243] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.079420   431 raft_consensus.cc:2272] T 6834f8da7764493a94dc534e4471b3dc P da3b57321031416cba9db75450ef302a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.083309   431 tablet_replica.cc:333] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.084167   431 raft_consensus.cc:2243] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.085832   431 raft_consensus.cc:2272] T 26917aa1f55f48918fedd4960446f38c P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.089531   431 tablet_replica.cc:333] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.090520   431 raft_consensus.cc:2243] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.092082   431 raft_consensus.cc:2272] T 363173567dd940e99e459f02148a42e0 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.096246   431 tablet_replica.cc:333] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a: stopping tablet replica
W20260709 13:05:07.096539  2344 consensus_peers.cc:597] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a -> Peer 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): Couldn't send request to peer 4f737473248e4c059fa74af3b8e1f77e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:05:07.097848   431 raft_consensus.cc:2243] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.099105   431 raft_consensus.cc:2272] T 5938eec8e53f4d08b26ec3011f1cdbfc P da3b57321031416cba9db75450ef302a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.103790   431 tablet_replica.cc:333] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.104773   431 raft_consensus.cc:2243] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.106500   431 raft_consensus.cc:2272] T 78d12cc2bc7b4654acd41e6c45eb9753 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.110383   431 tablet_replica.cc:333] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.111334   431 raft_consensus.cc:2243] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.112413   431 raft_consensus.cc:2272] T 08c9dfcb3286481dbbb396ccf74ada24 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.116472   431 tablet_replica.cc:333] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.118036   431 raft_consensus.cc:2243] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.119421   431 raft_consensus.cc:2272] T fb93f4d4caa2474bb5d42f9940f0dcdc P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.123942   431 tablet_replica.cc:333] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.124910   431 raft_consensus.cc:2243] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.126435   431 raft_consensus.cc:2272] T b837a5ec9a0d4f2a8635dedf45d51aa2 P da3b57321031416cba9db75450ef302a [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.130038   431 tablet_replica.cc:333] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.130923   431 raft_consensus.cc:2243] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.133297   431 raft_consensus.cc:2272] T 62836cb7c4a548b5a6c44c89782a958c P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.137084   431 tablet_replica.cc:333] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.137908   431 raft_consensus.cc:2243] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.138568   431 raft_consensus.cc:2272] T f6d77c909fe243b197ef61ea6b271908 P da3b57321031416cba9db75450ef302a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.141875   431 tablet_replica.cc:333] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.142620   431 raft_consensus.cc:2243] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.144140   431 raft_consensus.cc:2272] T fb755eb990bc40778167e79183a40b84 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.148172   431 tablet_replica.cc:333] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.149137   431 raft_consensus.cc:2243] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.150081   431 raft_consensus.cc:2272] T c55c47030a294c17ad5561abe541b993 P da3b57321031416cba9db75450ef302a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.153640   431 tablet_replica.cc:333] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.155175   431 raft_consensus.cc:2243] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.157338   431 raft_consensus.cc:2272] T 01c1afc8c26047b0b3d8c006f9df5a30 P da3b57321031416cba9db75450ef302a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.161685   431 tablet_replica.cc:333] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.162405   431 raft_consensus.cc:2243] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.163342   431 raft_consensus.cc:2272] T fac0a563dc584b4fbe82f01f362fa966 P da3b57321031416cba9db75450ef302a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.166540   431 tablet_replica.cc:333] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a: stopping tablet replica
I20260709 13:05:07.167263   431 raft_consensus.cc:2243] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:07.168606   431 raft_consensus.cc:2272] T b8513b00817042619ae0bf0c8da58904 P da3b57321031416cba9db75450ef302a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.955876   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:05:08.030148   431 master.cc:562] Master@127.0.107.254:32927 shutting down...
W20260709 13:05:08.095238  1522 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:05:08.095685  1522 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:08.095996  1522 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:12.802217  1460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111) [suppressed 109 similar messages]
W20260709 13:05:12.806574  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1acf73e529394004842219637cff7cfc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
W20260709 13:05:13.869124  1460 catalog_manager.cc:4755] TS 4f737473248e4c059fa74af3b8e1f77e (127.0.107.193:45275): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dde38eb3bd9341aa90d41a6f21bb29a1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45275: connect: Connection refused (error 111)
I20260709 13:05:15.090958   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:15.092590   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:15.093050   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P d61e0c3f41244926bf144dca0f9acf61: stopping tablet replica
I20260709 13:05:15.298139   431 master.cc:584] Master@127.0.107.254:32927 shutdown complete.
I20260709 13:05:15.368002   431 test_util.cc:182] -----------------------------------------------
I20260709 13:05:15.368351   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602002558854-431-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (286624 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:05:15.372884   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:37093
I20260709 13:05:15.374339   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:15.382146  9662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:15.382217  9663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:15.382941  9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:15.385105   431 server_base.cc:1061] running on GCE node
I20260709 13:05:15.390671   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:15.391016   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:15.391428   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602315391392 us; error 0 us; skew 500 ppm
I20260709 13:05:15.392320   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:15.395581   431 webserver.cc:533] Webserver started at http://127.0.107.254:36493/ using document root <none> and password file <none>
I20260709 13:05:15.396184   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:15.396466   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:15.396775   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:15.398329   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "7ac5359d00f54e37bd8a40ec7cebcc55"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.404704   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:05:15.410404  9670 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.411537   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:05:15.411983   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "7ac5359d00f54e37bd8a40ec7cebcc55"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.412570   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:15.469866   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:15.471524   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:15.530736   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:37093
I20260709 13:05:15.530851  9721 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:37093 every 8 connection(s)
I20260709 13:05:15.536587  9722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:15.555558  9722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55: Bootstrap starting.
I20260709 13:05:15.563093  9722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.568945  9722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55: No bootstrap required, opened a new log
I20260709 13:05:15.571394  9722 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" member_type: VOTER }
I20260709 13:05:15.571925  9722 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.572607  9722 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac5359d00f54e37bd8a40ec7cebcc55, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.573534  9722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" member_type: VOTER }
I20260709 13:05:15.574152  9722 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:15.574544  9722 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:15.575133  9722 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.582104  9722 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" member_type: VOTER }
I20260709 13:05:15.582881  9722 leader_election.cc:304] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ac5359d00f54e37bd8a40ec7cebcc55; no voters: 
I20260709 13:05:15.584794  9722 leader_election.cc:290] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:15.585196  9725 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.586625  9725 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 1 LEADER]: Becoming Leader. State: Replica: 7ac5359d00f54e37bd8a40ec7cebcc55, State: Running, Role: LEADER
I20260709 13:05:15.587482  9725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" member_type: VOTER }
I20260709 13:05:15.588795  9722 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:15.596949  9727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ac5359d00f54e37bd8a40ec7cebcc55. Latest consensus state: current_term: 1 leader_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" member_type: VOTER } }
I20260709 13:05:15.597174  9726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac5359d00f54e37bd8a40ec7cebcc55" member_type: VOTER } }
I20260709 13:05:15.598100  9727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:15.598364  9726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:15.601105  9735 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:15.611238  9735 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:15.612782   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:15.624506  9735 catalog_manager.cc:1383] Generated new cluster ID: 8727dd7338ba4836b586b13127bdaa79
I20260709 13:05:15.625029  9735 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:15.650867  9735 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:15.652573  9735 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:15.670367  9735 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55: Generated new TSK 0
I20260709 13:05:15.671167  9735 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:15.681627   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:15.691190  9743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:15.691857  9744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:15.695571   431 server_base.cc:1061] running on GCE node
W20260709 13:05:15.695979  9746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:15.699508   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:15.699887   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:15.700179   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602315700133 us; error 0 us; skew 500 ppm
I20260709 13:05:15.701228   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:15.704769   431 webserver.cc:533] Webserver started at http://127.0.107.193:44477/ using document root <none> and password file <none>
I20260709 13:05:15.705382   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:15.705642   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:15.705979   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:15.707674   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "e8769fa7061d433bb90a9411c71fa0df"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.714223   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:05:15.720321  9751 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.721537   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:05:15.721961   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "e8769fa7061d433bb90a9411c71fa0df"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.722450   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:15.748791   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:15.750438   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:15.752882   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:15.756052   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:15.756412   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.756742   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:15.756948   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.813020   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:32801
I20260709 13:05:15.813149  9813 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:32801 every 8 connection(s)
I20260709 13:05:15.820571   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:15.830664  9818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:15.832012  9819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:15.836056  9821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:15.836531   431 server_base.cc:1061] running on GCE node
I20260709 13:05:15.840325  9814 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37093
I20260709 13:05:15.840956  9814 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:15.841111   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:15.841619   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:15.841868   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602315841844 us; error 0 us; skew 500 ppm
I20260709 13:05:15.842425  9814 heartbeater.cc:507] Master 127.0.107.254:37093 requested a full tablet report, sending...
I20260709 13:05:15.842694   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:15.845992  9687 ts_manager.cc:194] Registered new tserver with Master: e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:15.846566   431 webserver.cc:533] Webserver started at http://127.0.107.194:46097/ using document root <none> and password file <none>
I20260709 13:05:15.847416   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:15.847707   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:15.848109   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:15.849529  9687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33594
I20260709 13:05:15.850607   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "24c6a28138294e46bf6b3d5ba324046d"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.857108   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:05:15.861941  9826 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.863125   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:05:15.863574   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "24c6a28138294e46bf6b3d5ba324046d"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.864171   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:15.885437   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:15.887193   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:15.889221   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:15.892165   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:15.892537   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:05:15.892827   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:15.893033   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:15.954671   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:32919
I20260709 13:05:15.954835  9888 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:32919 every 8 connection(s)
I20260709 13:05:15.961910   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:15.972615  9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:15.973675  9893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:15.977358   431 server_base.cc:1061] running on GCE node
W20260709 13:05:15.979384  9895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:15.980099  9889 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37093
I20260709 13:05:15.980715  9889 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:15.980844   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:15.981233   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:15.981442   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602315981423 us; error 0 us; skew 500 ppm
I20260709 13:05:15.981855  9889 heartbeater.cc:507] Master 127.0.107.254:37093 requested a full tablet report, sending...
I20260709 13:05:15.982178   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:15.985352  9687 ts_manager.cc:194] Registered new tserver with Master: 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:15.986268   431 webserver.cc:533] Webserver started at http://127.0.107.195:33689/ using document root <none> and password file <none>
I20260709 13:05:15.987162   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:15.987458   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:15.987735  9687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33610
I20260709 13:05:15.987951   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:15.990002   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "d02e8a054a634be599149db641901736"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:15.995993   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:05:16.001283  9900 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.002780   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:05:16.003394   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "d02e8a054a634be599149db641901736"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:16.003759   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:16.017035   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:16.018391   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:16.020053   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:16.022707   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:16.022949   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.023201   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:16.023367   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.084860   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:37607
I20260709 13:05:16.084946  9962 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:37607 every 8 connection(s)
I20260709 13:05:16.091357   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:16.102092  9966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:16.104591  9967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:16.108747  9969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:16.108819   431 server_base.cc:1061] running on GCE node
I20260709 13:05:16.110512   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:05:16.110347  9963 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37093
W20260709 13:05:16.110996   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:16.111317   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602316111297 us; error 0 us; skew 500 ppm
I20260709 13:05:16.111407  9963 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:16.112181   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:16.112856  9963 heartbeater.cc:507] Master 127.0.107.254:37093 requested a full tablet report, sending...
I20260709 13:05:16.115949  9687 ts_manager.cc:194] Registered new tserver with Master: d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:16.116735   431 webserver.cc:533] Webserver started at http://127.0.107.196:44953/ using document root <none> and password file <none>
I20260709 13:05:16.117412   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:16.117666   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:16.117988   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:16.118371  9687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33620
I20260709 13:05:16.119767   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-3-root/instance:
uuid: "5e087ae2ce3f417093313edd26b8b3ee"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-ftjc"
I20260709 13:05:16.126238   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:16.131049  9974 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.132447   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:05:16.132872   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-3-root
uuid: "5e087ae2ce3f417093313edd26b8b3ee"
format_stamp: "Formatted at 2026-07-09 13:05:16 on dist-test-slave-ftjc"
I20260709 13:05:16.133302   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:16.157650   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:16.159358   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:16.161336   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:16.164333   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:16.164707   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:05:16.164986   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:16.165176   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:16.225381   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:38131
I20260709 13:05:16.225493 10036 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:38131 every 8 connection(s)
I20260709 13:05:16.243356 10037 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37093
I20260709 13:05:16.243848 10037 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:16.244967 10037 heartbeater.cc:507] Master 127.0.107.254:37093 requested a full tablet report, sending...
I20260709 13:05:16.247673  9687 ts_manager.cc:194] Registered new tserver with Master: 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:16.247872   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.018458861s
I20260709 13:05:16.249814  9687 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33634
I20260709 13:05:16.290148  9687 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33650:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:05:16.587787 10002 tablet_service.cc:1511] Processing CreateTablet for tablet c48883428dea42d7b95a7cddbc07178e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:16.589958 10002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c48883428dea42d7b95a7cddbc07178e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.594491  9854 tablet_service.cc:1511] Processing CreateTablet for tablet c48883428dea42d7b95a7cddbc07178e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:16.596586  9854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c48883428dea42d7b95a7cddbc07178e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.598968 10001 tablet_service.cc:1511] Processing CreateTablet for tablet 1131589c5bd9480d838a7967131195ca (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:16.600863  9928 tablet_service.cc:1511] Processing CreateTablet for tablet c48883428dea42d7b95a7cddbc07178e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:16.602587  9928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c48883428dea42d7b95a7cddbc07178e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.602080  9927 tablet_service.cc:1511] Processing CreateTablet for tablet d775d5d3993743689257e4509a059770 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:16.604300  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d775d5d3993743689257e4509a059770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.606905  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 9625fd7e9ed046158552e5252f1ee3db (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:16.611289 10000 tablet_service.cc:1511] Processing CreateTablet for tablet 9625fd7e9ed046158552e5252f1ee3db (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:16.615320  9853 tablet_service.cc:1511] Processing CreateTablet for tablet d775d5d3993743689257e4509a059770 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:16.615305  9852 tablet_service.cc:1511] Processing CreateTablet for tablet 1131589c5bd9480d838a7967131195ca (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:16.617533  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d775d5d3993743689257e4509a059770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.619027  9925 tablet_service.cc:1511] Processing CreateTablet for tablet a693233cb65a40089d314773895f84a0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:16.623390  9851 tablet_service.cc:1511] Processing CreateTablet for tablet a693233cb65a40089d314773895f84a0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:16.625737  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a693233cb65a40089d314773895f84a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.630127  9852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1131589c5bd9480d838a7967131195ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.613562  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9625fd7e9ed046158552e5252f1ee3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.636857  9778 tablet_service.cc:1511] Processing CreateTablet for tablet 1131589c5bd9480d838a7967131195ca (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:16.637372  9777 tablet_service.cc:1511] Processing CreateTablet for tablet 9625fd7e9ed046158552e5252f1ee3db (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:16.639739  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1131589c5bd9480d838a7967131195ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.637223  9776 tablet_service.cc:1511] Processing CreateTablet for tablet a693233cb65a40089d314773895f84a0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:16.640144  9779 tablet_service.cc:1511] Processing CreateTablet for tablet d775d5d3993743689257e4509a059770 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:16.647325  9775 tablet_service.cc:1511] Processing CreateTablet for tablet 4a08704c3c3c4030bb48886795597fe1 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:16.647192  9999 tablet_service.cc:1511] Processing CreateTablet for tablet 4a08704c3c3c4030bb48886795597fe1 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:16.654410  9850 tablet_service.cc:1511] Processing CreateTablet for tablet 4a08704c3c3c4030bb48886795597fe1 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:16.666712  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 357a7ca5934d4fb6958de3cca90f5a36 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:16.666712  9774 tablet_service.cc:1511] Processing CreateTablet for tablet 357a7ca5934d4fb6958de3cca90f5a36 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:16.657055  9925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a693233cb65a40089d314773895f84a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.679283  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 69af1a39e3934236883aaa506f999dee (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
W20260709 13:05:16.605255  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:16.659230  9924 tablet_service.cc:1511] Processing CreateTablet for tablet 357a7ca5934d4fb6958de3cca90f5a36 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:16.601341 10001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1131589c5bd9480d838a7967131195ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.681468 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9625fd7e9ed046158552e5252f1ee3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.681717  9924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357a7ca5934d4fb6958de3cca90f5a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.675138  9849 tablet_service.cc:1511] Processing CreateTablet for tablet 1cfdee35c2f0491da78cbc51ed63ba5d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:16.682345  9997 tablet_service.cc:1511] Processing CreateTablet for tablet 1cfdee35c2f0491da78cbc51ed63ba5d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:16.684141  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357a7ca5934d4fb6958de3cca90f5a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.686131  9923 tablet_service.cc:1511] Processing CreateTablet for tablet 1cfdee35c2f0491da78cbc51ed63ba5d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:16.688369  9923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cfdee35c2f0491da78cbc51ed63ba5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.689196  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a08704c3c3c4030bb48886795597fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.691169  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cfdee35c2f0491da78cbc51ed63ba5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.702365  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a693233cb65a40089d314773895f84a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.704816  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69af1a39e3934236883aaa506f999dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.706598  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a08704c3c3c4030bb48886795597fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.708962  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cfdee35c2f0491da78cbc51ed63ba5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.711525  9777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9625fd7e9ed046158552e5252f1ee3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.715591  9775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a08704c3c3c4030bb48886795597fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.718446  9779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d775d5d3993743689257e4509a059770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.720890  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357a7ca5934d4fb6958de3cca90f5a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.732990 10058 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:16.749773  9848 tablet_service.cc:1511] Processing CreateTablet for tablet 69af1a39e3934236883aaa506f999dee (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:16.763070  9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69af1a39e3934236883aaa506f999dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.770118 10058 tablet_bootstrap.cc:654] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.786187  9924 tablet_service.cc:1511] Processing CreateTablet for tablet 69af1a39e3934236883aaa506f999dee (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:16.786325  9996 tablet_service.cc:1511] Processing CreateTablet for tablet 877ba104634d4068a2f26aa62d860b7b (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:16.788304  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 877ba104634d4068a2f26aa62d860b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.788304  9924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69af1a39e3934236883aaa506f999dee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.825505 10057 tablet_bootstrap.cc:492] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:16.846061 10059 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:16.854434  9814 heartbeater.cc:499] Master 127.0.107.254:37093 was elected leader, sending a full tablet report...
I20260709 13:05:16.880271 10058 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:16.881079 10058 ts_tablet_manager.cc:1403] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.149s	user 0.034s	sys 0.013s
I20260709 13:05:16.899760 10057 tablet_bootstrap.cc:654] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.927793  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 877ba104634d4068a2f26aa62d860b7b (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:16.929531  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 877ba104634d4068a2f26aa62d860b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.929255  9852 tablet_service.cc:1511] Processing CreateTablet for tablet 877ba104634d4068a2f26aa62d860b7b (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:16.931450  9852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 877ba104634d4068a2f26aa62d860b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.942447 10061 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:16.947170  9926 tablet_service.cc:1511] Processing CreateTablet for tablet f8a86072500f497585b83ac124a8a562 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:16.949245  9778 tablet_service.cc:1511] Processing CreateTablet for tablet f8a86072500f497585b83ac124a8a562 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:16.950555  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a86072500f497585b83ac124a8a562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.950776  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a86072500f497585b83ac124a8a562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.940279 10058 raft_consensus.cc:359] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:16.952970 10058 raft_consensus.cc:385] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.953397 10058 raft_consensus.cc:740] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.949064 10000 tablet_service.cc:1511] Processing CreateTablet for tablet f8a86072500f497585b83ac124a8a562 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:16.955593 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a86072500f497585b83ac124a8a562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.956732 10057 tablet_bootstrap.cc:492] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:16.957479 10057 ts_tablet_manager.cc:1403] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.133s	user 0.013s	sys 0.013s
I20260709 13:05:16.961781 10057 raft_consensus.cc:359] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:16.962716 10057 raft_consensus.cc:385] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.951078 10059 tablet_bootstrap.cc:654] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.963039 10057 raft_consensus.cc:740] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.963201 10058 consensus_queue.cc:260] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:16.964319 10057 consensus_queue.cc:260] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:16.980211  9848 tablet_service.cc:1511] Processing CreateTablet for tablet 582cebb9b4674942bb40b7be9936c3f5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:16.985888  9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582cebb9b4674942bb40b7be9936c3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.991917  9889 heartbeater.cc:499] Master 127.0.107.254:37093 was elected leader, sending a full tablet report...
I20260709 13:05:16.993202 10057 ts_tablet_manager.cc:1434] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.035s	user 0.018s	sys 0.013s
I20260709 13:05:16.994261 10037 heartbeater.cc:499] Master 127.0.107.254:37093 was elected leader, sending a full tablet report...
I20260709 13:05:16.994767 10057 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:16.997349  9923 tablet_service.cc:1511] Processing CreateTablet for tablet 582cebb9b4674942bb40b7be9936c3f5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:16.998538 10061 tablet_bootstrap.cc:654] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.004319  9997 tablet_service.cc:1511] Processing CreateTablet for tablet 582cebb9b4674942bb40b7be9936c3f5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:17.006893  9923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582cebb9b4674942bb40b7be9936c3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.016788 10058 ts_tablet_manager.cc:1434] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.135s	user 0.013s	sys 0.003s
I20260709 13:05:17.022825  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582cebb9b4674942bb40b7be9936c3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.025079  9963 heartbeater.cc:499] Master 127.0.107.254:37093 was elected leader, sending a full tablet report...
I20260709 13:05:17.027879  9778 tablet_service.cc:1511] Processing CreateTablet for tablet 4716f93906794e73b45089786b4a75f0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:17.030110  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4716f93906794e73b45089786b4a75f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.027925 10058 tablet_bootstrap.cc:492] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:17.042915  9851 tablet_service.cc:1511] Processing CreateTablet for tablet 4716f93906794e73b45089786b4a75f0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:17.048434 10059 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:17.049058  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4716f93906794e73b45089786b4a75f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.049158 10059 ts_tablet_manager.cc:1403] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.204s	user 0.036s	sys 0.015s
I20260709 13:05:17.050738  9925 tablet_service.cc:1511] Processing CreateTablet for tablet 4716f93906794e73b45089786b4a75f0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:17.052392  9925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4716f93906794e73b45089786b4a75f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.056264 10064 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.056166 10059 raft_consensus.cc:359] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.057319 10064 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.057721  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 47be166313094f0fbe17bde042f05dfe (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:17.059891  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47be166313094f0fbe17bde042f05dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.057716 10059 raft_consensus.cc:385] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.068986 10059 raft_consensus.cc:740] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.070307 10059 consensus_queue.cc:260] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.093057  9774 tablet_service.cc:1511] Processing CreateTablet for tablet 47be166313094f0fbe17bde042f05dfe (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:17.095685  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47be166313094f0fbe17bde042f05dfe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:17.096598  9964 tablet.cc:2406] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:17.084646 10064 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:17.119602  9852 tablet_service.cc:1511] Processing CreateTablet for tablet 47be166313094f0fbe17bde042f05dfe (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:17.122126  9852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47be166313094f0fbe17bde042f05dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.141230 10059 ts_tablet_manager.cc:1434] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.092s	user 0.021s	sys 0.033s
I20260709 13:05:17.142848 10059 tablet_bootstrap.cc:492] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:17.147600 10057 tablet_bootstrap.cc:654] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.150702 10058 tablet_bootstrap.cc:654] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.153118 10059 tablet_bootstrap.cc:654] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.153779  9996 tablet_service.cc:1511] Processing CreateTablet for tablet 4e31a807457a433b9c28f7a67c98095c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:17.155544  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e31a807457a433b9c28f7a67c98095c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.150704  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 4e31a807457a433b9c28f7a67c98095c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:17.159300  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e31a807457a433b9c28f7a67c98095c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.184420 10057 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:17.185184 10057 ts_tablet_manager.cc:1403] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.191s	user 0.024s	sys 0.007s
I20260709 13:05:17.189271 10057 raft_consensus.cc:359] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.190483 10057 raft_consensus.cc:385] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.190831 10057 raft_consensus.cc:740] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.192907 10061 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:17.193481 10061 ts_tablet_manager.cc:1403] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.252s	user 0.014s	sys 0.018s
I20260709 13:05:17.201999 10057 consensus_queue.cc:260] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.204892 10057 ts_tablet_manager.cc:1434] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 13:05:17.206068 10057 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:17.206889 10061 raft_consensus.cc:359] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.207264  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 4e31a807457a433b9c28f7a67c98095c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:17.207831 10061 raft_consensus.cc:385] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.208478 10061 raft_consensus.cc:740] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.209154  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e31a807457a433b9c28f7a67c98095c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.209599 10061 consensus_queue.cc:260] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.216769 10057 tablet_bootstrap.cc:654] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.247303 10058 tablet_bootstrap.cc:492] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:17.247997 10058 ts_tablet_manager.cc:1403] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.221s	user 0.013s	sys 0.016s
I20260709 13:05:17.250340  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:17.251469  9864 raft_consensus.cc:2468] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:17.256279  9851 tablet_service.cc:1511] Processing CreateTablet for tablet e9ea20d35c3b4150ba41cec32b7c9e0c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:17.258069  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ea20d35c3b4150ba41cec32b7c9e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.264102  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:17.252108 10058 raft_consensus.cc:359] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.265810 10058 raft_consensus.cc:385] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.266183 10058 raft_consensus.cc:740] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
W20260709 13:05:17.266453  9901 leader_election.cc:343] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.266309 10000 tablet_service.cc:1511] Processing CreateTablet for tablet e9ea20d35c3b4150ba41cec32b7c9e0c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:17.267211 10058 consensus_queue.cc:260] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.268478 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ea20d35c3b4150ba41cec32b7c9e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.268113  9904 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:17.272959  9923 tablet_service.cc:1511] Processing CreateTablet for tablet e9ea20d35c3b4150ba41cec32b7c9e0c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:17.273826 10059 tablet_bootstrap.cc:492] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:17.281136 10059 ts_tablet_manager.cc:1403] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.139s	user 0.012s	sys 0.016s
I20260709 13:05:17.283852  9776 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2d6cf525014f80a8f95687135fff6f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:17.285614  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2d6cf525014f80a8f95687135fff6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.285501 10059 raft_consensus.cc:359] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.286724 10059 raft_consensus.cc:385] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.286919 10063 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.287176 10059 raft_consensus.cc:740] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.276623 10057 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:17.287760 10063 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.289574 10057 ts_tablet_manager.cc:1403] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.029s
I20260709 13:05:17.293251 10064 raft_consensus.cc:2804] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.298281 10073 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.305864 10073 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.292135 10063 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:17.296504 10064 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.323055 10073 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:17.314126 10058 ts_tablet_manager.cc:1434] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.066s	user 0.012s	sys 0.001s
I20260709 13:05:17.329418  9923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ea20d35c3b4150ba41cec32b7c9e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.325721 10064 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.293870 10059 consensus_queue.cc:260] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.339263 10075 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.345167 10068 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.344071 10075 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.345822 10068 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.327776  9848 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2d6cf525014f80a8f95687135fff6f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:17.348261  9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2d6cf525014f80a8f95687135fff6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.339263 10061 ts_tablet_manager.cc:1434] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.145s	user 0.014s	sys 0.009s
I20260709 13:05:17.363840 10075 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:17.367252 10068 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:17.367693  9778 tablet_service.cc:1511] Processing CreateTablet for tablet 82999b9832894f40bb6dbe02e2feb1df (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:17.369333  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82999b9832894f40bb6dbe02e2feb1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.372470 10058 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:17.378268 10057 raft_consensus.cc:359] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.379616 10057 raft_consensus.cc:385] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.380048 10057 raft_consensus.cc:740] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.381925 10064 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.390856 10057 consensus_queue.cc:260] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.394263 10057 ts_tablet_manager.cc:1434] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.104s	user 0.009s	sys 0.002s
I20260709 13:05:17.395992 10057 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:17.402031 10061 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:17.415719  9925 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2d6cf525014f80a8f95687135fff6f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:17.418973  9925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2d6cf525014f80a8f95687135fff6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.433748 10064 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:17.434625  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 16d91cae140a4615a015d3d34ac878c4 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:17.435413  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:17.436558  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d91cae140a4615a015d3d34ac878c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:17.437618  9901 leader_election.cc:343] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:17.429437  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 162385us
I20260709 13:05:17.464195  9924 tablet_service.cc:1511] Processing CreateTablet for tablet 178dcd0da5414a9785794b2412a9cf93 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:17.443023 10058 tablet_bootstrap.cc:654] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.466435  9924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 178dcd0da5414a9785794b2412a9cf93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.477636 10059 ts_tablet_manager.cc:1434] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.196s	user 0.009s	sys 0.020s
I20260709 13:05:17.459611  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 16d91cae140a4615a015d3d34ac878c4 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:17.485492  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:17.486641  9864 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:17.491179  9978 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:17.492600 10063 raft_consensus.cc:2804] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.496147 10063 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.496680 10063 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.499545 10061 tablet_bootstrap.cc:654] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.514000  9996 tablet_service.cc:1511] Processing CreateTablet for tablet 16d91cae140a4615a015d3d34ac878c4 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:17.514426 10000 tablet_service.cc:1511] Processing CreateTablet for tablet 82999b9832894f40bb6dbe02e2feb1df (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:17.514000  9774 tablet_service.cc:1511] Processing CreateTablet for tablet f0bdd04f0ff44f2e87f76f36a7c5e540 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:17.516197  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d91cae140a4615a015d3d34ac878c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.518622  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0bdd04f0ff44f2e87f76f36a7c5e540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.524835 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82999b9832894f40bb6dbe02e2feb1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.525578  9997 tablet_service.cc:1511] Processing CreateTablet for tablet 178dcd0da5414a9785794b2412a9cf93 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:17.523627  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d91cae140a4615a015d3d34ac878c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.527341  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 178dcd0da5414a9785794b2412a9cf93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.534085 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:17.535130 10012 raft_consensus.cc:2468] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:17.537498  9754 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:17.538888 10073 raft_consensus.cc:2804] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.539376 10073 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.539762 10073 raft_consensus.cc:3060] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.547051 10057 tablet_bootstrap.cc:654] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.523962 10059 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:17.519646 10063 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.569684  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:17.570679  9864 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.571934  9778 tablet_service.cc:1511] Processing CreateTablet for tablet 1078585b45d74186be92d54ac68f787a (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:17.567625  9927 tablet_service.cc:1511] Processing CreateTablet for tablet f0bdd04f0ff44f2e87f76f36a7c5e540 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:17.573841  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0bdd04f0ff44f2e87f76f36a7c5e540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.577519  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:17.578941  9938 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:17.577665  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 1078585b45d74186be92d54ac68f787a (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:17.581324  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1078585b45d74186be92d54ac68f787a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.581959  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1078585b45d74186be92d54ac68f787a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.592892 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:17.593556  9928 tablet_service.cc:1511] Processing CreateTablet for tablet 36f8b3f8e9ff4e1ab10c104b1a3c2da6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:17.595307  9928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f8b3f8e9ff4e1ab10c104b1a3c2da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.594054 10012 raft_consensus.cc:2393] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:17.599292 10063 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:17.600759  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:17.601827  9938 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.602268  9864 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:17.604053  9978 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:17.606406 10063 raft_consensus.cc:2804] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.606889 10073 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.615540 10057 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:17.617908  9999 tablet_service.cc:1511] Processing CreateTablet for tablet 36f8b3f8e9ff4e1ab10c104b1a3c2da6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:17.619737  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f8b3f8e9ff4e1ab10c104b1a3c2da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.622530 10061 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:17.622668 10058 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:17.623551 10061 ts_tablet_manager.cc:1403] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.222s	user 0.015s	sys 0.003s
I20260709 13:05:17.623692 10058 ts_tablet_manager.cc:1403] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.279s	user 0.014s	sys 0.005s
I20260709 13:05:17.631207 10059 tablet_bootstrap.cc:654] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.639221 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:17.668927 10012 raft_consensus.cc:3060] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.611901  9938 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:17.627723 10058 raft_consensus.cc:359] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:17.680943 10058 raft_consensus.cc:385] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.681420 10058 raft_consensus.cc:740] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.682829 10058 consensus_queue.cc:260] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:17.685068  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:17.632735 10063 raft_consensus.cc:697] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:17.706296 10063 consensus_queue.cc:237] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.616598 10057 ts_tablet_manager.cc:1403] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.221s	user 0.016s	sys 0.003s
I20260709 13:05:17.665174  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 36f8b3f8e9ff4e1ab10c104b1a3c2da6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:17.713374 10073 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:17.687602 10061 raft_consensus.cc:359] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.722463 10061 raft_consensus.cc:385] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.722885 10061 raft_consensus.cc:740] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.723250  9922 tablet_service.cc:1511] Processing CreateTablet for tablet eb7d115ef55d410a88514a918dee8efc (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:17.723763 10061 consensus_queue.cc:260] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.663008 10092 raft_consensus.cc:493] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.721655  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f8b3f8e9ff4e1ab10c104b1a3c2da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.730901 10092 raft_consensus.cc:515] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.734360 10092 leader_election.cc:290] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:17.748960  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7d115ef55d410a88514a918dee8efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.767129 10058 ts_tablet_manager.cc:1434] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.143s	user 0.008s	sys 0.002s
I20260709 13:05:17.773128  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:17.788453 10058 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:17.790235 10078 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.790838 10078 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:17.787977 10012 raft_consensus.cc:2468] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
W20260709 13:05:17.792935  9977 leader_election.cc:343] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.792580 10057 raft_consensus.cc:359] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.798372 10057 raft_consensus.cc:385] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.798766 10057 raft_consensus.cc:740] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.800089 10061 ts_tablet_manager.cc:1434] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.176s	user 0.007s	sys 0.003s
I20260709 13:05:17.799800 10057 consensus_queue.cc:260] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.801856 10061 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:17.803349 10057 ts_tablet_manager.cc:1434] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.002s
I20260709 13:05:17.804855 10057 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:17.807101 10078 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:17.814224 10057 tablet_bootstrap.cc:654] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.822592 10058 tablet_bootstrap.cc:654] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.827913  9997 tablet_service.cc:1511] Processing CreateTablet for tablet eb7d115ef55d410a88514a918dee8efc (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:17.829706  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7d115ef55d410a88514a918dee8efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:17.840745 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:17.842556 10059 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
W20260709 13:05:17.843106  9903 leader_election.cc:343] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.843583 10059 ts_tablet_manager.cc:1403] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.353s	user 0.006s	sys 0.011s
I20260709 13:05:17.864931 10059 raft_consensus.cc:359] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.873024 10059 raft_consensus.cc:385] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.873452 10059 raft_consensus.cc:740] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.874521 10059 consensus_queue.cc:260] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.875648 10084 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.876266 10084 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.886422 10061 tablet_bootstrap.cc:654] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.904042 10058 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:17.904839 10058 ts_tablet_manager.cc:1403] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.122s	user 0.015s	sys 0.013s
I20260709 13:05:17.908860 10058 raft_consensus.cc:359] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.909849 10058 raft_consensus.cc:385] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.910164 10058 raft_consensus.cc:740] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.914201 10058 consensus_queue.cc:260] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.917029 10058 ts_tablet_manager.cc:1434] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.002s
I20260709 13:05:17.918690 10058 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:17.929395 10061 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:17.930187 10061 ts_tablet_manager.cc:1403] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.129s	user 0.019s	sys 0.003s
I20260709 13:05:17.932643 10058 tablet_bootstrap.cc:654] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.933027 10057 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:17.933702 10057 ts_tablet_manager.cc:1403] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.129s	user 0.010s	sys 0.014s
I20260709 13:05:17.941691 10058 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:17.942454 10058 ts_tablet_manager.cc:1403] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.007s
I20260709 13:05:17.945286 10061 raft_consensus.cc:359] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.946039 10061 raft_consensus.cc:385] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.946394 10061 raft_consensus.cc:740] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.946322 10058 raft_consensus.cc:359] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.947408 10058 raft_consensus.cc:385] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.947773 10058 raft_consensus.cc:740] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.947721 10061 consensus_queue.cc:260] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:17.948961 10058 consensus_queue.cc:260] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:17.949268 10057 raft_consensus.cc:359] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.950268 10057 raft_consensus.cc:385] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.950708 10057 raft_consensus.cc:740] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.951306 10061 ts_tablet_manager.cc:1434] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.004s
I20260709 13:05:17.952462 10058 ts_tablet_manager.cc:1434] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.002s
I20260709 13:05:17.951982 10057 consensus_queue.cc:260] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:17.952747 10061 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:17.953841 10058 tablet_bootstrap.cc:492] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:17.955421 10057 ts_tablet_manager.cc:1434] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 13:05:17.947248 10084 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:17.957089 10057 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:17.964133 10058 tablet_bootstrap.cc:654] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.969616 10061 tablet_bootstrap.cc:654] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.971766 10057 tablet_bootstrap.cc:654] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.997038 10058 tablet_bootstrap.cc:492] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:17.997758 10058 ts_tablet_manager.cc:1403] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.008s
I20260709 13:05:18.001389 10057 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:18.002278 10057 ts_tablet_manager.cc:1403] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.046s	user 0.020s	sys 0.001s
I20260709 13:05:18.002606 10058 raft_consensus.cc:359] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.003969 10058 raft_consensus.cc:385] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.004467 10058 raft_consensus.cc:740] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.005630 10058 consensus_queue.cc:260] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.006994 10057 raft_consensus.cc:359] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:18.008038 10057 raft_consensus.cc:385] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.008668 10057 raft_consensus.cc:740] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.009756 10057 consensus_queue.cc:260] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:18.016654 10061 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:18.014277  9754 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:18.022253 10061 ts_tablet_manager.cc:1403] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.070s	user 0.033s	sys 0.024s
I20260709 13:05:18.022680 10073 raft_consensus.cc:2804] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.024371 10057 ts_tablet_manager.cc:1434] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.022s	user 0.014s	sys 0.005s
I20260709 13:05:18.025921 10057 tablet_bootstrap.cc:492] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:18.027237 10061 raft_consensus.cc:359] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.028530 10061 raft_consensus.cc:385] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.028957 10061 raft_consensus.cc:740] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.010004 10058 ts_tablet_manager.cc:1434] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.009s
I20260709 13:05:18.030170 10061 consensus_queue.cc:260] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.030764 10058 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:18.035961  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 0f50ff8860ba431d9ee421ff76612df6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:18.037868  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f50ff8860ba431d9ee421ff76612df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.038146 10078 raft_consensus.cc:493] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.038784 10078 raft_consensus.cc:515] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.041357 10061 ts_tablet_manager.cc:1434] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:05:18.041927 10073 raft_consensus.cc:697] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:18.042853 10061 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:18.043072 10073 consensus_queue.cc:237] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.052129 10061 tablet_bootstrap.cc:654] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.053150 10058 tablet_bootstrap.cc:654] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.053844  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.054782  9788 raft_consensus.cc:2468] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:18.056847  9901 leader_election.cc:304] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:18.060853 10078 leader_election.cc:290] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:18.061228 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:18.061816 10078 raft_consensus.cc:2804] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.062402 10012 raft_consensus.cc:2468] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:18.062424 10078 raft_consensus.cc:493] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.064731 10078 raft_consensus.cc:3060] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.035992 10057 tablet_bootstrap.cc:654] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.067343 10058 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:18.068011 10058 ts_tablet_manager.cc:1403] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.004s
I20260709 13:05:18.073913 10058 raft_consensus.cc:359] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.074971 10058 raft_consensus.cc:385] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.075348 10058 raft_consensus.cc:740] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.078488 10078 raft_consensus.cc:515] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.082957 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:18.083758 10012 raft_consensus.cc:3060] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.090361 10059 ts_tablet_manager.cc:1434] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.246s	user 0.015s	sys 0.003s
I20260709 13:05:18.093421  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.094848 10078 leader_election.cc:290] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:18.094069 10012 raft_consensus.cc:2468] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:18.128840 10073 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.135970 10073 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.137022 10106 raft_consensus.cc:493] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.137657 10106 raft_consensus.cc:515] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.138291 10061 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:18.138985 10061 ts_tablet_manager.cc:1403] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.096s	user 0.020s	sys 0.000s
I20260709 13:05:18.139099 10073 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:18.076284 10058 consensus_queue.cc:260] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.141161 10058 ts_tablet_manager.cc:1434] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.073s	user 0.009s	sys 0.000s
I20260709 13:05:18.141705 10106 leader_election.cc:290] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:18.142409 10058 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:18.143745 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:18.144766  9774 consensus_meta.cc:287] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Time spent flushing consensus metadata: real 0.534s	user 0.019s	sys 0.000s
I20260709 13:05:18.129204 10063 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.128672  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 71957.1us
I20260709 13:05:18.147495 10059 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
W20260709 13:05:18.152541  9901 leader_election.cc:343] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.157145  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:18.161696  9789 raft_consensus.cc:3060] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.163414  9850 tablet_service.cc:1511] Processing CreateTablet for tablet eb7d115ef55d410a88514a918dee8efc (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
W20260709 13:05:18.150372  9903 leader_election.cc:343] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.164716  9853 tablet_service.cc:1511] Processing CreateTablet for tablet 0f50ff8860ba431d9ee421ff76612df6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:18.165311  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7d115ef55d410a88514a918dee8efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.165477  9903 leader_election.cc:304] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:18.149101 10063 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.166393  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f50ff8860ba431d9ee421ff76612df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.161410  9848 tablet_service.cc:1511] Processing CreateTablet for tablet 82999b9832894f40bb6dbe02e2feb1df (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:18.168459 10063 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.159192  9851 tablet_service.cc:1511] Processing CreateTablet for tablet 178dcd0da5414a9785794b2412a9cf93 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:18.169051  9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82999b9832894f40bb6dbe02e2feb1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.169934  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 178dcd0da5414a9785794b2412a9cf93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.170171  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.170867 10106 raft_consensus.cc:2804] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.175593 10106 raft_consensus.cc:697] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:18.171571 10073 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.183627 10073 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.186062  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.163429  9849 tablet_service.cc:1511] Processing CreateTablet for tablet 1078585b45d74186be92d54ac68f787a (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
W20260709 13:05:18.188127  9904 leader_election.cc:343] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.188557  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1078585b45d74186be92d54ac68f787a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.188740  9904 leader_election.cc:304] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:18.161270  9852 tablet_service.cc:1511] Processing CreateTablet for tablet f0bdd04f0ff44f2e87f76f36a7c5e540 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:18.173658  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.174233  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:18.170996  9864 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:18.184181 10106 consensus_queue.cc:237] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.190263 10078 raft_consensus.cc:2749] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.197810  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.191805  9852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0bdd04f0ff44f2e87f76f36a7c5e540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.198714  9787 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:18.198902  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.200026  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.192481  9863 raft_consensus.cc:2393] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:18.201884  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:18.192845  9862 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.194551 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:18.196836  9978 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:18.197317 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:18.192245  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:18.206707  9938 raft_consensus.cc:2393] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:18.208519 10063 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.209221 10011 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 0.
I20260709 13:05:18.182175 10061 raft_consensus.cc:359] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.213097 10063 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.218955 10063 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.213033 10073 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.221405 10061 raft_consensus.cc:385] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.221936 10061 raft_consensus.cc:740] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.223332 10061 consensus_queue.cc:260] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.230310 10063 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.231834  9901 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:18.233426  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 80934.6us
I20260709 13:05:18.238201  9789 raft_consensus.cc:2468] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
W20260709 13:05:18.243472  9755 leader_election.cc:343] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.243538  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.245710  9863 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
W20260709 13:05:18.246661  9755 leader_election.cc:343] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.247493  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.248271  9788 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 0.
I20260709 13:05:18.248754  9829 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:18.249622  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.250511  9755 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:18.251210  9904 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:18.240803  9862 raft_consensus.cc:2468] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:18.258489  9927 tablet_service.cc:1511] Processing CreateTablet for tablet 0f50ff8860ba431d9ee421ff76612df6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:18.260712  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f50ff8860ba431d9ee421ff76612df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.265827  9827 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:18.267396  9774 tablet_service.cc:1511] Processing CreateTablet for tablet e18d9519bce7451aaee1e04623f7fddb (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:18.269655  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d9519bce7451aaee1e04623f7fddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.273079 10106 raft_consensus.cc:2749] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.274276 10106 raft_consensus.cc:2749] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.286540 10073 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.287153 10073 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.287514 10073 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.289340 10092 consensus_queue.cc:1048] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.184240 10057 tablet_bootstrap.cc:492] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:18.310387 10057 ts_tablet_manager.cc:1403] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.285s	user 0.025s	sys 0.014s
I20260709 13:05:18.341629  9997 tablet_service.cc:1511] Processing CreateTablet for tablet e18d9519bce7451aaee1e04623f7fddb (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
W20260709 13:05:18.334323  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 131763us
I20260709 13:05:18.355695 10084 raft_consensus.cc:2749] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.364156 10084 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.364840 10084 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.365465 10084 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.344097  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d9519bce7451aaee1e04623f7fddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.390285 10063 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.432521 10059 tablet_bootstrap.cc:700] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: Time spent opening tablet: real 0.252s	user 0.010s	sys 0.000s
I20260709 13:05:18.433686 10059 tablet_bootstrap.cc:654] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.437844  9684 catalog_manager.cc:5697] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.441388 10058 tablet_bootstrap.cc:700] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: Time spent opening tablet: real 0.284s	user 0.011s	sys 0.002s
I20260709 13:05:18.442737 10058 tablet_bootstrap.cc:654] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:18.444922  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 179485us
I20260709 13:05:18.396001 10061 ts_tablet_manager.cc:1434] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.257s	user 0.011s	sys 0.003s
I20260709 13:05:18.447053  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:18.447355  9975 leader_election.cc:304] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:18.448828 10061 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:18.449581 10063 raft_consensus.cc:2749] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.450098  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:18.450412 10063 consensus_queue.cc:1048] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.459026  9922 tablet_service.cc:1511] Processing CreateTablet for tablet 7eab4bb2719246f2bf5af8bbccae641d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:18.461346  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eab4bb2719246f2bf5af8bbccae641d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.474763 10092 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.476104 10092 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:18.448619 10057 raft_consensus.cc:359] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.481803 10092 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:18.491501 10061 tablet_bootstrap.cc:654] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.504513  9849 tablet_service.cc:1511] Processing CreateTablet for tablet e18d9519bce7451aaee1e04623f7fddb (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:18.506925  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d9519bce7451aaee1e04623f7fddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.508764 10092 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.509491 10092 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.509745  9853 tablet_service.cc:1511] Processing CreateTablet for tablet 93fbf294836d47b29e0143174bbb3df4 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:18.510735  9924 tablet_service.cc:1511] Processing CreateTablet for tablet 93fbf294836d47b29e0143174bbb3df4 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:18.511862  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93fbf294836d47b29e0143174bbb3df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.509686 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:18.481864 10057 raft_consensus.cc:385] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.512880 10057 raft_consensus.cc:740] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.513041 10011 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:18.512514  9924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93fbf294836d47b29e0143174bbb3df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.514219  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 7eab4bb2719246f2bf5af8bbccae641d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:18.513975 10057 consensus_queue.cc:260] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.513301  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.515882  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eab4bb2719246f2bf5af8bbccae641d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.515970  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.516832 10092 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.518568  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:18.519304  9975 leader_election.cc:343] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:18.556128  9978 leader_election.cc:343] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.544709 10084 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.562963 10073 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:18.586177  9975 leader_election.cc:343] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.610306 10112 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.610879  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:18.610971 10112 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.611783  9938 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:18.609977  9850 tablet_service.cc:1511] Processing CreateTablet for tablet b253ddcc7d6046c6abdd94aec212612c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:18.614060  9789 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:18.614323  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253ddcc7d6046c6abdd94aec212612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.613763  9926 tablet_service.cc:1511] Processing CreateTablet for tablet b253ddcc7d6046c6abdd94aec212612c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:18.605571  9862 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:18.636214 10058 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:18.637022 10058 ts_tablet_manager.cc:1403] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.495s	user 0.038s	sys 0.017s
I20260709 13:05:18.650913 10061 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:18.650322  9778 tablet_service.cc:1511] Processing CreateTablet for tablet b253ddcc7d6046c6abdd94aec212612c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:18.651855 10061 ts_tablet_manager.cc:1403] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.203s	user 0.015s	sys 0.001s
I20260709 13:05:18.613785 10112 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.673223  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.615408  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 7eab4bb2719246f2bf5af8bbccae641d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:18.680996 10057 ts_tablet_manager.cc:1434] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.370s	user 0.012s	sys 0.019s
I20260709 13:05:18.681876  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eab4bb2719246f2bf5af8bbccae641d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.683643 10057 tablet_bootstrap.cc:492] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:18.689101 10084 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:18.690775  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:18.691398  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253ddcc7d6046c6abdd94aec212612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.644018  9999 tablet_service.cc:1511] Processing CreateTablet for tablet 93fbf294836d47b29e0143174bbb3df4 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:18.696997  9776 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6f1cfb3ffd4a418cafdda5af0b219d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:18.697640 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:18.698222 10073 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.698791 10011 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:18.699612 10059 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:18.700718 10059 ts_tablet_manager.cc:1403] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.568s	user 0.040s	sys 0.035s
I20260709 13:05:18.701133  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93fbf294836d47b29e0143174bbb3df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.704279  9829 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:18.706670 10084 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.638715  9977 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:18.717317 10063 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:18.717532  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 91475.3us
I20260709 13:05:18.717898 10063 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.718353 10063 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.720088 10000 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6f1cfb3ffd4a418cafdda5af0b219d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:18.721758 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6f1cfb3ffd4a418cafdda5af0b219d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.725751  9927 tablet_service.cc:1511] Processing CreateTablet for tablet 210c05e6675c4ba78ca28841a5315a07 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:18.726683  9996 tablet_service.cc:1511] Processing CreateTablet for tablet 210c05e6675c4ba78ca28841a5315a07 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:18.727461  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210c05e6675c4ba78ca28841a5315a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.728163  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210c05e6675c4ba78ca28841a5315a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.728610  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b253ddcc7d6046c6abdd94aec212612c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.725798  9777 tablet_service.cc:1511] Processing CreateTablet for tablet 210c05e6675c4ba78ca28841a5315a07 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:18.734148 10061 raft_consensus.cc:359] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.735172 10061 raft_consensus.cc:385] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.735545 10061 raft_consensus.cc:740] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
W20260709 13:05:18.737329 10118 log.cc:927] Time spent T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:05:18.741940  9851 tablet_service.cc:1511] Processing CreateTablet for tablet 7d6f1cfb3ffd4a418cafdda5af0b219d (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:18.743816  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6f1cfb3ffd4a418cafdda5af0b219d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.745522 10123 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.745726  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d6f1cfb3ffd4a418cafdda5af0b219d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.746533  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:18.748580  9862 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:18.750339 10058 raft_consensus.cc:359] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:18.751262 10058 raft_consensus.cc:385] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.751582 10058 raft_consensus.cc:740] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.755093  9975 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:18.756451 10115 raft_consensus.cc:2749] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.760470 10058 consensus_queue.cc:260] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:18.711602 10059 raft_consensus.cc:359] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.770354 10059 raft_consensus.cc:385] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:18.760785  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 201473us
I20260709 13:05:18.789302  9777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 210c05e6675c4ba78ca28841a5315a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.789135 10059 raft_consensus.cc:740] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.805698 10058 ts_tablet_manager.cc:1434] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.168s	user 0.011s	sys 0.000s
I20260709 13:05:18.807293  9848 tablet_service.cc:1511] Processing CreateTablet for tablet 1801426ac1e84ebaadda531ec03958bd (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:18.782876 10123 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.813416  9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1801426ac1e84ebaadda531ec03958bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.814599 10123 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:18.815516  9788 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:18.816135  9774 tablet_service.cc:1511] Processing CreateTablet for tablet 4a2a7189703643899a38fc9b896d3c78 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:18.818092  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a2a7189703643899a38fc9b896d3c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.825196  9975 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:18.827129  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.827538  9852 tablet_service.cc:1511] Processing CreateTablet for tablet 4a2a7189703643899a38fc9b896d3c78 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:18.810464 10057 tablet_bootstrap.cc:654] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.772794 10061 consensus_queue.cc:260] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.841910 10061 ts_tablet_manager.cc:1434] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.181s	user 0.010s	sys 0.000s
I20260709 13:05:18.842967 10073 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.843748 10061 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:18.831292 10002 tablet_service.cc:1511] Processing CreateTablet for tablet 1801426ac1e84ebaadda531ec03958bd (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:18.826968 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:18.800563 10115 raft_consensus.cc:493] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.832283  9755 leader_election.cc:343] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.845475 10011 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:18.845854 10115 raft_consensus.cc:515] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:18.833261 10063 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:18.808529  9922 tablet_service.cc:1511] Processing CreateTablet for tablet 1801426ac1e84ebaadda531ec03958bd (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:18.836853  9925 tablet_service.cc:1511] Processing CreateTablet for tablet 4a2a7189703643899a38fc9b896d3c78 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:18.843544 10073 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.829856 10059 consensus_queue.cc:260] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:18.859346  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:18.845465 10002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1801426ac1e84ebaadda531ec03958bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.860880 10059 ts_tablet_manager.cc:1434] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.160s	user 0.012s	sys 0.001s
I20260709 13:05:18.862304 10059 tablet_bootstrap.cc:492] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:18.863183 10084 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.863869 10084 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:18.807262 10058 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:18.854029 10063 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:18.855468  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
W20260709 13:05:18.846877  9904 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.018s	sys 0.035s
I20260709 13:05:18.856909  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1801426ac1e84ebaadda531ec03958bd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:18.880653  9904 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.018s	sys 0.035s
W20260709 13:05:18.881922  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 630703us
W20260709 13:05:18.846621  9755 consensus_peers.cc:597] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:18.838953  9852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a2a7189703643899a38fc9b896d3c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.884517  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.889402  9925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a2a7189703643899a38fc9b896d3c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.894179 10063 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.904567  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:18.854811  9687 catalog_manager.cc:5697] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.870652  9937 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.871896  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.875484 10115 leader_election.cc:290] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:18.877390  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:18.915135  9901 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:18.916555 10112 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.917265 10112 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:18.920596  9936 raft_consensus.cc:2468] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:18.924252  9977 leader_election.cc:304] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:18.924710 10084 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:18.925737 10112 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:18.926657 10112 raft_consensus.cc:2804] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.926811 10115 raft_consensus.cc:2804] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.927102 10112 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.927573 10112 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.927984 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:18.928998 10011 raft_consensus.cc:2393] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:18.927381 10115 raft_consensus.cc:493] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.930732 10115 raft_consensus.cc:3060] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:18.929869  9978 leader_election.cc:343] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.929657  9999 tablet_service.cc:1511] Processing CreateTablet for tablet 04298f45db464a09bd0d2b82492e61b3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:18.936257  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04298f45db464a09bd0d2b82492e61b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:18.941587 10073 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:18.930526  9829 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:18.944525 10084 raft_consensus.cc:3060] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.954366  9775 tablet_service.cc:1511] Processing CreateTablet for tablet 04298f45db464a09bd0d2b82492e61b3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:18.956876  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:18.957381  9775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04298f45db464a09bd0d2b82492e61b3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:18.958756  9755 leader_election.cc:343] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.961617 10058 tablet_bootstrap.cc:654] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:18.961169  9904 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:37093, user_credentials={real_user=slave}} blocked reactor thread for 47720.7us
W20260709 13:05:18.969746  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 144672us
I20260709 13:05:18.985536  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:18.992975  9754 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:18.993996  9850 tablet_service.cc:1511] Processing CreateTablet for tablet 04298f45db464a09bd0d2b82492e61b3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:19.005752  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04298f45db464a09bd0d2b82492e61b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.006719  9937 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:19.012070 10073 consensus_queue.cc:1048] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:05:19.014742  9975 leader_election.cc:343] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.006783 10122 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.029925  9977 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:19.000945  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 67532.5us
I20260709 13:05:19.040498 10133 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.044230  9927 tablet_service.cc:1511] Processing CreateTablet for tablet c416362f795e46e28a60937c9ef5164f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:19.053053  9851 tablet_service.cc:1511] Processing CreateTablet for tablet 4125c95bf84c45538728679b308f6aca (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:19.057435  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c416362f795e46e28a60937c9ef5164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.058522  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125c95bf84c45538728679b308f6aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.049984 10133 raft_consensus.cc:697] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:19.055260  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:19.062147  9997 tablet_service.cc:1511] Processing CreateTablet for tablet c416362f795e46e28a60937c9ef5164f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:19.069790  9777 tablet_service.cc:1511] Processing CreateTablet for tablet c416362f795e46e28a60937c9ef5164f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:19.013854  9904 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:19.008159 10084 raft_consensus.cc:2749] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.080907 10059 tablet_bootstrap.cc:654] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.091207  9788 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:19.080811  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 4125c95bf84c45538728679b308f6aca (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:19.096364  9901 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:19.097556  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125c95bf84c45538728679b308f6aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.097694 10134 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:19.098331 10134 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.098845 10134 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.092145 10133 consensus_queue.cc:237] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:19.092901  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c416362f795e46e28a60937c9ef5164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.107525 10057 tablet_bootstrap.cc:492] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:19.108480 10057 ts_tablet_manager.cc:1403] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.426s	user 0.019s	sys 0.004s
I20260709 13:05:19.117405 10000 tablet_service.cc:1511] Processing CreateTablet for tablet 4125c95bf84c45538728679b308f6aca (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:19.119284 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125c95bf84c45538728679b308f6aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.122437  9776 tablet_service.cc:1511] Processing CreateTablet for tablet b5fa7348f6384183abeb9619f1fd0d75 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:19.123629  9777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c416362f795e46e28a60937c9ef5164f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.128577  9684 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33634 (request call id 8) took 1043 ms. Trace:
I20260709 13:05:19.133606 10058 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:19.134479 10058 ts_tablet_manager.cc:1403] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.328s	user 0.035s	sys 0.040s
I20260709 13:05:19.150249  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fa7348f6384183abeb9619f1fd0d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.129225  9684 rpcz_store.cc:276] 0709 13:05:18.085647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.085847 (+   200us) service_pool.cc:224] Handling call
0709 13:05:19.128472 (+1042625us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:18.607523 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:18.608885 (+  1362us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:18.608926 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:05:18.608945 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:18.609313 (+   368us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:18.609410 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:18.609448 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:18.609643 (+   195us) tablet.cc:701] Row locks acquired
0709 13:05:18.609660 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:05:18.609833 (+   173us) write_op.cc:270] Start()
0709 13:05:18.609957 (+   124us) write_op.cc:276] Timestamp: P: 1783602318609782 usec, L: 0
0709 13:05:18.609978 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:05:18.610818 (+   840us) log.cc:844] Serialized 536 byte log entry
0709 13:05:18.987728 (+376910us) op_driver.cc:464] REPLICATION: finished
0709 13:05:19.123449 (+135721us) write_op.cc:301] APPLY: starting
0709 13:05:19.123593 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:19.123721 (+   128us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:19.123867 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:19.124224 (+   357us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:19.124456 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:19.124521 (+    65us) write_op.cc:312] APPLY: finished
0709 13:05:19.124994 (+   473us) log.cc:844] Serialized 22 byte log entry
0709 13:05:19.125629 (+   635us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:19.125737 (+   108us) write_op.cc:454] Released schema lock
0709 13:05:19.125787 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:19.126197 (+   410us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:19.126225 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":209446,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":121618,"apply.run_cpu_time_us":2757,"apply.run_wall_time_us":15743,"num_ops":1,"prepare.queue_time_us":144247,"prepare.run_cpu_time_us":97397,"prepare.run_wall_time_us":167476,"raft.queue_time_us":211151,"raft.run_cpu_time_us":35879,"raft.run_wall_time_us":111749,"replication_time_us":377577,"thread_start_us":460621,"threads_started":4,"wal-append.queue_time_us":188314}]]}
I20260709 13:05:19.138963 10115 raft_consensus.cc:515] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:19.178831 10112 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:19.197126  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:19.197932  9937 raft_consensus.cc:3060] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.181377 10061 tablet_bootstrap.cc:654] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.192469 10134 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:19.194481 10115 leader_election.cc:290] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:19.195013 10058 raft_consensus.cc:359] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:19.196362 10139 raft_consensus.cc:2804] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.212253 10057 raft_consensus.cc:359] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:19.217727 10058 raft_consensus.cc:385] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.218398 10058 raft_consensus.cc:740] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.217940 10057 raft_consensus.cc:385] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.219084 10057 raft_consensus.cc:740] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.220865  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:19.227658  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:19.228590  9788 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.228859 10140 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.229441 10140 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:19.233253  9978 leader_election.cc:343] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.234042  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:19.235666  9959 debug-util.cc:398] Leaking SignalData structure 0x7b08001e4680 after lost signal to thread 434
I20260709 13:05:19.235419  9852 tablet_service.cc:1511] Processing CreateTablet for tablet b5fa7348f6384183abeb9619f1fd0d75 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 13:05:19.237381  9959 debug-util.cc:398] Leaking SignalData structure 0x7b080035a2e0 after lost signal to thread 9721
I20260709 13:05:19.237895 10058 consensus_queue.cc:260] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:19.238574  9901 leader_election.cc:343] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.248489 10140 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:19.251811 10134 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:19.252589 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:19.253373 10057 consensus_queue.cc:260] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:19.141652  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 133697us
I20260709 13:05:19.277511  9852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fa7348f6384183abeb9619f1fd0d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.303606 10057 ts_tablet_manager.cc:1434] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.195s	user 0.011s	sys 0.000s
I20260709 13:05:19.310248 10057 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:19.286403 10112 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:19.303398  9904 leader_election.cc:343] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:19.346282  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 45099.3us
I20260709 13:05:19.357326  9937 raft_consensus.cc:2468] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:19.358366  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:19.358052  9922 tablet_service.cc:1511] Processing CreateTablet for tablet b5fa7348f6384183abeb9619f1fd0d75 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:19.359201  9789 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.359905  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5fa7348f6384183abeb9619f1fd0d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.370918  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:19.372154  9903 leader_election.cc:343] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.372962  9903 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:19.381335  9959 debug-util.cc:398] Leaking SignalData structure 0x7b08001048e0 after lost signal to thread 9888
I20260709 13:05:19.379915 10139 raft_consensus.cc:697] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:19.382654 10134 raft_consensus.cc:2749] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.383759  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:19.411401 10061 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:19.411136  9977 leader_election.cc:304] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:19.414419  9774 tablet_service.cc:1511] Processing CreateTablet for tablet 22c6e691c1264b8e8de49c1df4cd8bb5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:19.425149 10059 tablet_bootstrap.cc:492] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:19.432988 10140 consensus_queue.cc:1048] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:19.436283  9848 tablet_service.cc:1511] Processing CreateTablet for tablet 22c6e691c1264b8e8de49c1df4cd8bb5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:19.453490 10061 ts_tablet_manager.cc:1403] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.610s	user 0.048s	sys 0.068s
I20260709 13:05:19.471282  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:19.466563 10092 raft_consensus.cc:2804] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.473325 10092 raft_consensus.cc:697] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:19.474478 10092 consensus_queue.cc:237] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:19.461335 10059 ts_tablet_manager.cc:1403] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.596s	user 0.026s	sys 0.030s
W20260709 13:05:19.476553  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 80408.6us
I20260709 13:05:19.479362  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c6e691c1264b8e8de49c1df4cd8bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.479979 10000 tablet_service.cc:1511] Processing CreateTablet for tablet 22c6e691c1264b8e8de49c1df4cd8bb5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:19.481874 10000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c6e691c1264b8e8de49c1df4cd8bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.486730 10140 consensus_queue.cc:1048] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:19.491457  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:19.496600  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:19.500901  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:19.501789  9936 raft_consensus.cc:2393] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:19.502121  9937 raft_consensus.cc:2393] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:19.497859  9754 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:19.504098 10122 raft_consensus.cc:2749] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:19.463964  9959 debug-util.cc:398] Leaking SignalData structure 0x7b0800260800 after lost signal to thread 9962
W20260709 13:05:19.499374  9904 leader_election.cc:343] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.521901  9848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22c6e691c1264b8e8de49c1df4cd8bb5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:19.532133  9959 debug-util.cc:398] Leaking SignalData structure 0x7b080026e420 after lost signal to thread 10036
W20260709 13:05:19.538136  9671 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 48053us
I20260709 13:05:19.538133 10059 raft_consensus.cc:359] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:19.538093  9685 catalog_manager.cc:5697] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.542001 10058 ts_tablet_manager.cc:1434] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.407s	user 0.029s	sys 0.051s
I20260709 13:05:19.546922 10057 tablet_bootstrap.cc:654] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.547061 10058 tablet_bootstrap.cc:492] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736: Bootstrap starting.
W20260709 13:05:19.533128  9673 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 45578.4us
I20260709 13:05:19.545267 10059 raft_consensus.cc:385] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.551807 10059 raft_consensus.cc:740] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.550267  9788 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:19.558657 10059 consensus_queue.cc:260] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:19.568724  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad9129f493c417eaa0d52805e5b1144 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 13:05:19.569173  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.569921 10061 raft_consensus.cc:359] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:19.570574  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad9129f493c417eaa0d52805e5b1144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.571026 10061 raft_consensus.cc:385] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.571523 10061 raft_consensus.cc:740] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.582515 10115 raft_consensus.cc:493] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.584506 10059 ts_tablet_manager.cc:1434] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.109s	user 0.011s	sys 0.001s
W20260709 13:05:19.551589  9904 leader_election.cc:343] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.586208  9789 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:19.588743  9925 tablet_service.cc:1511] Processing CreateTablet for tablet d3324f833b7b462d8aa8db06fb4bd74e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:19.591866 10002 tablet_service.cc:1511] Processing CreateTablet for tablet d3324f833b7b462d8aa8db06fb4bd74e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:05:19.599596  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 51132.4us
I20260709 13:05:19.597739 10115 raft_consensus.cc:515] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:19.600701 10002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3324f833b7b462d8aa8db06fb4bd74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.601394  9775 tablet_service.cc:1511] Processing CreateTablet for tablet bd7ef372f79649dda9b18919bfc39298 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:19.602641 10115 leader_election.cc:290] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:19.603168  9775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7ef372f79649dda9b18919bfc39298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.609505  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:19.613360  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:05:19.614574  9978 leader_election.cc:343] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.549785 10139 consensus_queue.cc:237] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:19.607903  9904 leader_election.cc:343] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.612887 10059 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:19.599665  9925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3324f833b7b462d8aa8db06fb4bd74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.621913  9851 tablet_service.cc:1511] Processing CreateTablet for tablet d3324f833b7b462d8aa8db06fb4bd74e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:19.622208  9776 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad9129f493c417eaa0d52805e5b1144 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 13:05:19.615676  9977 leader_election.cc:343] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.624643  9977 leader_election.cc:304] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:19.625069  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad9129f493c417eaa0d52805e5b1144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.584654  9997 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad9129f493c417eaa0d52805e5b1144 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:19.629316  9901 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:19.630056  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad9129f493c417eaa0d52805e5b1144. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:19.630582  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 59342.3us
I20260709 13:05:19.631673  9901 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:19.632790 10147 raft_consensus.cc:2804] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.633509 10147 raft_consensus.cc:697] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:19.634209 10140 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.634647 10147 consensus_queue.cc:237] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:19.634935 10140 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:19.611275  9683 catalog_manager.cc:5697] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } }
I20260709 13:05:19.634864  9850 tablet_service.cc:1511] Processing CreateTablet for tablet bd7ef372f79649dda9b18919bfc39298 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:19.640125 10140 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:19.675647 10115 raft_consensus.cc:2749] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.684616 10057 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:19.685340 10057 ts_tablet_manager.cc:1403] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.376s	user 0.020s	sys 0.010s
I20260709 13:05:19.687436  9927 tablet_service.cc:1511] Processing CreateTablet for tablet bd7ef372f79649dda9b18919bfc39298 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:19.689139  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7ef372f79649dda9b18919bfc39298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.667611  9851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3324f833b7b462d8aa8db06fb4bd74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.663065 10140 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.699924 10057 raft_consensus.cc:359] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:19.700590  9777 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf82e92a7f845f4a7c658d0783ea4a5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:19.708495 10140 raft_consensus.cc:697] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:19.709499  9777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf82e92a7f845f4a7c658d0783ea4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.708458 10057 raft_consensus.cc:385] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.711855 10140 consensus_queue.cc:237] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:19.712781 10057 raft_consensus.cc:740] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.713790 10057 consensus_queue.cc:260] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:19.701535  9849 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf82e92a7f845f4a7c658d0783ea4a5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:19.716892  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf82e92a7f845f4a7c658d0783ea4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.706476  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:19.723066 10057 ts_tablet_manager.cc:1434] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
W20260709 13:05:19.724099  9901 leader_election.cc:343] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:19.697643  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 81125.7us
I20260709 13:05:19.725200 10057 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:19.728785  9996 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf82e92a7f845f4a7c658d0783ea4a5 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:19.733122  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf82e92a7f845f4a7c658d0783ea4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.616743 10061 consensus_queue.cc:260] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:19.716231  9922 tablet_service.cc:1511] Processing CreateTablet for tablet e12836e61c6d4427a5d883c4bdf5b5b6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:19.737773  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12836e61c6d4427a5d883c4bdf5b5b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.739455 10061 ts_tablet_manager.cc:1434] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.270s	user 0.012s	sys 0.000s
I20260709 13:05:19.707867 10058 tablet_bootstrap.cc:654] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.756008  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd7ef372f79649dda9b18919bfc39298. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:19.667001  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 60050.6us
I20260709 13:05:19.771955  9778 tablet_service.cc:1511] Processing CreateTablet for tablet e12836e61c6d4427a5d883c4bdf5b5b6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:19.773699  9853 tablet_service.cc:1511] Processing CreateTablet for tablet 82893adc39f24309a241ee3e5ff6611e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:19.775875  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82893adc39f24309a241ee3e5ff6611e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.777633  9754 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:19.779093 10092 consensus_queue.cc:1048] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:19.779592  9754 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:19.781222 10123 raft_consensus.cc:2749] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.783584  9999 tablet_service.cc:1511] Processing CreateTablet for tablet e12836e61c6d4427a5d883c4bdf5b5b6 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:19.785936  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12836e61c6d4427a5d883c4bdf5b5b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.773726  9778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12836e61c6d4427a5d883c4bdf5b5b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.789453 10123 raft_consensus.cc:2749] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.804051  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:19.806205  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 6bab397fa40041c281ce616134672828 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:19.809521  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 82893adc39f24309a241ee3e5ff6611e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:19.797309 10061 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:19.811597  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82893adc39f24309a241ee3e5ff6611e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:19.821525  9904 leader_election.cc:343] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.822057  9904 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:19.822974 10139 raft_consensus.cc:2749] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.821153  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 82893adc39f24309a241ee3e5ff6611e (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:19.830832  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82893adc39f24309a241ee3e5ff6611e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.834067  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bab397fa40041c281ce616134672828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.854373  9854 tablet_service.cc:1511] Processing CreateTablet for tablet 6bab397fa40041c281ce616134672828 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:19.856321  9854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bab397fa40041c281ce616134672828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.862635 10123 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.863554 10123 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:19.864053  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:19.867524 10123 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:19.868436  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:19.871150  9755 leader_election.cc:343] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.873325 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:19.874317 10011 raft_consensus.cc:2468] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:20.005686  9754 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:20.018460 10123 raft_consensus.cc:2804] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.019026 10123 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.019395 10123 raft_consensus.cc:3060] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:20.021615  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 142810us
I20260709 13:05:20.032635 10084 raft_consensus.cc:493] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.033705 10084 raft_consensus.cc:515] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.195459 10123 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.203737 10092 raft_consensus.cc:493] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.237586 10058 tablet_bootstrap.cc:492] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:20.238313 10058 ts_tablet_manager.cc:1403] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.692s	user 0.006s	sys 0.020s
I20260709 13:05:20.231062 10059 tablet_bootstrap.cc:700] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent opening tablet: real 0.420s	user 0.002s	sys 0.002s
I20260709 13:05:20.254326 10059 tablet_bootstrap.cc:654] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.255106  9863 raft_consensus.cc:3060] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.228858 10092 raft_consensus.cc:515] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.281507 10115 consensus_queue.cc:1048] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:20.272238  9848 consensus_meta.cc:287] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.005s
I20260709 13:05:20.291270 10058 raft_consensus.cc:359] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.292061 10058 raft_consensus.cc:385] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.292315 10058 raft_consensus.cc:740] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
W20260709 13:05:20.277298  9997 consensus_meta.cc:287] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent flushing consensus metadata: real 0.548s	user 0.014s	sys 0.000s
I20260709 13:05:20.294902 10058 consensus_queue.cc:260] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.309127 10057 tablet_bootstrap.cc:654] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.355552 10061 tablet_bootstrap.cc:654] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.302242 10084 leader_election.cc:290] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:20.367010 10123 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:20.375661 10058 ts_tablet_manager.cc:1434] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.137s	user 0.010s	sys 0.001s
I20260709 13:05:20.377136 10058 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:20.382562 10092 leader_election.cc:290] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:20.407932 10059 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:20.408979 10059 ts_tablet_manager.cc:1403] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.796s	user 0.039s	sys 0.048s
I20260709 13:05:20.413767 10059 raft_consensus.cc:359] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.415482 10059 raft_consensus.cc:385] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.424547 10059 raft_consensus.cc:740] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.426319 10059 consensus_queue.cc:260] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.429404 10061 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:20.429613 10059 ts_tablet_manager.cc:1434] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 13:05:20.430179 10061 ts_tablet_manager.cc:1403] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.640s	user 0.023s	sys 0.023s
I20260709 13:05:20.431372 10059 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:20.434337 10061 raft_consensus.cc:359] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.422704 10058 tablet_bootstrap.cc:654] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.439103 10061 raft_consensus.cc:385] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.439625 10061 raft_consensus.cc:740] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.440639 10061 consensus_queue.cc:260] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.443245 10059 tablet_bootstrap.cc:654] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.443730 10061 ts_tablet_manager.cc:1434] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:05:20.446836 10061 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:20.455816 10058 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:20.456918 10058 ts_tablet_manager.cc:1403] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.001s
I20260709 13:05:20.460193 10057 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:20.464273 10059 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:20.465030 10057 ts_tablet_manager.cc:1403] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.740s	user 0.025s	sys 0.007s
I20260709 13:05:20.465193 10059 ts_tablet_manager.cc:1403] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.014s
I20260709 13:05:20.468695 10057 raft_consensus.cc:359] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.469807 10057 raft_consensus.cc:385] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.469309 10059 raft_consensus.cc:359] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.470363 10059 raft_consensus.cc:385] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.470734 10059 raft_consensus.cc:740] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.471709 10059 consensus_queue.cc:260] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.472736 10058 raft_consensus.cc:359] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.473640 10058 raft_consensus.cc:385] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.473974 10058 raft_consensus.cc:740] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.475275 10059 ts_tablet_manager.cc:1434] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:05:20.474902 10058 consensus_queue.cc:260] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.477053 10059 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:20.478930 10061 tablet_bootstrap.cc:654] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.482123 10058 ts_tablet_manager.cc:1434] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:05:20.483654 10058 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:20.488005 10059 tablet_bootstrap.cc:654] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.470168 10057 raft_consensus.cc:740] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.490922 10057 consensus_queue.cc:260] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.494230 10057 ts_tablet_manager.cc:1434] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 13:05:20.495797 10057 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:20.506457 10058 tablet_bootstrap.cc:654] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.516733 10057 tablet_bootstrap.cc:654] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.517637  9996 tablet_service.cc:1511] Processing CreateTablet for tablet f5dd44a42946450e936b89718864c5e3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:20.519629  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5dd44a42946450e936b89718864c5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:20.532624 10058 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:20.533419 10058 ts_tablet_manager.cc:1403] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.001s
W20260709 13:05:20.533484  9777 consensus_meta.cc:287] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Time spent flushing consensus metadata: real 0.733s	user 0.007s	sys 0.007s
I20260709 13:05:20.567241 10061 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:20.568317 10061 ts_tablet_manager.cc:1403] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.122s	user 0.019s	sys 0.017s
I20260709 13:05:20.572677 10061 raft_consensus.cc:359] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.573629 10061 raft_consensus.cc:385] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.573956 10061 raft_consensus.cc:740] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.574717 10061 consensus_queue.cc:260] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.577730 10061 ts_tablet_manager.cc:1434] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
W20260709 13:05:20.228820  9776 consensus_meta.cc:287] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.006s
I20260709 13:05:20.579167 10061 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:20.558130 10058 raft_consensus.cc:359] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.580874 10058 raft_consensus.cc:385] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.581151 10058 raft_consensus.cc:740] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.582415 10058 consensus_queue.cc:260] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.584988 10058 ts_tablet_manager.cc:1434] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.001s
I20260709 13:05:20.586315 10058 tablet_bootstrap.cc:492] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:20.594334 10058 tablet_bootstrap.cc:654] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.609990 10061 tablet_bootstrap.cc:654] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.610672 10059 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:20.611053 10057 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:20.616994 10057 ts_tablet_manager.cc:1403] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.121s	user 0.018s	sys 0.023s
I20260709 13:05:20.621724 10057 raft_consensus.cc:359] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.622576 10057 raft_consensus.cc:385] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.622866 10057 raft_consensus.cc:740] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.616994 10059 ts_tablet_manager.cc:1403] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.140s	user 0.037s	sys 0.035s
I20260709 13:05:20.629914 10059 raft_consensus.cc:359] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.630750 10059 raft_consensus.cc:385] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.631049 10059 raft_consensus.cc:740] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.631948 10061 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:20.623610 10057 consensus_queue.cc:260] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.632629 10061 ts_tablet_manager.cc:1403] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.005s
I20260709 13:05:20.634563 10057 ts_tablet_manager.cc:1434] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:05:20.631836 10059 consensus_queue.cc:260] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.643652 10057 tablet_bootstrap.cc:492] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:20.636572 10061 raft_consensus.cc:359] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:20.639766  9774 consensus_meta.cc:287] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Time spent flushing consensus metadata: real 0.993s	user 0.012s	sys 0.003s
I20260709 13:05:20.646118 10059 ts_tablet_manager.cc:1434] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:05:20.647557  9774 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34138 (request call id 28) took 1246 ms. Trace:
I20260709 13:05:20.647740 10059 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:20.648043  9774 rpcz_store.cc:276] 0709 13:05:19.405078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.412578 (+  7500us) service_pool.cc:224] Handling call
0709 13:05:19.448982 (+ 36404us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:19.462033 (+ 13051us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:19.646578 (+184545us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:20.646893 (+1000315us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:20.647481 (+   588us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:20.652598 10057 tablet_bootstrap.cc:654] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.653256 10058 tablet_bootstrap.cc:492] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:20.653950 10058 ts_tablet_manager.cc:1403] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.000s
I20260709 13:05:20.658150 10058 raft_consensus.cc:359] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.659296 10058 raft_consensus.cc:385] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.659678 10058 raft_consensus.cc:740] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.660508 10061 raft_consensus.cc:385] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.660953 10061 raft_consensus.cc:740] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.662005 10061 consensus_queue.cc:260] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.664878 10061 ts_tablet_manager.cc:1434] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 13:05:20.665266 10057 tablet_bootstrap.cc:492] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:20.665838 10057 ts_tablet_manager.cc:1403] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 13:05:20.666042 10061 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:20.674486 10061 tablet_bootstrap.cc:654] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.676991 10058 consensus_queue.cc:260] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.679699 10058 ts_tablet_manager.cc:1434] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:05:20.681082 10058 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:20.681656 10059 tablet_bootstrap.cc:654] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.690393 10058 tablet_bootstrap.cc:654] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.707058 10147 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.798870 10123 raft_consensus.cc:493] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.785987  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:20.819482 10123 raft_consensus.cc:515] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:20.755584  9927 consensus_meta.cc:287] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: Time spent flushing consensus metadata: real 0.869s	user 0.007s	sys 0.009s
I20260709 13:05:20.784030 10084 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.824793  9863 raft_consensus.cc:3060] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.731235 10112 consensus_queue.cc:1048] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.085s
I20260709 13:05:20.831307 10061 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:20.823196  9927 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40624 (request call id 30) took 1137 ms. Trace:
I20260709 13:05:20.833206  9927 rpcz_store.cc:276] 0709 13:05:19.686045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.686226 (+   181us) service_pool.cc:224] Handling call
0709 13:05:19.688536 (+  2310us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:19.688655 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:19.885663 (+197008us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:20.821936 (+936273us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:20.823104 (+  1168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:20.817960 10147 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:20.839449 10058 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:20.840277 10058 ts_tablet_manager.cc:1403] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.159s	user 0.038s	sys 0.014s
I20260709 13:05:20.757656  9774 tablet_service.cc:1511] Processing CreateTablet for tablet f5dd44a42946450e936b89718864c5e3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:20.832404 10061 ts_tablet_manager.cc:1403] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.167s	user 0.033s	sys 0.024s
I20260709 13:05:20.729290  9787 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.824395 10084 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.851150  9789 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.849105 10147 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:20.844712 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:20.880669 10012 raft_consensus.cc:2468] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:20.848011  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5dd44a42946450e936b89718864c5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:20.883471  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 12cd0294fa614c61b200754ffbbf7601 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:20.851559  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:20.885375  9927 tablet_service.cc:1511] Processing CreateTablet for tablet 6bab397fa40041c281ce616134672828 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:20.886775  9786 raft_consensus.cc:2468] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:20.844164 10058 raft_consensus.cc:359] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.885447  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12cd0294fa614c61b200754ffbbf7601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:20.888312 10058 raft_consensus.cc:385] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.888808 10058 raft_consensus.cc:740] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.854874  9861 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.894392  9903 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:20.903663 10058 consensus_queue.cc:260] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.901106  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bab397fa40041c281ce616134672828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:20.910857 10058 ts_tablet_manager.cc:1434] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.070s	user 0.005s	sys 0.004s
I20260709 13:05:20.919337 10058 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:20.907331 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:20.903162 10115 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.931653 10134 raft_consensus.cc:493] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.933557  9862 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.935192  9863 raft_consensus.cc:2468] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:20.937844 10115 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:20.939514  9998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:32906 (request call id 32) took 1119 ms. Trace:
I20260709 13:05:20.902789 10147 raft_consensus.cc:2804] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.949012 10147 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.949615 10147 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.942260  9998 rpcz_store.cc:276] 0709 13:05:19.819696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.820029 (+   333us) service_pool.cc:224] Handling call
0709 13:05:19.829944 (+  9915us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:19.830186 (+   242us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:20.250865 (+420679us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:20.938759 (+687894us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:20.939407 (+   648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:20.953549 10134 raft_consensus.cc:515] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.957875 10115 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:20.917990 10057 raft_consensus.cc:359] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.966135 10057 raft_consensus.cc:385] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.966660 10057 raft_consensus.cc:740] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.972359 10057 consensus_queue.cc:260] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:20.926265 10123 leader_election.cc:290] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:20.982623 10061 raft_consensus.cc:359] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:20.984211 10061 raft_consensus.cc:385] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.984659 10061 raft_consensus.cc:740] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.993196 10059 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:20.994024 10059 ts_tablet_manager.cc:1403] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.347s	user 0.017s	sys 0.025s
I20260709 13:05:21.001011 10061 consensus_queue.cc:260] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.010746 10057 ts_tablet_manager.cc:1434] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.345s	user 0.011s	sys 0.002s
I20260709 13:05:20.984037 10112 raft_consensus.cc:493] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.960232 10115 consensus_queue.cc:1048] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 13:05:20.986177 10145 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.018167 10057 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:21.031100  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.033079  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:21.035068 10163 log.cc:927] Time spent T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.781s	user 0.007s	sys 0.006s
W20260709 13:05:20.990311  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 73607us
I20260709 13:05:21.147274  9850 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52508 (request call id 30) took 1524 ms. Trace:
I20260709 13:05:21.215277  9850 rpcz_store.cc:276] 0709 13:05:19.622654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.633355 (+ 10701us) service_pool.cc:224] Handling call
0709 13:05:19.648050 (+ 14695us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:19.648163 (+   113us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:20.849889 (+1201726us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:21.146355 (+296466us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:21.147178 (+   823us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:21.158334  9975 leader_election.cc:304] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
W20260709 13:05:21.218657  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 60538.5us
I20260709 13:05:21.218974 10191 raft_consensus.cc:2804] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.219553 10191 raft_consensus.cc:493] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.219954 10191 raft_consensus.cc:3060] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.220711 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:21.221705 10012 raft_consensus.cc:3060] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:21.168794  9904 leader_election.cc:343] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.228781 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.174983 10187 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.231390 10187 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.232231 10191 raft_consensus.cc:515] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:21.234479 10187 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.235184 10187 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.236490 10191 leader_election.cc:290] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:21.176283  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.192762 10185 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.237825 10185 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.242619 10011 raft_consensus.cc:2393] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:21.243283  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:21.244437  9786 raft_consensus.cc:3060] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:20.988324  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 97894.4us
I20260709 13:05:21.276213 10061 ts_tablet_manager.cc:1434] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.424s	user 0.024s	sys 0.014s
I20260709 13:05:21.280807  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.282320  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.283286 10061 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:21.235716 10187 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:21.284988  9787 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:21.270125  9775 consensus_meta.cc:287] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Time spent flushing consensus metadata: real 1.404s	user 0.015s	sys 0.018s
I20260709 13:05:21.287570  9775 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:34138 (request call id 30) took 1687 ms. Trace:
I20260709 13:05:21.288015  9775 rpcz_store.cc:276] 0709 13:05:19.599955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.600164 (+   209us) service_pool.cc:224] Handling call
0709 13:05:19.602475 (+  2311us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:19.602674 (+   199us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:19.864853 (+262179us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:21.286735 (+1421882us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:21.287395 (+   660us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:21.295181 10134 leader_election.cc:290] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:21.296532 10134 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.298729 10134 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:21.301373  9850 tablet_service.cc:1511] Processing CreateTablet for tablet f5dd44a42946450e936b89718864c5e3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:21.303674  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5dd44a42946450e936b89718864c5e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.310472 10115 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.327634  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:21.363816 10191 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.416584  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.417474 10186 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:21.431046  9683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33594 (request call id 8) took 1828 ms. Trace:
I20260709 13:05:21.450114  9862 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:21.425243  9978 leader_election.cc:343] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.447585 10191 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.480502 10059 raft_consensus.cc:359] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.482190 10059 raft_consensus.cc:385] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.483376 10059 raft_consensus.cc:740] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.450196 10115 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:21.467398  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 237006us
W20260709 13:05:21.488605  9674 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 154403us
I20260709 13:05:21.488683 10145 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.499533 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.500667 10009 raft_consensus.cc:2468] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:21.501101 10145 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:21.503947 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.495563  9786 raft_consensus.cc:2468] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:21.504897 10009 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
W20260709 13:05:21.496851  9922 consensus_meta.cc:287] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: Time spent flushing consensus metadata: real 1.223s	user 0.026s	sys 0.009s
I20260709 13:05:21.496567 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.456776 10186 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.508800  9922 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40624 (request call id 31) took 1806 ms. Trace:
I20260709 13:05:21.508913 10010 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:21.509383  9922 rpcz_store.cc:276] 0709 13:05:19.702316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.714915 (+ 12599us) service_pool.cc:224] Handling call
0709 13:05:19.736987 (+ 22072us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:19.737125 (+   138us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:20.259748 (+522623us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:21.508059 (+1248311us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:21.508721 (+   662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:21.446907  9683 rpcz_store.cc:276] 0709 13:05:19.602668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.604761 (+  2093us) service_pool.cc:224] Handling call
0709 13:05:21.430936 (+1826175us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:19.710180 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:19.710736 (+   556us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:19.710769 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:05:19.710788 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:19.711198 (+   410us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:19.711256 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:19.711289 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:19.711566 (+   277us) tablet.cc:701] Row locks acquired
0709 13:05:19.711591 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:05:19.711767 (+   176us) write_op.cc:270] Start()
0709 13:05:19.711903 (+   136us) write_op.cc:276] Timestamp: P: 1783602319711701 usec, L: 0
0709 13:05:19.711925 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:05:19.712681 (+   756us) log.cc:844] Serialized 540 byte log entry
0709 13:05:20.298031 (+585350us) op_driver.cc:464] REPLICATION: finished
0709 13:05:20.357982 (+ 59951us) write_op.cc:301] APPLY: starting
0709 13:05:20.358074 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:20.358181 (+   107us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:20.358324 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:20.358556 (+   232us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:20.358725 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:20.358781 (+    56us) write_op.cc:312] APPLY: finished
0709 13:05:20.359192 (+   411us) log.cc:844] Serialized 22 byte log entry
0709 13:05:20.364630 (+  5438us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.365065 (+   435us) write_op.cc:454] Released schema lock
0709 13:05:20.365476 (+   411us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":107515,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":51041,"apply.run_cpu_time_us":2261,"apply.run_wall_time_us":8689,"num_ops":1,"prepare.queue_time_us":68341,"prepare.run_cpu_time_us":2771,"prepare.run_wall_time_us":2804,"replication_time_us":582011,"thread_start_us":49318,"threads_started":1}]]}
I20260709 13:05:21.511711 10061 tablet_bootstrap.cc:654] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:21.505395  9861 consensus_meta.cc:287] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: Time spent flushing consensus metadata: real 0.612s	user 0.014s	sys 0.010s
I20260709 13:05:21.506489  9975 leader_election.cc:304] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.506232 10194 consensus_queue.cc:1048] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:21.482649  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 68907us
I20260709 13:05:21.516121 10059 consensus_queue.cc:260] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.532881  9829 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:21.537036 10201 raft_consensus.cc:493] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:21.538973 10201 raft_consensus.cc:515] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:21.540784 10189 raft_consensus.cc:2804] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:21.546952  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:21.537312 10195 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.551086  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 12cd0294fa614c61b200754ffbbf7601 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:21.551204 10195 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.554116 10195 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:21.555255  9775 tablet_service.cc:1511] Processing CreateTablet for tablet 12cd0294fa614c61b200754ffbbf7601 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:21.557332  9775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12cd0294fa614c61b200754ffbbf7601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.560636 10187 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:21.024456 10112 raft_consensus.cc:515] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.563556  9975 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.539913  9685 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33634 (request call id 9) took 2180 ms. Trace:
I20260709 13:05:21.564088 10112 leader_election.cc:290] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:21.490684  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 212183us
I20260709 13:05:21.575380 10201 leader_election.cc:290] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:21.578639 10188 log.cc:927] Time spent T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: Append to log took a long time: real 0.149s	user 0.003s	sys 0.000s
I20260709 13:05:21.574013 10189 raft_consensus.cc:697] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:21.547873  9863 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:21.590277  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12cd0294fa614c61b200754ffbbf7601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.592067 10197 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.592660 10197 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.593077 10197 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.614660  9687 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33620 (request call id 7) took 2238 ms. Trace:
I20260709 13:05:21.615418  9687 rpcz_store.cc:276] 0709 13:05:19.376330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.377467 (+  1137us) service_pool.cc:224] Handling call
0709 13:05:21.614546 (+2237079us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":348708,"reported_tablets":7}
I20260709 13:05:21.621474 10059 ts_tablet_manager.cc:1434] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.627s	user 0.019s	sys 0.003s
I20260709 13:05:21.572539 10134 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:21.624709 10059 ts_tablet_manager.cc:1467] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: Tablet startup took 627ms
I20260709 13:05:21.572962  9903 leader_election.cc:304] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
W20260709 13:05:21.625386 10059 ts_tablet_manager.cc:1469] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: Trace:
0709 13:05:17.040781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.040998 (+   217us) service_pool.cc:224] Handling call
0709 13:05:17.046071 (+  5073us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:17.048537 (+  2466us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:17.169028 (+120491us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:17.188807 (+ 19779us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:17.189363 (+   556us) inbound_call.cc:177] Queueing success response
0709 13:05:20.647275 (+3457912us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:20.994403 (+347128us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:21.429085 (+434682us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:21.429730 (+   645us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":63043,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":137,"tablet-open.queue_time_us":3458012}
I20260709 13:05:21.642122 10059 tablet_bootstrap.cc:492] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:21.641196 10134 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.643599 10134 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.638296 10012 raft_consensus.cc:2468] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:21.645514  9927 tablet_service.cc:1511] Processing CreateTablet for tablet b17aba5ba2fe46a49bec25310d428a26 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:21.646844  9755 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.647176  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17aba5ba2fe46a49bec25310d428a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.648260  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.650024  9783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:21.626860  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 47918.8us
I20260709 13:05:21.650727  9996 tablet_service.cc:1511] Processing CreateTablet for tablet b17aba5ba2fe46a49bec25310d428a26 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 13:05:21.651496  9978 leader_election.cc:343] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.652544 10203 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.653080 10203 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:21.653272  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 126416us
I20260709 13:05:21.653533 10203 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.631843 10182 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.652113  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 79129676db004c7bbe29ab7e9ec1fbe3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:21.657140  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79129676db004c7bbe29ab7e9ec1fbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.646183  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.662437  9922 tablet_service.cc:1511] Processing CreateTablet for tablet 79129676db004c7bbe29ab7e9ec1fbe3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:21.665711  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79129676db004c7bbe29ab7e9ec1fbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.665881  9754 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.668569  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17aba5ba2fe46a49bec25310d428a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.623699 10189 consensus_queue.cc:237] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:21.655400 10182 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.676556 10202 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.677328 10202 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.681921 10191 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:21.663146 10194 raft_consensus.cc:2804] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.683583 10115 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.683746 10194 raft_consensus.cc:493] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.684283 10194 raft_consensus.cc:3060] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.686408  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.687129  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.687657  9937 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:21.688999 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:21.689298  9755 leader_election.cc:343] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.570290  9685 rpcz_store.cc:276] 0709 13:05:19.359111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.360478 (+  1367us) service_pool.cc:224] Handling call
0709 13:05:21.539804 (+2179326us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:19.645109 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:19.645838 (+   729us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:19.645886 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:05:19.645908 (+    22us) tablet.cc:662] Decoding operations
0709 13:05:19.658165 (+ 12257us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:19.661930 (+  3765us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:19.667803 (+  5873us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:19.698105 (+ 30302us) tablet.cc:701] Row locks acquired
0709 13:05:19.708410 (+ 10305us) write_op.cc:260] PREPARE: finished
0709 13:05:19.708615 (+   205us) write_op.cc:270] Start()
0709 13:05:19.708746 (+   131us) write_op.cc:276] Timestamp: P: 1783602319708560 usec, L: 0
0709 13:05:19.708767 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:05:19.709596 (+   829us) log.cc:844] Serialized 540 byte log entry
0709 13:05:20.204693 (+495097us) op_driver.cc:464] REPLICATION: finished
0709 13:05:20.345610 (+140917us) write_op.cc:301] APPLY: starting
0709 13:05:20.345764 (+   154us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:20.346038 (+   274us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:20.346204 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:20.346493 (+   289us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:20.346703 (+   210us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:20.346751 (+    48us) write_op.cc:312] APPLY: finished
0709 13:05:20.347004 (+   253us) log.cc:844] Serialized 22 byte log entry
0709 13:05:20.356560 (+  9556us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.356657 (+    97us) write_op.cc:454] Released schema lock
0709 13:05:20.356711 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:20.357339 (+   628us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:20.357384 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":643076,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":128233,"apply.run_cpu_time_us":2243,"apply.run_wall_time_us":19444,"num_ops":1,"prepare.queue_time_us":65327,"prepare.run_cpu_time_us":3804,"prepare.run_wall_time_us":64839,"replication_time_us":453889,"thread_start_us":72608,"threads_started":2}]]}
I20260709 13:05:21.690088 10204 raft_consensus.cc:2804] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:21.695320 10204 raft_consensus.cc:697] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:21.696604 10198 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.697290 10198 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.697729 10198 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.697896 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.698954 10010 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:21.701371  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.696605 10204 consensus_queue.cc:237] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:21.702314  9999 tablet_service.cc:1511] Processing CreateTablet for tablet 625608f2b39c495d9f5b471dcabde2de (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:05:21.629650  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 101768us
W20260709 13:05:21.703680  9978 leader_election.cc:343] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.680007 10202 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:21.628548  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 65099.5us
I20260709 13:05:21.711303 10197 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:21.712836 10185 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.713985  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.714838  9787 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:21.716888 10186 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.719708  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.720235  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
W20260709 13:05:21.719731 10147 consensus_meta.cc:287] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: Time spent flushing consensus metadata: real 0.770s	user 0.024s	sys 0.000s
I20260709 13:05:21.720960  9788 raft_consensus.cc:2468] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:21.719781  9827 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.737272  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.738292  9937 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
W20260709 13:05:21.739334  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 598408us
I20260709 13:05:21.737352 10186 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.740680 10186 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.741163 10186 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.741420  9977 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:21.743657 10207 raft_consensus.cc:2804] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.728153  9755 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.744648 10207 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.745086 10207 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:21.747602  9903 leader_election.cc:343] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.748580 10201 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.748726  9903 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:21.749161 10201 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.714041 10197 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:21.728763  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625608f2b39c495d9f5b471dcabde2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.743810  9977 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:21.749578 10201 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.749795 10112 raft_consensus.cc:2804] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.754343 10112 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.749593 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:21.755327 10112 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.761266 10115 raft_consensus.cc:2804] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.749893 10147 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.761906 10115 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.762439  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.763478 10115 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.763746  9788 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:21.765893  9827 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.770100 10147 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:21.773082  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.773962  9937 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:21.774186 10134 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:21.776018  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.776913  9937 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:21.779692  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.780936 10187 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.781631 10187 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.783878  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:21.785054  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.786346  9788 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:21.798718 10201 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.812270 10059 tablet_bootstrap.cc:654] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.755333 10182 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.808531  9827 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:21.791941 10185 raft_consensus.cc:2804] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.815444 10185 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:21.816107 10185 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.819484 10194 raft_consensus.cc:515] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:21.820793  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:21.821573 10187 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.822299 10186 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.824904 10197 raft_consensus.cc:2804] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.825433 10197 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.825884 10197 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.825181  9853 tablet_service.cc:1511] Processing CreateTablet for tablet 79129676db004c7bbe29ab7e9ec1fbe3 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:21.827329  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79129676db004c7bbe29ab7e9ec1fbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.827104  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.828791  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.830842  9849 tablet_service.cc:1511] Processing CreateTablet for tablet b17aba5ba2fe46a49bec25310d428a26 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:21.792182 10195 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:21.836530 10058 tablet_bootstrap.cc:654] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:21.847290  9901 leader_election.cc:343] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:21.848599  9901 leader_election.cc:343] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.849166  9901 leader_election.cc:304] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:21.844270 10195 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.840067 10145 consensus_queue.cc:1048] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 13:05:21.850827  9901 leader_election.cc:343] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.851544  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.852241 10145 raft_consensus.cc:2749] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.852633  9859 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:21.852854  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:21.848976 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:21.855501 10010 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.856455  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17aba5ba2fe46a49bec25310d428a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.805526  9754 leader_election.cc:304] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
W20260709 13:05:21.858803  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 66750.6us
I20260709 13:05:21.860909  9829 leader_election.cc:304] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:21.862376 10182 raft_consensus.cc:2749] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.860958  9788 raft_consensus.cc:2393] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:21.863978 10203 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:21.866380 10011 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 2: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:21.867667  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.868067 10187 raft_consensus.cc:2749] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.868821  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:21.874421 10061 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:21.875192 10061 ts_tablet_manager.cc:1403] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.594s	user 0.037s	sys 0.007s
I20260709 13:05:21.881814 10194 leader_election.cc:290] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:21.886623  9684 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33610 (request call id 10) took 2315 ms. Trace:
I20260709 13:05:21.888159 10115 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:21.887794  9684 rpcz_store.cc:276] 0709 13:05:19.571365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.571633 (+   268us) service_pool.cc:224] Handling call
0709 13:05:21.886516 (+2314883us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":241005,"reported_tablets":2}
I20260709 13:05:21.897421  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:21.897734 10115 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.811057 10147 consensus_queue.cc:1048] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:05:21.901274 10057 tablet_bootstrap.cc:654] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:21.911784  9901 leader_election.cc:343] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.912686 10185 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.913690 10207 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.920912 10140 consensus_queue.cc:1048] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:05:21.933280 10207 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:21.876744 10112 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:21.933692 10197 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:21.936173 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.941411 10012 raft_consensus.cc:2393] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:21.941252 10201 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:21.945050 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:21.952746 10185 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:21.966842  9978 leader_election.cc:343] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.959082 10196 consensus_queue.cc:1048] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 13:05:21.968426 10112 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:21.952064 10195 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:21.941522  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:21.984964  9773 tablet_service.cc:1511] Processing CreateTablet for tablet 625608f2b39c495d9f5b471dcabde2de (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:21.987152  9773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625608f2b39c495d9f5b471dcabde2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:21.990114 10061 raft_consensus.cc:359] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:21.991091 10061 raft_consensus.cc:385] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.991390 10061 raft_consensus.cc:740] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.992074 10061 consensus_queue.cc:260] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:22.000844 10209 log.cc:927] Time spent T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:05:22.003639  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:22.004081  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:22.004165  9789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34158 (request call id 7) took 1154 ms. Trace:
I20260709 13:05:22.004638  9863 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:22.006512  9774 tablet_service.cc:1511] Processing CreateTablet for tablet 158c95c12c924f2ea6752771f4a48f0f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:22.008738  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 158c95c12c924f2ea6752771f4a48f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.009404 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:22.009559  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:22.010485 10012 raft_consensus.cc:2393] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:22.010604  9936 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:22.013298 10216 log.cc:927] Time spent T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:05:22.015237 10203 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:22.022156 10061 ts_tablet_manager.cc:1434] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.147s	user 0.011s	sys 0.002s
I20260709 13:05:22.023518 10061 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:22.025346  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:22.029192  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:22.030514  9785 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.034341  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.037796  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.039037 10197 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:22.039909  9937 raft_consensus.cc:2393] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:22.043597  9927 tablet_service.cc:1511] Processing CreateTablet for tablet 5673ca542b1745a38fdc5b65d4357b42 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:22.043656  9922 tablet_service.cc:1511] Processing CreateTablet for tablet 158c95c12c924f2ea6752771f4a48f0f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:22.043974  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.045804  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5673ca542b1745a38fdc5b65d4357b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.004910  9789 rpcz_store.cc:276] 0709 13:05:20.849948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.850483 (+   535us) service_pool.cc:224] Handling call
0709 13:05:20.850935 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:21.635606 (+784671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.637073 (+  1467us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:21.662358 (+ 25285us) log.cc:844] Serialized 26 byte log entry
0709 13:05:21.757380 (+ 95022us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:05:21.760232 (+  2852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.790911 (+ 30679us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.003760 (+212849us) raft_consensus.cc:1685] finished
0709 13:05:22.003796 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.004062 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":158280,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":108,"thread_start_us":305171,"threads_started":2,"wal-append.queue_time_us":222470}
I20260709 13:05:21.992779  9864 raft_consensus.cc:2393] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:21.979202 10084 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:22.029271 10198 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:22.069478  9787 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 2: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:22.070547  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:22.051196  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:22.052181  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.058081  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:22.074841  9850 tablet_service.cc:1511] Processing CreateTablet for tablet 625608f2b39c495d9f5b471dcabde2de (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:22.059294  9859 raft_consensus.cc:1240] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Rejecting Update request from peer d02e8a054a634be599149db641901736 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:22.060653  9776 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb3e93354824df58589f250b6264d54 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:22.078383  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb3e93354824df58589f250b6264d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.066393 10186 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:22.067366  9854 tablet_service.cc:1511] Processing CreateTablet for tablet 5673ca542b1745a38fdc5b65d4357b42 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:22.070960  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:22.084084  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:22.085501  9859 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:22.051435  9922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 158c95c12c924f2ea6752771f4a48f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.086817 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:22.092815 10010 raft_consensus.cc:2468] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:22.093721  9850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625608f2b39c495d9f5b471dcabde2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.095614  9854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5673ca542b1745a38fdc5b65d4357b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.096798  9926 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb3e93354824df58589f250b6264d54 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:22.098423  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb3e93354824df58589f250b6264d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.075605 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:22.070580 10198 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:22.104905 10059 tablet_bootstrap.cc:492] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:22.105758 10059 ts_tablet_manager.cc:1403] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.464s	user 0.026s	sys 0.024s
I20260709 13:05:22.109279 10059 raft_consensus.cc:359] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:22.110224 10059 raft_consensus.cc:385] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.110539 10059 raft_consensus.cc:740] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.111399 10059 consensus_queue.cc:260] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:22.121035 10061 tablet_bootstrap.cc:654] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:22.126621 10058 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:22.127655 10058 ts_tablet_manager.cc:1403] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 1.209s	user 0.042s	sys 0.066s
I20260709 13:05:22.134174 10059 ts_tablet_manager.cc:1434] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:05:22.132594 10058 raft_consensus.cc:359] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.142048 10058 raft_consensus.cc:385] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.142457 10058 raft_consensus.cc:740] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.105003 10009 raft_consensus.cc:3060] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.072083  9754 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:22.144982 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:22.164722 10012 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:22.158792 10058 consensus_queue.cc:260] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.105448  9978 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.161487  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:22.141253  9977 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:22.163607 10186 raft_consensus.cc:493] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.186064 10057 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:22.186712 10057 ts_tablet_manager.cc:1403] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 1.169s	user 0.026s	sys 0.025s
W20260709 13:05:22.188984 10033 debug-util.cc:398] Leaking SignalData structure 0x7b0800360e40 after lost signal to thread 434
W20260709 13:05:22.190490 10033 debug-util.cc:398] Leaking SignalData structure 0x7b080008ef80 after lost signal to thread 9721
I20260709 13:05:22.192255  9933 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.193836  9785 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:22.196750  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 91399.5us
I20260709 13:05:22.198220 10207 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.198953 10059 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:22.202417 10058 ts_tablet_manager.cc:1434] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.074s	user 0.010s	sys 0.007s
W20260709 13:05:22.200945 10033 debug-util.cc:398] Leaking SignalData structure 0x7b08001e66e0 after lost signal to thread 9813
I20260709 13:05:22.215310  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.207509 10189 raft_consensus.cc:2749] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:22.216730 10033 debug-util.cc:398] Leaking SignalData structure 0x7b08001b58e0 after lost signal to thread 9888
I20260709 13:05:22.200109 10198 raft_consensus.cc:2764] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:22.214215 10009 raft_consensus.cc:2468] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:22.205899 10186 raft_consensus.cc:515] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:22.214653  9936 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:22.205910  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:22.222667  9863 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
W20260709 13:05:22.163695  9904 leader_election.cc:343] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.223536 10186 leader_election.cc:290] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:22.223757  9904 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:22.224582  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 105394us
I20260709 13:05:22.225510 10112 raft_consensus.cc:2749] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:22.226078  9853 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb3e93354824df58589f250b6264d54 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 13:05:22.226891  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 63228.2us
W20260709 13:05:22.227847  9904 leader_election.cc:343] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.228269  9996 tablet_service.cc:1511] Processing CreateTablet for tablet 158c95c12c924f2ea6752771f4a48f0f (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:22.228304 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:22.228997  9904 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.229697 10012 raft_consensus.cc:2393] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:22.227852  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb3e93354824df58589f250b6264d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.229918  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 158c95c12c924f2ea6752771f4a48f0f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:22.233394  9904 leader_election.cc:343] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.233039  9684 catalog_manager.cc:5697] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } }
I20260709 13:05:22.235179  9903 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.233709  9998 tablet_service.cc:1511] Processing CreateTablet for tablet 5673ca542b1745a38fdc5b65d4357b42 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:22.238977  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5673ca542b1745a38fdc5b65d4357b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.235869 10002 tablet_service.cc:1511] Processing CreateTablet for tablet c68f14a359d54249b7cab91d0e63d7b0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:22.240818 10002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c68f14a359d54249b7cab91d0e63d7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.241345  9975 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
W20260709 13:05:22.242976  9978 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:22.185422  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 113417us
I20260709 13:05:22.244354 10189 raft_consensus.cc:2804] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:22.245455  9754 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:22.245671 10189 raft_consensus.cc:697] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:22.247308  9754 leader_election.cc:304] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:22.255513 10196 consensus_queue.cc:1059] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.253223 10207 consensus_queue.cc:1048] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:22.258054  9903 leader_election.cc:304] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:22.258955 10140 raft_consensus.cc:2804] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:22.252846 10033 debug-util.cc:398] Leaking SignalData structure 0x7b080016a500 after lost signal to thread 9962
I20260709 13:05:22.276189 10058 tablet_bootstrap.cc:492] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:22.283160 10057 raft_consensus.cc:359] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.285188 10057 raft_consensus.cc:385] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.285634 10057 raft_consensus.cc:740] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.282922 10140 raft_consensus.cc:697] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:22.298178 10140 consensus_queue.cc:237] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.280135 10201 raft_consensus.cc:2749] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.286427 10196 raft_consensus.cc:2804] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:22.305641  9774 tablet_service.cc:1511] Processing CreateTablet for tablet c68f14a359d54249b7cab91d0e63d7b0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:22.311017  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c68f14a359d54249b7cab91d0e63d7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.316817  9849 tablet_service.cc:1511] Processing CreateTablet for tablet c68f14a359d54249b7cab91d0e63d7b0 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:22.319196  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c68f14a359d54249b7cab91d0e63d7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.322171 10201 consensus_queue.cc:1048] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:22.322705 10033 debug-util.cc:398] Leaking SignalData structure 0x7b080021ac20 after lost signal to thread 10036
I20260709 13:05:22.324843  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:22.250011 10189 consensus_queue.cc:237] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.339061  9933 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:22.304322 10134 raft_consensus.cc:3055] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:05:22.310300  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 63083.5us
I20260709 13:05:22.341063 10112 raft_consensus.cc:2764] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:22.359915 10057 consensus_queue.cc:260] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.342545 10134 raft_consensus.cc:740] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:22.387701  9827 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.389781 10186 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:22.394778 10223 log.cc:927] Time spent T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:05:22.399441  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:22.399799  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582cebb9b4674942bb40b7be9936c3f5" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.384500 10198 raft_consensus.cc:2749] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:22.402578  9904 leader_election.cc:343] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.403105  9927 tablet_service.cc:1511] Processing CreateTablet for tablet e62c671eeefb49758edb7a7273fc6a2c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:22.403461  9904 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:22.403644 10134 consensus_queue.cc:260] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:22.405440 10134 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.406231 10194 raft_consensus.cc:2749] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:22.409185  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62c671eeefb49758edb7a7273fc6a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.410740 10059 tablet_bootstrap.cc:654] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:22.413834  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:22.419447  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:22.420673  9784 raft_consensus.cc:2393] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 2: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:22.384490 10196 raft_consensus.cc:697] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:22.422709 10196 consensus_queue.cc:237] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:22.426954  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:22.445549 10057 ts_tablet_manager.cc:1434] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.258s	user 0.016s	sys 0.002s
I20260709 13:05:22.447747 10057 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
W20260709 13:05:22.455211  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 52654.6us
I20260709 13:05:22.430459  9754 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:22.506441 10195 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.509984  9776 tablet_service.cc:1511] Processing CreateTablet for tablet e62c671eeefb49758edb7a7273fc6a2c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:22.511735  9776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62c671eeefb49758edb7a7273fc6a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.557205 10061 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:22.557960 10061 ts_tablet_manager.cc:1403] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.535s	user 0.034s	sys 0.021s
W20260709 13:05:22.512475  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 53309.3us
W20260709 13:05:22.539551  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 119502us
I20260709 13:05:22.582274  9829 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.583612 10186 raft_consensus.cc:2749] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.584173  9829 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.597605  9927 tablet_service.cc:1511] Processing CreateTablet for tablet e63bec87e7a64ac0bb9eea74f15908ba (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:22.598596 10186 raft_consensus.cc:2749] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.599305  9927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63bec87e7a64ac0bb9eea74f15908ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.588542 10195 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.603295  9859 raft_consensus.cc:1240] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Rejecting Update request from peer d02e8a054a634be599149db641901736 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:05:22.603762 10195 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.606534 10195 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:22.608711  9827 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.609946 10186 raft_consensus.cc:2749] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.612365  9859 raft_consensus.cc:1240] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Rejecting Update request from peer d02e8a054a634be599149db641901736 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:05:22.617666 10112 consensus_queue.cc:1059] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.618564  9774 tablet_service.cc:1511] Processing CreateTablet for tablet fca2ecb398d14bd48a7c452af841da65 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:22.620440  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca2ecb398d14bd48a7c452af841da65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.620256  9996 tablet_service.cc:1511] Processing CreateTablet for tablet e63bec87e7a64ac0bb9eea74f15908ba (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:22.618758 10112 raft_consensus.cc:3055] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:22.639155 10112 raft_consensus.cc:740] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:22.618551 10140 consensus_queue.cc:1059] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:05:22.640749 10140 raft_consensus.cc:3055] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:22.641177 10140 raft_consensus.cc:740] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:22.642349 10140 consensus_queue.cc:260] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:22.643406  9853 tablet_service.cc:1511] Processing CreateTablet for tablet e63bec87e7a64ac0bb9eea74f15908ba (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:22.645102  9853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63bec87e7a64ac0bb9eea74f15908ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.646837 10112 consensus_queue.cc:260] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:22.632803  9754 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:22.649669  9999 tablet_service.cc:1511] Processing CreateTablet for tablet e62c671eeefb49758edb7a7273fc6a2c (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:22.659292 10112 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.654314 10229 raft_consensus.cc:493] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.649727  9997 tablet_service.cc:1511] Processing CreateTablet for tablet 404c2d54fc4e4b6c8a892de6e4988671 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:22.652648 10061 raft_consensus.cc:359] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:22.664666 10061 raft_consensus.cc:385] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.660691 10057 tablet_bootstrap.cc:654] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:22.665068 10061 raft_consensus.cc:740] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.660966 10058 tablet_bootstrap.cc:654] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:22.661924  9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63bec87e7a64ac0bb9eea74f15908ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.669766 10061 consensus_queue.cc:260] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:22.644078 10140 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.664007 10195 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.687871 10061 ts_tablet_manager.cc:1434] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.130s	user 0.007s	sys 0.005s
I20260709 13:05:22.711867 10061 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:22.714648 10191 raft_consensus.cc:493] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.715449 10191 raft_consensus.cc:515] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.698398  9926 tablet_service.cc:1511] Processing CreateTablet for tablet fca2ecb398d14bd48a7c452af841da65 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:22.722919  9926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca2ecb398d14bd48a7c452af841da65. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:22.676939  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 51173.6us
W20260709 13:05:22.697655  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:22.668365 10229 raft_consensus.cc:515] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:22.752308 10057 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:22.753618 10057 ts_tablet_manager.cc:1403] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.306s	user 0.014s	sys 0.005s
I20260709 13:05:22.737821 10198 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.764219 10059 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:22.765112 10059 ts_tablet_manager.cc:1403] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.566s	user 0.032s	sys 0.034s
I20260709 13:05:22.690038  9775 tablet_service.cc:1511] Processing CreateTablet for tablet 404c2d54fc4e4b6c8a892de6e4988671 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:22.773958  9775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404c2d54fc4e4b6c8a892de6e4988671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.777292  9849 tablet_service.cc:1511] Processing CreateTablet for tablet fca2ecb398d14bd48a7c452af841da65 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:22.779244  9849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca2ecb398d14bd48a7c452af841da65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.781445  9854 tablet_service.cc:1511] Processing CreateTablet for tablet 404c2d54fc4e4b6c8a892de6e4988671 (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:22.701120  9925 tablet_service.cc:1511] Processing CreateTablet for tablet b3695e4783714136ad332a175348e9fc (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:22.782967 10058 tablet_bootstrap.cc:492] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:22.783190  9854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404c2d54fc4e4b6c8a892de6e4988671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.783540 10058 ts_tablet_manager.cc:1403] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.508s	user 0.005s	sys 0.017s
I20260709 13:05:22.783808  9925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3695e4783714136ad332a175348e9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.785766  9774 tablet_service.cc:1511] Processing CreateTablet for tablet b3695e4783714136ad332a175348e9fc (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:22.786968  9785 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.787274  9774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3695e4783714136ad332a175348e9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.766978 10198 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:22.795342 10198 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:22.795392 10229 leader_election.cc:290] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:22.801044 10229 consensus_queue.cc:1048] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:22.802194 10002 tablet_service.cc:1511] Processing CreateTablet for tablet b3695e4783714136ad332a175348e9fc (DEFAULT_TABLE table=test-workload [id=8f40ac778b504d3990666303829ed94b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:22.793103  9997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404c2d54fc4e4b6c8a892de6e4988671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.767905  9754 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:22.808313 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:22.808774 10191 leader_election.cc:290] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:22.809302 10009 raft_consensus.cc:2468] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:22.809679 10002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3695e4783714136ad332a175348e9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.813969  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:22.815614  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 52796.6us
I20260709 13:05:22.805902  9829 leader_election.cc:304] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:22.821041 10058 raft_consensus.cc:359] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:22.825012  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:22.816843  9754 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
W20260709 13:05:22.822870  9901 leader_election.cc:343] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.822695  9999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62c671eeefb49758edb7a7273fc6a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:22.790591 10061 tablet_bootstrap.cc:654] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:22.823858 10186 raft_consensus.cc:3060] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.832640 10058 raft_consensus.cc:385] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.839248 10058 raft_consensus.cc:740] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.842013 10058 consensus_queue.cc:260] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:22.847635 10201 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:22.848436 10201 raft_consensus.cc:697] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:22.849503 10201 consensus_queue.cc:237] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:22.855006  9755 leader_election.cc:343] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.831351 10198 raft_consensus.cc:2749] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.869452  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.870349  9936 raft_consensus.cc:2468] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:22.865927 10057 raft_consensus.cc:359] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:22.873734 10057 raft_consensus.cc:385] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.874142 10057 raft_consensus.cc:740] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.876861 10057 consensus_queue.cc:260] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:22.882455 10058 ts_tablet_manager.cc:1434] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.099s	user 0.014s	sys 0.000s
I20260709 13:05:22.885946  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:22.889937  9903 leader_election.cc:304] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.892839 10147 raft_consensus.cc:2804] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:22.893070  9975 leader_election.cc:343] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.893476 10147 raft_consensus.cc:493] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.893944 10147 raft_consensus.cc:3060] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.894768 10242 consensus_queue.cc:1048] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.902004  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.903409  9934 raft_consensus.cc:2468] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:22.904960  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:22.905903 10238 consensus_queue.cc:1048] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.907792  9754 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:22.908160 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:22.909487 10057 ts_tablet_manager.cc:1434] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.155s	user 0.009s	sys 0.001s
I20260709 13:05:22.910353 10238 raft_consensus.cc:2804] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:22.910799 10238 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.911185 10238 raft_consensus.cc:3060] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.912955  9977 leader_election.cc:304] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:22.914166 10191 raft_consensus.cc:2804] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:22.914638 10191 raft_consensus.cc:493] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.915064 10191 raft_consensus.cc:3060] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.928992 10058 tablet_bootstrap.cc:492] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:22.910806 10057 tablet_bootstrap.cc:492] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:22.974728 10059 raft_consensus.cc:359] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:22.943087 10195 raft_consensus.cc:2749] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.975761 10059 raft_consensus.cc:385] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.977468 10059 raft_consensus.cc:740] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.990527 10059 consensus_queue.cc:260] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:22.952646  9754 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:23.000288 10059 ts_tablet_manager.cc:1434] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.235s	user 0.006s	sys 0.003s
I20260709 13:05:23.004042 10057 tablet_bootstrap.cc:654] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.010165 10186 raft_consensus.cc:2749] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.016767 10198 raft_consensus.cc:2749] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.034646 10229 consensus_queue.cc:1048] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:23.032893  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 99449us
I20260709 13:05:23.025243 10059 tablet_bootstrap.cc:492] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:23.024170 10058 tablet_bootstrap.cc:654] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.041493 10140 consensus_queue.cc:1048] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:23.070884 10229 consensus_queue.cc:1048] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:23.082805 10145 raft_consensus.cc:493] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.083523 10145 raft_consensus.cc:515] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.085927 10061 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:23.086465 10145 leader_election.cc:290] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:23.086782 10061 ts_tablet_manager.cc:1403] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.375s	user 0.023s	sys 0.034s
I20260709 13:05:23.090322 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.090662  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:23.094247  9904 leader_election.cc:343] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.094338 10061 raft_consensus.cc:359] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.095414 10061 raft_consensus.cc:385] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.095801 10061 raft_consensus.cc:740] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.096798 10061 consensus_queue.cc:260] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.101524 10061 ts_tablet_manager.cc:1434] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:05:23.102533  9685 catalog_manager.cc:5697] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.103564 10191 raft_consensus.cc:515] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.104005 10061 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:23.053066 10147 raft_consensus.cc:515] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:23.110194  9903 leader_election.cc:343] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.111819  9903 leader_election.cc:304] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:23.115665 10145 raft_consensus.cc:2749] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.115948 10092 consensus_queue.cc:1048] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:05:23.125452  9684 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33610 (request call id 11) took 1024 ms. Trace:
I20260709 13:05:23.147472  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:23.148610  9934 raft_consensus.cc:2393] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:23.149549 10059 tablet_bootstrap.cc:654] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.150393 10147 leader_election.cc:290] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:23.151985  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:23.152910  9788 raft_consensus.cc:3060] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.152189  9685 catalog_manager.cc:5697] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.159054 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:23.160454 10012 raft_consensus.cc:2393] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:23.170632 10061 tablet_bootstrap.cc:654] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.199508 10189 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.201460 10189 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.155857 10238 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.212239 10057 tablet_bootstrap.cc:492] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:23.221019 10057 ts_tablet_manager.cc:1403] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.310s	user 0.019s	sys 0.028s
I20260709 13:05:23.225188  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:23.227128  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:23.230703  9975 leader_election.cc:304] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:23.231999  9975 leader_election.cc:343] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.233377 10189 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.233454 10242 raft_consensus.cc:2749] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.237166 10186 raft_consensus.cc:493] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.237959  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:23.241438  9978 leader_election.cc:343] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.242224  9978 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.246937 10242 raft_consensus.cc:2749] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.237911 10186 raft_consensus.cc:515] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.252442 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.254959  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:23.255963 10186 leader_election.cc:290] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:23.258278  9829 leader_election.cc:304] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.259227 10238 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:23.220954 10191 leader_election.cc:290] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:23.146814  9684 rpcz_store.cc:276] 0709 13:05:22.101032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.101201 (+   169us) service_pool.cc:224] Handling call
0709 13:05:23.123444 (+1022243us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:22.500810 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:22.501790 (+   980us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:22.501850 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:05:22.501879 (+    29us) tablet.cc:662] Decoding operations
0709 13:05:22.502409 (+   530us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:22.502512 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:22.502558 (+    46us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:22.502874 (+   316us) tablet.cc:701] Row locks acquired
0709 13:05:22.502896 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:05:22.503143 (+   247us) write_op.cc:270] Start()
0709 13:05:22.503301 (+   158us) write_op.cc:276] Timestamp: P: 1783602322503071 usec, L: 0
0709 13:05:22.503333 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:05:22.566158 (+ 62825us) log.cc:844] Serialized 540 byte log entry
0709 13:05:22.836202 (+270044us) op_driver.cc:464] REPLICATION: finished
0709 13:05:22.987091 (+150889us) write_op.cc:301] APPLY: starting
0709 13:05:22.990125 (+  3034us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:22.996216 (+  6091us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:23.016964 (+ 20748us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:23.017316 (+   352us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:23.017540 (+   224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:23.017597 (+    57us) write_op.cc:312] APPLY: finished
0709 13:05:23.017865 (+   268us) log.cc:844] Serialized 22 byte log entry
0709 13:05:23.018100 (+   235us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:23.018214 (+   114us) write_op.cc:454] Released schema lock
0709 13:05:23.018264 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:23.018733 (+   469us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:23.018774 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":142269,"apply.run_cpu_time_us":2696,"apply.run_wall_time_us":38032,"num_ops":1,"prepare.queue_time_us":193064,"prepare.run_cpu_time_us":19437,"prepare.run_wall_time_us":186201,"raft.queue_time_us":117658,"raft.run_cpu_time_us":20611,"raft.run_wall_time_us":134965,"replication_time_us":332698,"thread_start_us":381128,"threads_started":4,"wal-append.queue_time_us":146034}]]}
I20260709 13:05:23.261157  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:23.263409  9755 leader_election.cc:343] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.255769 10057 raft_consensus.cc:359] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.266798 10057 raft_consensus.cc:385] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.267246 10057 raft_consensus.cc:740] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.268270 10057 consensus_queue.cc:260] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.271338 10057 ts_tablet_manager.cc:1434] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:05:23.272800 10057 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:23.263789  9788 raft_consensus.cc:2468] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:23.278152 10195 consensus_queue.cc:1048] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:23.278848  9901 leader_election.cc:304] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:23.280526 10229 raft_consensus.cc:2804] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:23.264524 10186 raft_consensus.cc:2749] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.281282 10229 raft_consensus.cc:697] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:23.282008  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:23.282752 10057 tablet_bootstrap.cc:654] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.283486  9936 raft_consensus.cc:3060] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.302469 10058 tablet_bootstrap.cc:492] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:23.303561 10058 ts_tablet_manager.cc:1403] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.375s	user 0.038s	sys 0.021s
I20260709 13:05:23.282474 10229 consensus_queue.cc:237] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.316951 10058 raft_consensus.cc:359] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.318326 10058 raft_consensus.cc:385] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.318753 10058 raft_consensus.cc:740] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.328861 10058 consensus_queue.cc:260] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.343115 10059 tablet_bootstrap.cc:492] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:23.343811 10059 ts_tablet_manager.cc:1403] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.339s	user 0.012s	sys 0.028s
I20260709 13:05:23.349598 10061 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:23.350373 10061 ts_tablet_manager.cc:1403] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.248s	user 0.035s	sys 0.021s
I20260709 13:05:23.354008 10061 raft_consensus.cc:359] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.354943 10061 raft_consensus.cc:385] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.349663 10058 ts_tablet_manager.cc:1434] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.000s
I20260709 13:05:23.355278 10061 raft_consensus.cc:740] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.357084 10058 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:23.357200 10061 consensus_queue.cc:260] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.357669 10059 raft_consensus.cc:359] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.358685 10059 raft_consensus.cc:385] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.359091 10059 raft_consensus.cc:740] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.360544 10229 raft_consensus.cc:493] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.361208 10229 raft_consensus.cc:515] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.360127 10059 consensus_queue.cc:260] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.365648 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.367410 10059 ts_tablet_manager.cc:1434] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
W20260709 13:05:23.367856  9903 leader_election.cc:343] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.368789 10059 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:23.368799  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:23.371224  9901 leader_election.cc:343] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.371785  9901 leader_election.cc:304] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.372488 10061 ts_tablet_manager.cc:1434] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.002s
I20260709 13:05:23.372912 10112 raft_consensus.cc:2749] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.373848 10061 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:23.390791 10057 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:23.380231  9687 catalog_manager.cc:5697] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.391450 10057 ts_tablet_manager.cc:1403] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.119s	user 0.023s	sys 0.017s
I20260709 13:05:23.388942  9863 raft_consensus.cc:3060] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.393185  9687 catalog_manager.cc:5697] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.383131 10229 leader_election.cc:290] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:23.397334 10058 tablet_bootstrap.cc:654] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.401444 10057 raft_consensus.cc:359] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.402419 10057 raft_consensus.cc:385] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.402745 10057 raft_consensus.cc:740] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.404201  9936 raft_consensus.cc:2468] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:23.405364  9859 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.405900 10059 tablet_bootstrap.cc:654] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.409636  9754 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:23.411724 10201 raft_consensus.cc:2804] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:23.412410 10201 raft_consensus.cc:697] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:23.413463 10201 consensus_queue.cc:237] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.415827 10057 consensus_queue.cc:260] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.425590 10057 ts_tablet_manager.cc:1434] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.034s	user 0.011s	sys 0.001s
I20260709 13:05:23.429147 10061 tablet_bootstrap.cc:654] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.429294  9684 catalog_manager.cc:5697] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } }
I20260709 13:05:23.432258 10198 consensus_queue.cc:1048] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:23.433156 10260 raft_consensus.cc:493] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.434284 10260 raft_consensus.cc:515] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.436189 10229 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.436939 10229 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.438450 10260 leader_election.cc:290] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:23.439129  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:23.441958  9975 leader_election.cc:343] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.443545  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:23.444595  9934 raft_consensus.cc:2468] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:23.445348 10058 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:23.446017 10058 ts_tablet_manager.cc:1403] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.000s
I20260709 13:05:23.447216  9977 leader_election.cc:304] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.448726 10260 raft_consensus.cc:2804] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:23.449169 10260 raft_consensus.cc:493] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.449532 10260 raft_consensus.cc:3060] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.449694 10058 raft_consensus.cc:359] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.450574 10058 raft_consensus.cc:385] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.450860 10058 raft_consensus.cc:740] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.432104 10057 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:23.456947 10058 consensus_queue.cc:260] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.456853  9683 catalog_manager.cc:5697] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } }
I20260709 13:05:23.460083  9683 catalog_manager.cc:5697] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } }
I20260709 13:05:23.462903  9683 catalog_manager.cc:5697] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } }
I20260709 13:05:23.464975 10112 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.466179 10207 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.466089 10112 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.466771 10207 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.469271 10207 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:23.470526  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:23.472930  9901 leader_election.cc:343] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.475770  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:23.479602  9978 leader_election.cc:343] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.472075  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:23.481592 10058 ts_tablet_manager.cc:1434] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.000s
I20260709 13:05:23.483845 10058 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:23.485045 10112 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.489603 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.490025 10057 tablet_bootstrap.cc:654] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.481668  9934 raft_consensus.cc:2468] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:23.491060 10012 raft_consensus.cc:2468] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:23.492787  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:23.493777  9903 leader_election.cc:304] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:23.496616  9977 leader_election.cc:304] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:23.498699 10207 raft_consensus.cc:2804] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:23.499287 10207 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.500620 10207 raft_consensus.cc:3060] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.490415 10229 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:23.491284  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:23.508487 10260 raft_consensus.cc:515] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.503211 10112 raft_consensus.cc:2804] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:23.511896 10112 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.513134 10112 raft_consensus.cc:3060] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.518124 10058 tablet_bootstrap.cc:654] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:23.529284  9904 leader_election.cc:343] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:23.530800  9904 leader_election.cc:343] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.531399  9904 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.532548 10140 raft_consensus.cc:2749] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.535759  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:23.536677  9934 raft_consensus.cc:3060] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.539278 10059 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:23.539897 10059 ts_tablet_manager.cc:1403] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.171s	user 0.028s	sys 0.009s
I20260709 13:05:23.545125 10260 leader_election.cc:290] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:23.546363 10059 raft_consensus.cc:359] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.546829  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
W20260709 13:05:23.548451  9975 leader_election.cc:343] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.553885  9934 raft_consensus.cc:2468] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:23.562700 10059 raft_consensus.cc:385] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.563206 10059 raft_consensus.cc:740] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.563753  9977 leader_election.cc:304] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.565434 10260 raft_consensus.cc:2804] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:23.565027 10112 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.566601 10260 raft_consensus.cc:697] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:23.567083  9685 catalog_manager.cc:5697] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } }
I20260709 13:05:23.568526 10260 consensus_queue.cc:237] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.571246 10061 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:23.572039 10061 ts_tablet_manager.cc:1403] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.198s	user 0.016s	sys 0.009s
I20260709 13:05:23.560859 10207 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.569589 10059 consensus_queue.cc:260] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.577531  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:23.578274 10140 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.579111 10140 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:23.580065  9904 leader_election.cc:343] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.581103 10057 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:23.582154 10057 ts_tablet_manager.cc:1403] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.150s	user 0.020s	sys 0.004s
I20260709 13:05:23.583407 10059 ts_tablet_manager.cc:1434] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.001s
I20260709 13:05:23.584520 10059 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:23.586257  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:23.588476 10057 raft_consensus.cc:359] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.589401 10057 raft_consensus.cc:385] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.590176 10057 raft_consensus.cc:740] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.591064 10057 consensus_queue.cc:260] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.594746  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:23.595686  9859 raft_consensus.cc:2468] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:23.597764 10057 ts_tablet_manager.cc:1434] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 13:05:23.599166 10057 tablet_bootstrap.cc:492] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:23.606107  9904 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.607699 10059 tablet_bootstrap.cc:654] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.590620 10140 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.609040 10229 raft_consensus.cc:2804] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:23.609498 10229 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.609854 10229 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:23.613332 10112 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:23.619339 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:23.620447 10012 raft_consensus.cc:2393] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:23.626920 10058 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:23.627978 10058 ts_tablet_manager.cc:1403] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.145s	user 0.014s	sys 0.008s
I20260709 13:05:23.630750  9903 leader_election.cc:304] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:23.631569 10058 raft_consensus.cc:359] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.632280 10058 raft_consensus.cc:385] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.632665 10058 raft_consensus.cc:740] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.637451 10057 tablet_bootstrap.cc:654] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.647035 10207 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:23.651830 10140 raft_consensus.cc:2749] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.654764  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:23.655910 10229 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:23.658798  9978 leader_election.cc:343] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.654762  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:23.637511 10058 consensus_queue.cc:260] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.664968 10229 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.665050 10112 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.669234 10112 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.673000  9934 raft_consensus.cc:2393] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:23.672832  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:23.675369  9977 leader_election.cc:304] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:23.675237  9684 catalog_manager.cc:5697] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.679036 10191 raft_consensus.cc:2749] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.680197 10112 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.681133  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:23.684437  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:23.686105  9904 leader_election.cc:343] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:23.687700  9901 leader_election.cc:343] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.688273  9901 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.688812 10059 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:23.689520 10059 ts_tablet_manager.cc:1403] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.003s
I20260709 13:05:23.690611 10112 raft_consensus.cc:2749] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.681103  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:23.693288 10059 raft_consensus.cc:359] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.694236 10059 raft_consensus.cc:385] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.694064  9859 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:23.701105 10058 ts_tablet_manager.cc:1434] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.000s
I20260709 13:05:23.706288  9687 catalog_manager.cc:5697] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.708518 10059 raft_consensus.cc:740] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.710573 10058 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:23.710326 10059 consensus_queue.cc:260] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.715479  9859 raft_consensus.cc:2468] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:05:23.719630  9904 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.722376 10140 raft_consensus.cc:2804] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:23.722647 10112 consensus_queue.cc:1048] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:23.723780 10140 raft_consensus.cc:697] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 3 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:23.726814 10059 ts_tablet_manager.cc:1434] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.002s
I20260709 13:05:23.728956 10059 tablet_bootstrap.cc:492] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:23.728880 10140 consensus_queue.cc:237] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:23.732982 10057 tablet_bootstrap.cc:492] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:23.751644 10057 ts_tablet_manager.cc:1403] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.153s	user 0.019s	sys 0.037s
I20260709 13:05:23.729187 10061 raft_consensus.cc:359] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.757295 10061 raft_consensus.cc:385] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.757658 10061 raft_consensus.cc:740] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.759274 10061 consensus_queue.cc:260] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.755996 10057 raft_consensus.cc:359] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.763836 10057 raft_consensus.cc:385] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:23.768234  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:23.776590 10057 raft_consensus.cc:740] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.777354 10195 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.778045 10057 consensus_queue.cc:260] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.778386 10195 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.763193 10061 ts_tablet_manager.cc:1434] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.190s	user 0.005s	sys 0.004s
I20260709 13:05:23.782697 10061 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:23.738202 10058 tablet_bootstrap.cc:654] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.784871 10059 tablet_bootstrap.cc:654] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.787285 10057 ts_tablet_manager.cc:1434] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.006s
I20260709 13:05:23.789011 10057 tablet_bootstrap.cc:492] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:23.775372  9685 catalog_manager.cc:5697] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:23.795158 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.796276 10009 raft_consensus.cc:2468] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:23.801098  9754 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:23.804039 10061 tablet_bootstrap.cc:654] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.804970 10201 raft_consensus.cc:2804] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:23.805563 10201 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.805961 10201 raft_consensus.cc:3060] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.809491 10191 raft_consensus.cc:493] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.809963 10195 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.810200 10191 raft_consensus.cc:515] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.818781 10057 tablet_bootstrap.cc:654] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.830363  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:23.831286  9859 raft_consensus.cc:2468] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:23.832799 10059 tablet_bootstrap.cc:492] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:23.833434 10059 ts_tablet_manager.cc:1403] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.105s	user 0.012s	sys 0.009s
I20260709 13:05:23.837100 10191 leader_election.cc:290] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.840132  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:23.841609  9785 raft_consensus.cc:2468] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:23.842211  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:23.843890 10186 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.845978 10059 raft_consensus.cc:359] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.849218 10059 raft_consensus.cc:385] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.848073 10186 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.852279  9975 leader_election.cc:304] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:23.852948 10186 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:23.844267 10195 consensus_queue.cc:1048] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:05:23.853577 10186 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.858808 10186 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:23.859362  9978 leader_election.cc:343] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.853827 10191 raft_consensus.cc:2804] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:23.861018 10191 raft_consensus.cc:493] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.862042 10191 raft_consensus.cc:3060] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.850299 10059 raft_consensus.cc:740] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.878371 10186 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:23.880376  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:23.884186 10140 consensus_queue.cc:1048] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:23.885324 10059 consensus_queue.cc:260] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.899554 10059 ts_tablet_manager.cc:1434] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.003s
I20260709 13:05:23.907159 10061 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:23.907886 10061 ts_tablet_manager.cc:1403] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.126s	user 0.017s	sys 0.015s
I20260709 13:05:23.909911 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.910930 10012 raft_consensus.cc:2393] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:23.891431 10191 raft_consensus.cc:515] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.912279  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:23.913311  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:23.913307  9936 raft_consensus.cc:2393] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:23.916267 10191 leader_election.cc:290] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.915549  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:23.917248  9785 raft_consensus.cc:3060] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.920655 10282 raft_consensus.cc:493] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.910332  9683 catalog_manager.cc:5697] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:23.921461 10282 raft_consensus.cc:515] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:23.924469  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:23.927719 10282 leader_election.cc:290] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:23.930677 10061 raft_consensus.cc:359] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.962074 10061 raft_consensus.cc:385] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:23.962626 10061 raft_consensus.cc:740] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:23.938614  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:05:23.939154  9978 leader_election.cc:343] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:23.943642 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:23.952088  9859 raft_consensus.cc:3060] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:23.954056 10059 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:23.919632 10201 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:23.957410  9829 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:23.964834  9936 raft_consensus.cc:2468] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 0.
I20260709 13:05:23.966586 10058 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:24.016775 10058 ts_tablet_manager.cc:1403] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.306s	user 0.024s	sys 0.034s
I20260709 13:05:23.964229 10061 consensus_queue.cc:260] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:24.021190 10058 raft_consensus.cc:359] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:24.022320 10058 raft_consensus.cc:385] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.022863 10058 raft_consensus.cc:740] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.023819 10059 tablet_bootstrap.cc:654] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:23.991514 10201 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:23.966328 10012 raft_consensus.cc:2468] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 0.
I20260709 13:05:24.025189 10058 consensus_queue.cc:260] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:23.994588 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:24.027637 10009 raft_consensus.cc:3060] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.022362 10061 ts_tablet_manager.cc:1434] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.114s	user 0.009s	sys 0.000s
I20260709 13:05:24.029517 10058 ts_tablet_manager.cc:1434] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.005s
I20260709 13:05:24.004693 10191 consensus_queue.cc:1048] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:05:24.037309 10058 tablet_bootstrap.cc:492] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:24.039841  9788 raft_consensus.cc:3060] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.042788 10061 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:24.050974 10058 tablet_bootstrap.cc:654] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.046031 10282 raft_consensus.cc:2749] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:23.996958  9829 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:24.052601 10186 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.059423 10057 tablet_bootstrap.cc:492] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:24.060025 10057 ts_tablet_manager.cc:1403] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.271s	user 0.030s	sys 0.015s
I20260709 13:05:24.061769 10061 tablet_bootstrap.cc:654] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.024685  9785 raft_consensus.cc:2468] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:24.077847  9975 leader_election.cc:304] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:24.078208 10186 raft_consensus.cc:2749] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:24.082907 10191 raft_consensus.cc:2804] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:24.083689 10191 raft_consensus.cc:697] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:24.085631 10191 consensus_queue.cc:237] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.081453 10057 raft_consensus.cc:359] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.090773 10057 raft_consensus.cc:385] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:24.070255  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 73391.8us
I20260709 13:05:24.094048 10260 consensus_queue.cc:1048] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:24.091308 10057 raft_consensus.cc:740] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.101713 10057 consensus_queue.cc:260] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.114390 10057 ts_tablet_manager.cc:1434] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.003s
I20260709 13:05:24.116679 10057 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:24.128511 10061 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:24.133740 10061 ts_tablet_manager.cc:1403] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.104s	user 0.024s	sys 0.020s
I20260709 13:05:24.105997 10201 consensus_queue.cc:1048] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:05:24.121882 10195 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:24.122805 10009 raft_consensus.cc:2468] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:24.145488 10059 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:24.143560  9829 leader_election.cc:304] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:24.148641  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:24.142169 10195 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.161197 10059 ts_tablet_manager.cc:1403] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.207s	user 0.008s	sys 0.012s
I20260709 13:05:24.159078  9685 catalog_manager.cc:5697] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:24.165149 10058 tablet_bootstrap.cc:492] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:24.165771 10058 ts_tablet_manager.cc:1403] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.129s	user 0.023s	sys 0.018s
I20260709 13:05:24.164925 10186 raft_consensus.cc:2804] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:24.167622 10186 raft_consensus.cc:493] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.168030 10186 raft_consensus.cc:3060] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.171051 10195 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.173693  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.174734  9936 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:24.176318  9754 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:24.168076  9859 raft_consensus.cc:3060] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.180421 10195 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.180998 10195 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:24.181473 10195 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.185173 10059 raft_consensus.cc:359] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.188452 10059 raft_consensus.cc:385] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.188776 10059 raft_consensus.cc:740] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.189481 10059 consensus_queue.cc:260] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.185683  9754 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:24.193209 10238 raft_consensus.cc:2804] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:24.193810 10238 raft_consensus.cc:697] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:24.194875 10238 consensus_queue.cc:237] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.197808  9861 raft_consensus.cc:1275] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:24.204700 10284 consensus_queue.cc:1048] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:24.205408 10186 raft_consensus.cc:515] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.209383 10058 raft_consensus.cc:359] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.197973  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.210561 10058 raft_consensus.cc:385] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.210956 10058 raft_consensus.cc:740] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.211115  9864 raft_consensus.cc:2393] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:24.211309 10059 ts_tablet_manager.cc:1434] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:05:24.212527 10059 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:24.217808  9859 raft_consensus.cc:2468] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:24.224956 10057 tablet_bootstrap.cc:654] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.224491 10058 consensus_queue.cc:260] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.226971 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:24.228487 10201 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:24.251515 10201 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.253253  9685 catalog_manager.cc:5697] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:24.254253 10201 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.255187 10059 tablet_bootstrap.cc:654] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.229887  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:24.255800  9936 raft_consensus.cc:3060] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.256317  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.255379  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.210244 10186 leader_election.cc:290] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:24.251377 10009 raft_consensus.cc:3060] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.262759  9755 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:24.263912 10201 raft_consensus.cc:2749] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.233845 10195 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.228262 10061 raft_consensus.cc:359] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.264739 10191 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.274178 10191 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.275837 10058 ts_tablet_manager.cc:1434] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.000s
I20260709 13:05:24.269237 10061 raft_consensus.cc:385] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.277670 10061 raft_consensus.cc:740] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.279323 10058 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:24.279466 10061 consensus_queue.cc:260] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.282325  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:24.285084 10260 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.285766 10260 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.265497  9785 raft_consensus.cc:3060] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.293896 10061 ts_tablet_manager.cc:1434] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.160s	user 0.011s	sys 0.000s
I20260709 13:05:24.273010 10112 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.270865 10195 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.299551 10057 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:24.304570 10112 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.305289 10057 ts_tablet_manager.cc:1403] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.189s	user 0.022s	sys 0.001s
I20260709 13:05:24.297290 10061 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:24.308948 10112 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:24.272496  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:24.314754  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:24.320926  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:24.322955  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.327332 10058 tablet_bootstrap.cc:654] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.328450  9789 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:24.329515  9784 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:24.339303  9901 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:24.333741  9861 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:24.326894  9936 raft_consensus.cc:2468] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:24.341429 10284 raft_consensus.cc:2804] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:24.342046 10284 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.343222 10191 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.344367  9829 leader_election.cc:304] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:24.343739 10284 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.345341 10195 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.345937 10195 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.346537 10057 raft_consensus.cc:359] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.347216  9975 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:24.347553 10057 raft_consensus.cc:385] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.347915 10057 raft_consensus.cc:740] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.348435  9859 raft_consensus.cc:2393] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:24.351747 10191 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:24.349668 10186 raft_consensus.cc:2804] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:24.355068 10186 raft_consensus.cc:697] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 1 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:24.355883  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.356894 10057 consensus_queue.cc:260] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.357131  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.358340  9861 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:24.356281 10186 consensus_queue.cc:237] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.352474 10260 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:24.360124  9978 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:24.351374 10195 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.353777 10191 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.365180 10191 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.368654 10260 raft_consensus.cc:2804] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.369442 10260 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.369951 10260 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.357066  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.379267  9859 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:24.360000 10057 ts_tablet_manager.cc:1434] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:05:24.381554 10057 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:24.357005  9936 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:24.387482 10059 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:24.388427 10059 ts_tablet_manager.cc:1403] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.176s	user 0.018s	sys 0.018s
I20260709 13:05:24.399088 10140 raft_consensus.cc:493] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.399935 10140 raft_consensus.cc:515] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:24.410007 10061 tablet_bootstrap.cc:654] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.403903 10140 leader_election.cc:290] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:24.393590 10009 raft_consensus.cc:2468] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:24.413388 10186 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.414542  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.415467  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.423323  9936 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:24.425575 10058 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:24.426345 10058 ts_tablet_manager.cc:1403] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.149s	user 0.014s	sys 0.012s
I20260709 13:05:24.436095  9934 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:24.438603 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:24.428846 10186 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.441143  9861 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:24.441720 10191 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.442333 10058 raft_consensus.cc:359] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.443745 10058 raft_consensus.cc:385] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.444062 10058 raft_consensus.cc:740] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.444598 10186 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.445441 10058 consensus_queue.cc:260] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.451567 10058 ts_tablet_manager.cc:1434] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:05:24.455006 10058 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:24.460726 10284 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.463476 10057 tablet_bootstrap.cc:654] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.464939 10284 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:24.453065 10191 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:24.472409 10058 tablet_bootstrap.cc:654] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:24.480595 10059 raft_consensus.cc:359] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.481577 10059 raft_consensus.cc:385] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.482061 10059 raft_consensus.cc:740] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.475890 10260 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.540486  9754 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:24.552868  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:24.561749  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:24.646823  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 107397us
I20260709 13:05:24.647759  9859 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.648200  9754 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
W20260709 13:05:24.575755  9903 leader_election.cc:343] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:24.650077  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 74787.7us
I20260709 13:05:24.578418  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
W20260709 13:05:24.578409  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 159015us
I20260709 13:05:24.585078  9785 raft_consensus.cc:1275] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:24.652741  9789 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.597805 10112 raft_consensus.cc:493] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.654881 10112 raft_consensus.cc:515] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.657191 10195 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:24.657884 10195 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.658309 10195 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.658576 10112 leader_election.cc:290] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:24.604436 10186 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.605010 10229 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.629040 10061 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:24.631088 10207 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.631474 10309 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.643673 10059 consensus_queue.cc:260] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.669227 10061 ts_tablet_manager.cc:1403] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.372s	user 0.020s	sys 0.014s
I20260709 13:05:24.669082 10309 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:24.676689 10201 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:24.660164 10229 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.680527 10207 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:24.683334 10229 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:24.659785 10186 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:24.690358  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 250118us
I20260709 13:05:24.700280  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.701570  9934 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 0.
W20260709 13:05:24.657299  9901 leader_election.cc:343] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:24.702502 10309 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:24.703094  9901 leader_election.cc:304] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:24.705209  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 48171.9us
I20260709 13:05:24.705600 10112 raft_consensus.cc:2749] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:24.692412  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:24.708798 10140 consensus_queue.cc:1048] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:24.712841  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:24.685187 10207 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.695020 10186 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.715245  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.716802  9934 raft_consensus.cc:2410] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:05:24.718499  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:24.715250 10061 raft_consensus.cc:359] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.718667  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.719372 10061 raft_consensus.cc:385] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.719610  9863 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.719901 10061 raft_consensus.cc:740] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.721170  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:24.722417  9936 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.722543  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.727429  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:24.728576  9933 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:24.721027 10061 consensus_queue.cc:260] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.745098  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:24.746193  9861 raft_consensus.cc:3060] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:24.747525  9901 leader_election.cc:343] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:24.750128  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:24.753330  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:24.758117  9978 leader_election.cc:343] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:24.759706  9978 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:24.768857  9978 leader_election.cc:343] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:24.761632  9859 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:05:24.801157 10260 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:24.801759 10058 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:24.804220 10059 ts_tablet_manager.cc:1434] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.415s	user 0.009s	sys 0.007s
I20260709 13:05:24.808917 10058 ts_tablet_manager.cc:1403] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.354s	user 0.027s	sys 0.001s
I20260709 13:05:24.781301 10201 raft_consensus.cc:697] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:24.818044  9789 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:05:24.818873 10057 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:24.832288 10057 ts_tablet_manager.cc:1403] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.451s	user 0.025s	sys 0.003s
I20260709 13:05:24.832803  9936 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:24.823150 10058 raft_consensus.cc:359] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:24.824987 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:24.837919 10009 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:24.842480 10061 ts_tablet_manager.cc:1434] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.173s	user 0.012s	sys 0.013s
I20260709 13:05:24.830245 10059 tablet_bootstrap.cc:492] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:24.776847  9829 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:24.842706 10058 raft_consensus.cc:385] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.843521 10058 raft_consensus.cc:740] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.827647  9977 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:24.821233  9684 catalog_manager.cc:5697] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 0 to 1, leader changed from <none> to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:24.841024  9978 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:24.850723 10057 raft_consensus.cc:359] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:24.854837  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:24.853029 10058 consensus_queue.cc:260] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:24.857282 10057 raft_consensus.cc:385] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:24.863075 10057 raft_consensus.cc:740] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.855048 10308 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:24.855049 10201 consensus_queue.cc:237] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.876681  9861 raft_consensus.cc:2468] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:05:24.881902  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:24.882483 10186 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:24.883342  9789 raft_consensus.cc:3060] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.883275 10186 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.893142 10186 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.890959  9863 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:24.898427  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:24.866775  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 115889us
W20260709 13:05:24.903724  9904 leader_election.cc:343] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:24.904788  9904 leader_election.cc:343] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:24.905508  9904 leader_election.cc:304] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:24.907282 10112 raft_consensus.cc:2749] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:24.909309  9904 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:24.910684 10112 raft_consensus.cc:2804] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:24.911315 10112 raft_consensus.cc:697] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 3 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:24.913002 10112 consensus_queue.cc:237] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:24.915057 10308 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.915159 10058 ts_tablet_manager.cc:1434] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.104s	user 0.006s	sys 0.003s
I20260709 13:05:24.915658 10308 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:24.843865  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 108129us
I20260709 13:05:24.923467 10317 consensus_queue.cc:1048] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:24.929417 10061 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:24.930125  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:24.937556  9975 leader_election.cc:343] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:24.940231 10207 consensus_queue.cc:1048] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:24.945443  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 79462.6us
I20260709 13:05:24.946579  9827 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:24.948813 10282 raft_consensus.cc:2749] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:24.956147  9903 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:24.938964  9975 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:24.953094 10058 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:24.966985 10229 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:24.967746 10229 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.968267 10229 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:24.987560 10057 consensus_queue.cc:260] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:25.003875 10312 raft_consensus.cc:2749] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:24.866441 10195 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.015123 10057 ts_tablet_manager.cc:1434] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.182s	user 0.012s	sys 0.000s
I20260709 13:05:25.025871 10313 raft_consensus.cc:2804] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:25.026736 10313 raft_consensus.cc:697] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
W20260709 13:05:25.010782  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 171093us
I20260709 13:05:25.046433 10195 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:25.047399  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 222825us
I20260709 13:05:25.048290  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:25.049093  9934 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:25.051842  9789 raft_consensus.cc:2468] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:05:25.008659  9977 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:25.058508  9784 raft_consensus.cc:2393] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
W20260709 13:05:25.087006  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 94957us
I20260709 13:05:25.091275 10057 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:25.092490 10186 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.100561 10312 raft_consensus.cc:2804] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.101557 10312 raft_consensus.cc:697] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:25.103082 10312 consensus_queue.cc:237] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.105338 10186 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:25.028158 10313 consensus_queue.cc:237] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.112991 10186 consensus_queue.cc:1048] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:25.085011 10282 raft_consensus.cc:493] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.119583  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:25.121367 10282 raft_consensus.cc:515] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.122231  9934 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:25.122553  9785 raft_consensus.cc:2393] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:25.120725 10058 tablet_bootstrap.cc:654] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.124548 10282 leader_election.cc:290] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:25.152740  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 62456.8us
I20260709 13:05:25.156128 10059 tablet_bootstrap.cc:654] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.174399  9861 raft_consensus.cc:3060] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:25.158186 10061 tablet_bootstrap.cc:654] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.161813  9683 catalog_manager.cc:5697] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.168713  9754 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:25.183187 10308 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.212412 10195 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:25.213557 10195 raft_consensus.cc:697] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
W20260709 13:05:25.205963  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 116525us
W20260709 13:05:25.214709  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 46147.6us
I20260709 13:05:25.244462 10195 consensus_queue.cc:237] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.235870 10316 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.265924 10316 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.270761 10058 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:25.271423 10058 ts_tablet_manager.cc:1403] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.319s	user 0.012s	sys 0.020s
I20260709 13:05:25.289065  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:25.293237 10308 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:25.284804 10112 raft_consensus.cc:493] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.297511 10112 raft_consensus.cc:515] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:25.295663  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:25.334918  9863 raft_consensus.cc:2393] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:25.336683 10058 raft_consensus.cc:359] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.339498 10058 raft_consensus.cc:385] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.340077 10058 raft_consensus.cc:740] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.342628 10329 consensus_queue.cc:1048] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:25.309695  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:25.350731 10321 consensus_queue.cc:1048] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:25.354339 10316 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:25.355091  9978 leader_election.cc:343] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.358588 10229 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.341645 10058 consensus_queue.cc:260] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.364598 10112 leader_election.cc:290] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:25.366374 10058 ts_tablet_manager.cc:1434] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.095s	user 0.011s	sys 0.000s
I20260709 13:05:25.366107  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:25.369376  9936 raft_consensus.cc:1275] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.383894 10057 tablet_bootstrap.cc:654] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.384891 10140 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.385736 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:25.390583 10260 raft_consensus.cc:493] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.390483  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:25.399338 10059 tablet_bootstrap.cc:492] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:25.405094 10303 raft_consensus.cc:493] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.415180  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:25.419648  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:25.466436 10059 ts_tablet_manager.cc:1403] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.637s	user 0.018s	sys 0.025s
I20260709 13:05:25.501129  9934 raft_consensus.cc:2393] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:25.455458 10229 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:25.431897  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:25.528661  9933 raft_consensus.cc:2393] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:25.548770 10061 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:25.549465 10061 ts_tablet_manager.cc:1403] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.621s	user 0.029s	sys 0.009s
I20260709 13:05:25.473902 10260 raft_consensus.cc:515] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:25.544241 10059 raft_consensus.cc:359] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.554190 10059 raft_consensus.cc:385] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.476982 10303 raft_consensus.cc:515] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.555624 10059 raft_consensus.cc:740] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.477283 10140 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.558329 10058 tablet_bootstrap.cc:492] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:25.565655 10303 leader_election.cc:290] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:25.568444 10260 leader_election.cc:290] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:25.550817 10316 consensus_queue.cc:1048] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:05:25.573902  9784 raft_consensus.cc:1275] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:25.575870  9901 leader_election.cc:343] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.578498 10059 consensus_queue.cc:260] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.558228 10061 raft_consensus.cc:359] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.580387 10061 raft_consensus.cc:385] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.584033 10061 raft_consensus.cc:740] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.584723 10059 ts_tablet_manager.cc:1434] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.118s	user 0.005s	sys 0.006s
I20260709 13:05:25.585544 10061 consensus_queue.cc:260] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.593552  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:25.594653  9936 raft_consensus.cc:2468] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:25.595315  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:25.586346 10059 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:25.609627 10061 ts_tablet_manager.cc:1434] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.060s	user 0.011s	sys 0.000s
I20260709 13:05:25.607590  9938 raft_consensus.cc:2468] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:25.619922 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:25.629665  9687 catalog_manager.cc:5697] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from d02e8a054a634be599149db641901736 (127.0.107.195) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.634757 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:25.640728 10010 raft_consensus.cc:2393] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:25.637714  9754 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:25.644698 10195 raft_consensus.cc:2804] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:25.645493 10195 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.646183  9863 raft_consensus.cc:1275] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:25.637184  9685 catalog_manager.cc:5697] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 3, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.646059 10195 raft_consensus.cc:3060] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:25.647799 10059 tablet_bootstrap.cc:654] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.615069  9859 raft_consensus.cc:1275] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 13:05:25.674264 10303 consensus_queue.cc:1048] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:25.650370 10260 consensus_queue.cc:1048] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:05:25.654094  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:25.649904 10140 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:25.693135 10061 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:25.707538  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:25.711364  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:25.713848 10309 consensus_queue.cc:1048] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:25.650280  9977 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:25.717661  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:25.738361 10058 tablet_bootstrap.cc:654] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.731259 10207 raft_consensus.cc:2749] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:25.728973  9975 leader_election.cc:343] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.750249  9864 raft_consensus.cc:1275] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:25.748984 10195 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.758741 10057 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:25.759593 10057 ts_tablet_manager.cc:1403] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.669s	user 0.008s	sys 0.036s
I20260709 13:05:25.764506  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:25.737310  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 100945us
I20260709 13:05:25.774137  9788 raft_consensus.cc:2393] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
W20260709 13:05:25.777199  9901 leader_election.cc:343] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.779460 10057 raft_consensus.cc:359] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.782694 10057 raft_consensus.cc:385] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.783121 10057 raft_consensus.cc:740] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.789387 10195 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:25.791360  9977 leader_election.cc:304] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:25.794050 10207 raft_consensus.cc:2804] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:25.794689 10207 raft_consensus.cc:493] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.795146 10207 raft_consensus.cc:3060] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:25.792609 10057 consensus_queue.cc:260] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.799019  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:25.799913  9938 raft_consensus.cc:3060] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:25.803535  9903 leader_election.cc:343] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:25.804106  9755 leader_election.cc:343] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.804550  9903 leader_election.cc:304] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:25.806041 10229 raft_consensus.cc:2749] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:25.849207  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 45417.6us
I20260709 13:05:25.849520 10329 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.850291 10329 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.857764 10329 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:25.869122  9904 leader_election.cc:343] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.874708  9938 raft_consensus.cc:2468] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:25.876430  9754 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:25.876528  9788 raft_consensus.cc:1275] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 13:05:25.879048 10201 raft_consensus.cc:2804] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:25.878525 10140 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.880292 10201 raft_consensus.cc:697] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:25.880604  9687 catalog_manager.cc:5697] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 0 to 2, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } }
I20260709 13:05:25.881685 10201 consensus_queue.cc:237] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:25.869371 10057 ts_tablet_manager.cc:1434] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.109s	user 0.014s	sys 0.001s
I20260709 13:05:25.894953 10057 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:25.883317 10313 consensus_queue.cc:1048] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:05:25.880061 10140 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.903180 10140 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:25.903743  9904 leader_election.cc:343] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.904448  9904 leader_election.cc:304] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:25.914146  9829 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:25.918215 10229 consensus_queue.cc:1048] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:25.919426 10140 raft_consensus.cc:2749] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:25.918350  9904 leader_election.cc:343] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:25.921205  9904 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:25.922807 10140 raft_consensus.cc:2749] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:25.924088 10316 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.924777 10316 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:25.927294 10316 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:25.859383  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 55901.5us
I20260709 13:05:25.928604 10329 raft_consensus.cc:2749] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:25.933275 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:25.934273 10009 raft_consensus.cc:2468] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:25.935987 10058 tablet_bootstrap.cc:492] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:25.936857 10058 ts_tablet_manager.cc:1403] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.407s	user 0.017s	sys 0.023s
I20260709 13:05:25.936991  9754 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:25.939071 10195 raft_consensus.cc:2804] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:25.939725 10195 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.945976 10195 raft_consensus.cc:3060] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:25.960268 10061 tablet_bootstrap.cc:654] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.968622  9936 raft_consensus.cc:1275] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:25.973881 10057 tablet_bootstrap.cc:654] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:25.973662 10058 raft_consensus.cc:359] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.974692 10058 raft_consensus.cc:385] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:25.977654 10058 raft_consensus.cc:740] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.981537  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:25.982816  9829 leader_election.cc:304] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:25.983466  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:25.992453 10058 consensus_queue.cc:260] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:25.995843  9859 raft_consensus.cc:1275] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.997593 10058 ts_tablet_manager.cc:1434] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.001s
I20260709 13:05:25.999249 10058 tablet_bootstrap.cc:492] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:26.007825 10195 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.009207  9903 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:26.011044 10303 raft_consensus.cc:2749] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.011405 10195 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:26.012490 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:26.013392 10009 raft_consensus.cc:3060] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.017154 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:26.018535 10010 raft_consensus.cc:2468] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:26.012930 10260 consensus_queue.cc:1048] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.022183 10357 raft_consensus.cc:2749] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.023321 10058 tablet_bootstrap.cc:654] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.037309 10207 raft_consensus.cc:515] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.039105 10140 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.048651  9903 leader_election.cc:304] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:26.050382 10112 raft_consensus.cc:2804] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:26.050815 10112 raft_consensus.cc:493] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.051157 10112 raft_consensus.cc:3060] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.048200 10140 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.064455 10140 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:26.088055 10009 raft_consensus.cc:2468] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:26.089561 10201 consensus_queue.cc:1048] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:26.088086  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:26.096527  9754 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:26.116804 10201 raft_consensus.cc:2804] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.116863 10260 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.104606  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 121915us
I20260709 13:05:26.132058 10059 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:26.132725 10059 ts_tablet_manager.cc:1403] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.547s	user 0.025s	sys 0.021s
I20260709 13:05:26.129468 10201 raft_consensus.cc:697] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:26.138940 10201 consensus_queue.cc:237] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.131016 10260 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.148116  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:26.153055 10207 leader_election.cc:290] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:26.171792 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:26.173384 10010 raft_consensus.cc:2393] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:26.176121  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:26.177970  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:26.181703  9975 leader_election.cc:343] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:26.182713  9975 leader_election.cc:343] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.184528  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:26.187402  9978 leader_election.cc:343] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.188104  9978 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:26.190091 10313 raft_consensus.cc:2749] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.193138 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:26.191859 10260 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:26.203581 10059 raft_consensus.cc:359] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:26.214316 10061 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:26.215935 10061 ts_tablet_manager.cc:1403] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.542s	user 0.025s	sys 0.018s
I20260709 13:05:26.206720 10059 raft_consensus.cc:385] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.228736 10059 raft_consensus.cc:740] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.220799 10057 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:26.229868 10057 ts_tablet_manager.cc:1403] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.335s	user 0.027s	sys 0.019s
I20260709 13:05:26.230669 10059 consensus_queue.cc:260] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:26.209385  9755 leader_election.cc:343] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.234278 10059 ts_tablet_manager.cc:1434] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.101s	user 0.009s	sys 0.000s
I20260709 13:05:26.235972 10059 tablet_bootstrap.cc:492] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
W20260709 13:05:26.222620  9903 leader_election.cc:343] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.254887 10059 tablet_bootstrap.cc:654] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.260175 10112 raft_consensus.cc:515] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.261150  9687 catalog_manager.cc:5697] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.262212  9934 raft_consensus.cc:2368] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d for earlier term 1. Current term is 2.
W20260709 13:05:26.262212  9755 leader_election.cc:343] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.203915  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:26.270233 10195 consensus_queue.cc:1048] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.270911  9938 raft_consensus.cc:3060] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.278532  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:26.279371  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:26.279547  9864 raft_consensus.cc:2468] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:26.270218  9687 catalog_manager.cc:5697] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.285365  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:26.276093 10061 raft_consensus.cc:359] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.286736 10061 raft_consensus.cc:385] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.287173 10061 raft_consensus.cc:740] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.288533 10058 tablet_bootstrap.cc:492] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:26.288481 10061 consensus_queue.cc:260] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.289140 10058 ts_tablet_manager.cc:1403] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.290s	user 0.014s	sys 0.017s
I20260709 13:05:26.295269 10319 consensus_queue.cc:1048] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.298585 10057 raft_consensus.cc:359] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.299602 10057 raft_consensus.cc:385] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.299944 10057 raft_consensus.cc:740] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.292806 10058 raft_consensus.cc:359] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.311848 10058 raft_consensus.cc:385] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.312331 10058 raft_consensus.cc:740] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.313416 10058 consensus_queue.cc:260] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.319032 10061 ts_tablet_manager.cc:1434] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.000s
I20260709 13:05:26.329177 10061 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:26.326444 10058 ts_tablet_manager.cc:1434] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:05:26.332849 10112 leader_election.cc:290] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:26.333756 10058 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:26.334333 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:26.335376 10010 raft_consensus.cc:3060] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.310925 10057 consensus_queue.cc:260] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.336687 10260 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.337339 10260 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.338653 10057 ts_tablet_manager.cc:1434] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.000s
I20260709 13:05:26.343822 10260 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:26.344267  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:26.356175  9975 leader_election.cc:343] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.346630  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:26.348591 10057 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:26.358189 10201 raft_consensus.cc:493] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.358847  9978 leader_election.cc:343] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.359714  9978 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:26.361362 10313 raft_consensus.cc:2749] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:26.353045  9904 leader_election.cc:343] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.364954  9904 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:26.364140  9686 catalog_manager.cc:5697] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.358939 10201 raft_consensus.cc:515] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.367949 10140 raft_consensus.cc:2749] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.364766 10059 tablet_bootstrap.cc:492] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:26.372227 10059 ts_tablet_manager.cc:1403] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.137s	user 0.017s	sys 0.001s
W20260709 13:05:26.373169  9755 leader_election.cc:343] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.374689  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:26.375375 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:26.376423 10009 raft_consensus.cc:2393] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:26.377797  9829 leader_election.cc:400] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Vote denied by peer d02e8a054a634be599149db641901736 (127.0.107.195:37607) with higher term. Message: Invalid argument: T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d for earlier term 1. Current term is 2.
I20260709 13:05:26.378132  9754 leader_election.cc:304] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:26.379220 10319 raft_consensus.cc:3060] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.385882 10010 raft_consensus.cc:2468] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:26.388088  9903 leader_election.cc:304] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:26.391522 10319 raft_consensus.cc:2749] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.379235 10201 leader_election.cc:290] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:26.396519 10201 consensus_queue.cc:1048] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.400952 10140 raft_consensus.cc:2804] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.401170 10059 raft_consensus.cc:359] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.401546 10140 raft_consensus.cc:697] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:26.402529 10140 consensus_queue.cc:237] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.410377 10058 tablet_bootstrap.cc:654] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.417934 10059 raft_consensus.cc:385] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.418463 10059 raft_consensus.cc:740] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.419364 10059 consensus_queue.cc:260] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.421147 10061 tablet_bootstrap.cc:654] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.378214  9829 leader_election.cc:403] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:26.423856  9938 raft_consensus.cc:2468] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:26.440572  9977 leader_election.cc:304] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:26.442193 10260 raft_consensus.cc:2804] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:26.443300 10260 raft_consensus.cc:697] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:26.445132 10260 consensus_queue.cc:237] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.447288 10329 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.448235 10329 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.450685 10329 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:26.447187 10361 raft_consensus.cc:3060] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.448843 10057 tablet_bootstrap.cc:654] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:26.469709  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 92148.7us
I20260709 13:05:26.478710 10059 ts_tablet_manager.cc:1434] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.106s	user 0.007s	sys 0.000s
I20260709 13:05:26.480319 10059 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:26.488114 10361 raft_consensus.cc:2749] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer d02e8a054a634be599149db641901736 (127.0.107.195:37607) with higher term. Message: Invalid argument: T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d for earlier term 1. Current term is 2.
I20260709 13:05:26.492628  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "178dcd0da5414a9785794b2412a9cf93" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:26.498116  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:26.500285 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:26.500394  9938 raft_consensus.cc:2393] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:26.508116 10010 raft_consensus.cc:2393] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:26.511513  9683 catalog_manager.cc:5697] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 2, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.531176 10059 tablet_bootstrap.cc:654] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.536815  9860 raft_consensus.cc:2468] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
W20260709 13:05:26.553141 10375 log.cc:927] Time spent T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:05:26.554930  9859 raft_consensus.cc:3060] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.557633  9685 catalog_manager.cc:5697] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.594599 10057 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:26.595292 10057 ts_tablet_manager.cc:1403] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.247s	user 0.013s	sys 0.011s
I20260709 13:05:26.620054 10361 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.616816 10057 raft_consensus.cc:359] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.621271 10057 raft_consensus.cc:385] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.621662 10057 raft_consensus.cc:740] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.621085 10361 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:26.630743 10361 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:26.634394  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:26.637811 10303 raft_consensus.cc:493] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.638974 10303 raft_consensus.cc:515] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:26.641608 10377 log.cc:927] Time spent T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:05:26.644253 10057 consensus_queue.cc:260] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.615929  9829 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:26.648231 10361 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.650391 10303 leader_election.cc:290] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:26.651928 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125c95bf84c45538728679b308f6aca" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:26.659269 10061 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
W20260709 13:05:26.661334  9903 leader_election.cc:343] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:26.667790  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 55132.9us
I20260709 13:05:26.668220 10057 ts_tablet_manager.cc:1434] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.073s	user 0.011s	sys 0.000s
I20260709 13:05:26.670390 10058 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:26.670845 10061 ts_tablet_manager.cc:1403] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.342s	user 0.029s	sys 0.034s
I20260709 13:05:26.671268 10058 ts_tablet_manager.cc:1403] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.338s	user 0.024s	sys 0.005s
I20260709 13:05:26.675067 10058 raft_consensus.cc:359] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.680943 10058 raft_consensus.cc:385] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.681564  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125c95bf84c45538728679b308f6aca" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:26.684029 10058 raft_consensus.cc:740] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.685986 10058 consensus_queue.cc:260] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.686215 10316 consensus_queue.cc:1048] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:26.686661 10057 tablet_bootstrap.cc:492] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:26.687788 10361 raft_consensus.cc:2749] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.691226 10059 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:26.691895 10059 ts_tablet_manager.cc:1403] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.212s	user 0.019s	sys 0.025s
W20260709 13:05:26.696291  9904 leader_election.cc:343] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.697188  9904 leader_election.cc:304] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:26.695529 10059 raft_consensus.cc:359] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.700217 10059 raft_consensus.cc:385] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.699302 10347 raft_consensus.cc:2749] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.700639 10059 raft_consensus.cc:740] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.692471 10061 raft_consensus.cc:359] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.706110 10061 raft_consensus.cc:385] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.707232  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:26.706434 10061 raft_consensus.cc:740] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.717772 10061 consensus_queue.cc:260] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:26.722805  9755 consensus_peers.cc:597] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.724041 10059 consensus_queue.cc:260] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:26.726747 10059 ts_tablet_manager.cc:1434] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
I20260709 13:05:26.732059 10057 tablet_bootstrap.cc:654] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.733659 10061 ts_tablet_manager.cc:1434] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.062s	user 0.003s	sys 0.005s
I20260709 13:05:26.734582  9829 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:26.735608 10058 ts_tablet_manager.cc:1434] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.000s
I20260709 13:05:26.745242 10058 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:26.744081 10359 raft_consensus.cc:2749] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.750927 10061 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:26.737128 10059 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:26.802806 10260 raft_consensus.cc:493] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.803957 10260 raft_consensus.cc:515] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.804365 10381 consensus_queue.cc:1048] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.808045  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:26.809265  9938 raft_consensus.cc:2468] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:26.813542  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:26.810730 10260 leader_election.cc:290] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:26.817006  9977 leader_election.cc:304] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
W20260709 13:05:26.818437  9975 leader_election.cc:343] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.819604 10313 raft_consensus.cc:2804] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.820120 10313 raft_consensus.cc:493] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.820545 10313 raft_consensus.cc:3060] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.822309 10347 raft_consensus.cc:493] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.823177 10347 raft_consensus.cc:515] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.826826 10057 tablet_bootstrap.cc:492] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:26.827270 10347 leader_election.cc:290] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:26.827548 10057 ts_tablet_manager.cc:1403] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.006s
I20260709 13:05:26.828166  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:26.829268 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:26.829908  9901 leader_election.cc:343] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.830263 10058 tablet_bootstrap.cc:654] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:26.831348  9903 leader_election.cc:343] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.832054  9903 leader_election.cc:304] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:26.833173 10347 raft_consensus.cc:2749] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:26.834028 10061 tablet_bootstrap.cc:654] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.845677 10057 raft_consensus.cc:359] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.847100 10057 raft_consensus.cc:385] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.847487 10057 raft_consensus.cc:740] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.848018 10058 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:26.848770 10058 ts_tablet_manager.cc:1403] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.001s
I20260709 13:05:26.848584 10057 consensus_queue.cc:260] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.856745 10058 raft_consensus.cc:359] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:26.861649 10058 raft_consensus.cc:385] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:26.861994 10058 raft_consensus.cc:740] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.865708 10058 consensus_queue.cc:260] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:26.886469 10057 ts_tablet_manager.cc:1434] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.058s	user 0.009s	sys 0.004s
I20260709 13:05:26.888125 10057 tablet_bootstrap.cc:492] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:26.889096 10059 tablet_bootstrap.cc:654] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.892549 10058 ts_tablet_manager.cc:1434] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.002s
I20260709 13:05:26.887822 10313 raft_consensus.cc:515] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:26.907442 10057 tablet_bootstrap.cc:654] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:26.907847  9860 raft_consensus.cc:1275] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:26.915815 10058 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:26.917272 10303 consensus_queue.cc:1048] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.919754  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
W20260709 13:05:26.922080  9975 leader_election.cc:343] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.928877 10260 consensus_queue.cc:1048] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.929607  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:26.930346  9938 raft_consensus.cc:3060] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.937556 10313 leader_election.cc:290] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:26.953951 10308 consensus_queue.cc:1048] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.956789 10308 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.957707 10308 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:26.962438  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:26.965453 10009 raft_consensus.cc:1275] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 13:05:26.965142  9978 leader_election.cc:343] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:26.967823 10308 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:26.972852 10140 consensus_queue.cc:1048] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:26.976159  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:26.977214  9937 raft_consensus.cc:2468] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:26.979981  9977 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:26.981454 10207 raft_consensus.cc:2804] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:26.981876 10207 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.982401 10207 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:26.995802 10061 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:26.996872 10061 ts_tablet_manager.cc:1403] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.246s	user 0.021s	sys 0.000s
I20260709 13:05:26.998266 10058 tablet_bootstrap.cc:654] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.001253 10061 raft_consensus.cc:359] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.007499 10061 raft_consensus.cc:385] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.008140 10061 raft_consensus.cc:740] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.010092 10061 consensus_queue.cc:260] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.013259 10061 ts_tablet_manager.cc:1434] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:05:27.035904 10316 raft_consensus.cc:493] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.040568 10058 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:27.134220 10058 ts_tablet_manager.cc:1403] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.219s	user 0.024s	sys 0.001s
I20260709 13:05:27.133921 10316 raft_consensus.cc:515] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.050943 10112 raft_consensus.cc:493] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.137526 10112 raft_consensus.cc:515] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.055444 10303 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.064507 10361 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.073642 10057 tablet_bootstrap.cc:492] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:27.087867 10195 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.140789 10361 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.140280 10112 leader_election.cc:290] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:27.145756 10061 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:27.112715 10207 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.126109 10059 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:27.147171  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.147665 10059 ts_tablet_manager.cc:1403] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.411s	user 0.041s	sys 0.024s
I20260709 13:05:27.128783  9938 raft_consensus.cc:2468] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:27.136754 10316 leader_election.cc:290] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.142292 10057 ts_tablet_manager.cc:1403] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.254s	user 0.053s	sys 0.039s
I20260709 13:05:27.149489  9860 raft_consensus.cc:3060] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.142601 10195 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.154384 10207 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.154935 10195 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:27.155560  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:27.151780 10059 raft_consensus.cc:359] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.164374 10059 raft_consensus.cc:385] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.164774 10059 raft_consensus.cc:740] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.146875 10361 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:27.162102  9978 leader_election.cc:343] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.152040 10057 raft_consensus.cc:359] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.169816 10057 raft_consensus.cc:385] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.140730 10303 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.172192 10303 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:27.143514 10058 raft_consensus.cc:359] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.173547 10058 raft_consensus.cc:385] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.173457 10059 consensus_queue.cc:260] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.174234 10058 raft_consensus.cc:740] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.175627 10058 consensus_queue.cc:260] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.177695  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.178668 10061 tablet_bootstrap.cc:654] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.174124 10057 raft_consensus.cc:740] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.181078 10057 consensus_queue.cc:260] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.181722 10058 ts_tablet_manager.cc:1434] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.047s	user 0.003s	sys 0.005s
I20260709 13:05:27.190907 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:27.191519 10058 tablet_bootstrap.cc:492] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:27.174100 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:27.184325 10057 ts_tablet_manager.cc:1434] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:05:27.193797 10009 raft_consensus.cc:2468] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:27.194898 10057 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:27.195945  9754 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:27.185555  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:27.197314  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:27.187520 10059 ts_tablet_manager.cc:1434] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 13:05:27.198262  9937 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.187420  9977 leader_election.cc:304] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:27.197510 10195 raft_consensus.cc:2804] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:27.201351 10059 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:27.201471 10195 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.202579 10195 raft_consensus.cc:3060] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.204753 10397 raft_consensus.cc:2804] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:27.186753  9904 leader_election.cc:343] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:27.206419  9904 leader_election.cc:343] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.207317 10397 raft_consensus.cc:697] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:27.208595 10397 consensus_queue.cc:237] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.211714 10059 tablet_bootstrap.cc:654] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.214828  9827 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:27.226310  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.227959  9864 raft_consensus.cc:2468] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:27.230324  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:27.226313  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:27.235252  9934 raft_consensus.cc:2468] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:27.236985  9754 leader_election.cc:304] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:27.240700  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.241155  9937 raft_consensus.cc:2468] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:27.243088  9977 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:27.242761  9788 raft_consensus.cc:2468] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:27.249784 10376 raft_consensus.cc:2804] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:27.245512 10207 raft_consensus.cc:2804] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:27.254874  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:27.258265 10207 raft_consensus.cc:697] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:27.258410  9901 leader_election.cc:304] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:27.259991 10059 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:27.259509 10207 consensus_queue.cc:237] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.258752 10376 raft_consensus.cc:493] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.260962 10376 raft_consensus.cc:3060] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.261147 10059 ts_tablet_manager.cc:1403] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.000s
I20260709 13:05:27.267385 10057 tablet_bootstrap.cc:654] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.260141 10112 raft_consensus.cc:2804] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:27.272797 10112 raft_consensus.cc:493] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.278425 10112 raft_consensus.cc:3060] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.317576 10058 tablet_bootstrap.cc:654] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:27.321260  9755 leader_election.cc:343] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:27.326627  9901 leader_election.cc:343] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.327495  9901 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:27.332587 10061 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:27.333276 10061 ts_tablet_manager.cc:1403] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.188s	user 0.016s	sys 0.014s
I20260709 13:05:27.329249 10303 raft_consensus.cc:2749] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.335346 10112 raft_consensus.cc:515] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.338950 10112 leader_election.cc:290] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:27.345763 10303 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.357421 10303 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.357695 10059 raft_consensus.cc:359] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.359038 10059 raft_consensus.cc:385] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.359500 10059 raft_consensus.cc:740] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.360487 10303 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:27.350517 10061 raft_consensus.cc:359] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.361342 10061 raft_consensus.cc:385] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.361716 10061 raft_consensus.cc:740] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.360666 10059 consensus_queue.cc:260] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.362798 10061 consensus_queue.cc:260] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.413964 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:27.423839 10308 raft_consensus.cc:493] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.425392 10308 raft_consensus.cc:515] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.425889  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:27.426744  9864 raft_consensus.cc:3060] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.369313 10376 raft_consensus.cc:515] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.430996 10376 leader_election.cc:290] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.433627 10061 ts_tablet_manager.cc:1434] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.004s
I20260709 13:05:27.376983 10195 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:27.444965  9903 leader_election.cc:343] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.450851  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125c95bf84c45538728679b308f6aca" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.452188  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:27.454083  9978 leader_election.cc:343] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.455503  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:27.452047  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:27.461222  9788 raft_consensus.cc:2393] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:27.465682 10061 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:27.475454  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:27.479565  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:27.524987  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125c95bf84c45538728679b308f6aca" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:27.525425  9934 raft_consensus.cc:2393] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:27.526005  9938 raft_consensus.cc:2468] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:27.527673  9754 leader_election.cc:304] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:27.532176  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:27.483640 10361 raft_consensus.cc:2749] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.490031 10061 tablet_bootstrap.cc:654] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.488392  9685 catalog_manager.cc:5697] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.493506 10057 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:27.497751 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:27.500161 10195 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:27.509713 10308 leader_election.cc:290] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.524296 10401 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.532740 10059 ts_tablet_manager.cc:1434] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.270s	user 0.056s	sys 0.095s
I20260709 13:05:27.536921 10057 ts_tablet_manager.cc:1403] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.342s	user 0.020s	sys 0.005s
I20260709 13:05:27.537720  9685 catalog_manager.cc:5697] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.538870 10009 raft_consensus.cc:3060] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.533380  9977 leader_election.cc:304] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:27.481884 10058 tablet_bootstrap.cc:492] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:27.542851 10058 ts_tablet_manager.cc:1403] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.352s	user 0.020s	sys 0.030s
I20260709 13:05:27.535192 10376 raft_consensus.cc:2749] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.541049 10401 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.547523  9862 raft_consensus.cc:3060] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.549811 10401 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:27.551638  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:27.552361  9901 leader_election.cc:343] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.552814  9785 raft_consensus.cc:2393] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:27.552945  9901 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:27.555729 10303 raft_consensus.cc:2749] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.544170 10308 raft_consensus.cc:2804] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:27.564718 10308 raft_consensus.cc:493] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.565217 10308 raft_consensus.cc:3060] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.567178 10058 raft_consensus.cc:359] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.568044 10058 raft_consensus.cc:385] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.569869 10057 raft_consensus.cc:359] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.570735 10057 raft_consensus.cc:385] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.571027 10057 raft_consensus.cc:740] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.571936 10057 consensus_queue.cc:260] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.573565 10009 raft_consensus.cc:2468] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:27.575357  9754 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:27.576491 10195 raft_consensus.cc:2804] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:27.577476 10195 raft_consensus.cc:697] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:27.577739  9864 raft_consensus.cc:2468] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:27.578799 10195 consensus_queue.cc:237] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.579866 10059 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:27.584877  9904 leader_election.cc:304] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:27.586062 10112 raft_consensus.cc:2804] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:27.586721 10112 raft_consensus.cc:697] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:27.587836 10112 consensus_queue.cc:237] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.568364 10058 raft_consensus.cc:740] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.594161 10057 ts_tablet_manager.cc:1434] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
I20260709 13:05:27.596792 10058 consensus_queue.cc:260] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.605365 10058 ts_tablet_manager.cc:1434] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.002s
I20260709 13:05:27.620818  9686 catalog_manager.cc:5697] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.623180  9862 raft_consensus.cc:2468] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:27.616567 10061 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:27.624730 10057 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:27.625121 10061 ts_tablet_manager.cc:1403] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.160s	user 0.009s	sys 0.011s
I20260709 13:05:27.610438 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:27.645151 10061 raft_consensus.cc:359] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.645876 10061 raft_consensus.cc:385] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.646170 10061 raft_consensus.cc:740] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.646960 10061 consensus_queue.cc:260] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.650746 10207 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.651638 10207 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.654991 10397 consensus_queue.cc:1048] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.656883 10207 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.659003 10058 tablet_bootstrap.cc:492] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:27.660593 10009 raft_consensus.cc:2393] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:27.661986  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:27.662688  9829 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:27.662796  9938 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:27.664551 10406 raft_consensus.cc:2749] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.668740 10058 tablet_bootstrap.cc:654] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.672837  9977 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:27.680382 10207 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:27.684938 10207 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.689143 10207 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.684497  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.693243  9862 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:27.714752 10059 tablet_bootstrap.cc:654] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.729065 10061 ts_tablet_manager.cc:1434] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.103s	user 0.005s	sys 0.004s
I20260709 13:05:27.744840 10061 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:27.750209 10207 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.758862 10207 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.756189 10058 tablet_bootstrap.cc:492] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:27.760246 10058 ts_tablet_manager.cc:1403] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.113s	user 0.018s	sys 0.008s
I20260709 13:05:27.759965  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:27.752811 10057 tablet_bootstrap.cc:654] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.760970  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:27.762027  9934 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.761005  9862 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.767689 10058 raft_consensus.cc:359] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.769824 10058 raft_consensus.cc:385] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.770594 10058 raft_consensus.cc:740] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.772934 10058 consensus_queue.cc:260] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.776605 10397 raft_consensus.cc:493] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.780995 10058 ts_tablet_manager.cc:1434] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:05:27.777370 10397 raft_consensus.cc:515] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:27.792696 10397 leader_election.cc:290] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:27.794164 10059 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:27.794888 10059 ts_tablet_manager.cc:1403] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.215s	user 0.012s	sys 0.003s
I20260709 13:05:27.795970  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:27.796612 10058 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:27.796938  9938 raft_consensus.cc:2468] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:27.782918 10061 tablet_bootstrap.cc:654] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.780802 10308 raft_consensus.cc:515] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.798389  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:27.800747  9977 leader_election.cc:304] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:27.803170  9934 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
W20260709 13:05:27.807031  9975 leader_election.cc:343] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.815021 10260 consensus_queue.cc:1048] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:27.808501 10397 raft_consensus.cc:2804] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:27.808390 10057 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:27.816901 10057 ts_tablet_manager.cc:1403] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.211s	user 0.013s	sys 0.012s
I20260709 13:05:27.810377  9862 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:27.812615 10059 raft_consensus.cc:359] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.819260 10059 raft_consensus.cc:385] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.819577 10059 raft_consensus.cc:740] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.816497 10397 raft_consensus.cc:493] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.820986 10397 raft_consensus.cc:3060] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.824739  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125c95bf84c45538728679b308f6aca" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:27.825822  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125c95bf84c45538728679b308f6aca" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:27.826606  9934 raft_consensus.cc:3060] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:27.823237 10057 raft_consensus.cc:359] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:27.827626  9978 leader_election.cc:343] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.827641 10057 raft_consensus.cc:385] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:27.828857 10057 raft_consensus.cc:740] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.830368 10057 consensus_queue.cc:260] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:27.832072  9977 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:27.831840 10059 consensus_queue.cc:260] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:27.837878 10207 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:27.846623 10207 raft_consensus.cc:697] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:27.850342 10057 ts_tablet_manager.cc:1434] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:05:27.863407 10057 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:27.869895 10112 raft_consensus.cc:493] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.870253 10408 raft_consensus.cc:493] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.860482 10207 consensus_queue.cc:237] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.815059 10308 leader_election.cc:290] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:27.869810 10059 ts_tablet_manager.cc:1434] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.075s	user 0.012s	sys 0.000s
I20260709 13:05:27.874051 10112 raft_consensus.cc:515] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.880210 10058 tablet_bootstrap.cc:654] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:27.883174 10112 leader_election.cc:290] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:27.883970 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:27.877127 10408 raft_consensus.cc:515] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:27.886464  9903 leader_election.cc:343] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.886879  9934 raft_consensus.cc:2468] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:27.888307 10408 leader_election.cc:290] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:27.889101  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.894948  9977 leader_election.cc:304] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:27.890211 10376 raft_consensus.cc:493] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.895983  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:27.889634 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:27.902863 10059 tablet_bootstrap.cc:492] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:27.904158 10376 raft_consensus.cc:515] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:27.906785  9904 leader_election.cc:343] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:27.908607  9904 leader_election.cc:343] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:27.909042  9904 leader_election.cc:304] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:27.910799 10140 raft_consensus.cc:2749] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:27.912875  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:05:27.913367  9978 consensus_peers.cc:597] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.908042 10376 leader_election.cc:290] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:27.915081 10260 raft_consensus.cc:2804] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:27.929658 10260 raft_consensus.cc:697] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:27.938339 10260 consensus_queue.cc:237] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:27.959443 10061 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:27.962007 10061 ts_tablet_manager.cc:1403] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.225s	user 0.019s	sys 0.004s
I20260709 13:05:27.999152 10061 raft_consensus.cc:359] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.000214 10061 raft_consensus.cc:385] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.000566 10061 raft_consensus.cc:740] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.003453 10408 consensus_queue.cc:1048] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:28.009160  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 101286us
I20260709 13:05:28.001456 10061 consensus_queue.cc:260] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.013314 10057 tablet_bootstrap.cc:654] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:28.015887  9903 leader_election.cc:343] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.029469 10061 ts_tablet_manager.cc:1434] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.001s
I20260709 13:05:28.030907 10061 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:28.030745 10397 raft_consensus.cc:515] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.027633 10140 raft_consensus.cc:493] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.032917 10140 raft_consensus.cc:515] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:28.034797  9903 leader_election.cc:304] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:28.043383 10408 raft_consensus.cc:2749] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.045938  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:28.062107 10058 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:28.062925 10058 ts_tablet_manager.cc:1403] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.267s	user 0.018s	sys 0.000s
I20260709 13:05:28.065491  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:28.066067 10140 leader_election.cc:290] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:28.065976  9789 raft_consensus.cc:2468] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:28.066663  9934 raft_consensus.cc:3060] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.073005  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:28.073897  9864 raft_consensus.cc:2468] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:28.076653  9904 leader_election.cc:304] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:28.081585 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:28.087222  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:28.088129  9789 raft_consensus.cc:3060] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.090265 10059 tablet_bootstrap.cc:654] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.091185  9754 leader_election.cc:304] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:28.065608  9788 raft_consensus.cc:3060] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.107455  9683 catalog_manager.cc:5697] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 2, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.112763 10112 raft_consensus.cc:2804] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.113409 10112 raft_consensus.cc:493] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.113889 10112 raft_consensus.cc:3060] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.115262 10140 consensus_queue.cc:1048] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:28.069419 10397 leader_election.cc:290] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:28.129779 10112 raft_consensus.cc:515] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:28.131007  9789 raft_consensus.cc:2468] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:28.133162 10058 raft_consensus.cc:359] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:28.138286  9975 leader_election.cc:304] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:28.138989 10058 raft_consensus.cc:385] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.139443 10058 raft_consensus.cc:740] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.140377 10425 raft_consensus.cc:2804] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.141294 10425 raft_consensus.cc:697] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:28.113381  9683 catalog_manager.cc:5697] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.142702 10425 consensus_queue.cc:237] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.140455 10058 consensus_queue.cc:260] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:28.146803 10057 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:28.150426 10057 ts_tablet_manager.cc:1403] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.298s	user 0.025s	sys 0.024s
I20260709 13:05:28.157014 10058 ts_tablet_manager.cc:1434] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.002s
I20260709 13:05:28.150000 10112 leader_election.cc:290] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:28.165839 10426 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.166455 10426 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.168983  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:28.169714  9864 raft_consensus.cc:3060] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.170246 10426 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:28.171900 10061 tablet_bootstrap.cc:654] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.158946 10303 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:28.148552  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 88194.1us
I20260709 13:05:28.242710 10058 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:28.242594 10057 raft_consensus.cc:359] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.245559 10057 raft_consensus.cc:385] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.246039 10057 raft_consensus.cc:740] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.247994 10057 consensus_queue.cc:260] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.249030 10316 raft_consensus.cc:2749] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.197348 10303 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:28.257445 10057 ts_tablet_manager.cc:1434] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.095s	user 0.011s	sys 0.000s
I20260709 13:05:28.258118 10437 consensus_queue.cc:1048] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:28.264040  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:28.267797  9934 raft_consensus.cc:2468] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:28.269368  9978 leader_election.cc:343] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.284606  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:28.286515  9864 raft_consensus.cc:2468] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
W20260709 13:05:28.286859  9904 leader_election.cc:343] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.287582 10303 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:28.289168 10058 tablet_bootstrap.cc:654] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.290879 10057 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:28.307750  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:28.308921  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:28.309273  9686 catalog_manager.cc:5697] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.312143 10422 consensus_queue.cc:1048] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:28.315011  9904 leader_election.cc:304] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
W20260709 13:05:28.315577  9975 leader_election.cc:343] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.316187  9975 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:28.316493 10303 raft_consensus.cc:2804] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:28.317812 10207 raft_consensus.cc:2749] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.317886 10303 raft_consensus.cc:697] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
W20260709 13:05:28.322489  9901 leader_election.cc:343] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.323284  9901 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:28.324307 10112 raft_consensus.cc:2749] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:28.297292 10435 log.cc:927] Time spent T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:05:28.331243  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:28.331972  9785 raft_consensus.cc:3060] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.337519 10316 consensus_queue.cc:1048] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:28.330766 10303 consensus_queue.cc:237] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:28.381783 10059 tablet_bootstrap.cc:492] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:28.382442  9785 raft_consensus.cc:2468] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:28.382879 10059 ts_tablet_manager.cc:1403] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.490s	user 0.016s	sys 0.038s
I20260709 13:05:28.385445  9685 catalog_manager.cc:5697] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.386106 10425 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.392644 10425 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.398535  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:28.398604  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:28.399467  9934 raft_consensus.cc:2468] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
W20260709 13:05:28.402052  9975 leader_election.cc:343] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.409904  9977 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:28.410979 10422 raft_consensus.cc:2804] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.411435 10422 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.411813 10422 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.416842 10425 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:28.428258 10061 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:28.427892 10059 raft_consensus.cc:359] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:28.429037 10061 ts_tablet_manager.cc:1403] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.398s	user 0.021s	sys 0.004s
I20260709 13:05:28.429066 10059 raft_consensus.cc:385] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.429898 10059 raft_consensus.cc:740] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.431414 10059 consensus_queue.cc:260] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:28.435542 10061 raft_consensus.cc:359] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.437624 10058 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:28.441210 10058 ts_tablet_manager.cc:1403] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.215s	user 0.012s	sys 0.008s
I20260709 13:05:28.440327 10059 ts_tablet_manager.cc:1434] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.057s	user 0.010s	sys 0.000s
I20260709 13:05:28.446662 10057 tablet_bootstrap.cc:654] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.436508 10061 raft_consensus.cc:385] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.447209 10061 raft_consensus.cc:740] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.448331 10061 consensus_queue.cc:260] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.475196 10207 consensus_queue.cc:1048] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:28.448861  9978 consensus_peers.cc:597] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:28.488386  9934 raft_consensus.cc:1275] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:28.497897 10061 ts_tablet_manager.cc:1434] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.005s
I20260709 13:05:28.492858 10260 consensus_queue.cc:1048] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:05:28.499727 10058 raft_consensus.cc:359] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.500681 10058 raft_consensus.cc:385] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.501030 10058 raft_consensus.cc:740] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.502206 10059 tablet_bootstrap.cc:492] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:28.503283 10376 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.504297 10376 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.507606 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:28.508498 10008 raft_consensus.cc:2468] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:28.510639  9754 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:28.516763 10058 consensus_queue.cc:260] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.526495  9859 raft_consensus.cc:1275] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:28.524058  9685 catalog_manager.cc:5697] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.538801 10061 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:28.521307 10316 raft_consensus.cc:2804] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:28.550216 10058 ts_tablet_manager.cc:1434] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.002s
I20260709 13:05:28.541081 10316 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.557585  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:28.559125 10316 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.547334 10207 consensus_queue.cc:1048] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:05:28.574074 10057 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:28.574812 10057 ts_tablet_manager.cc:1403] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.284s	user 0.014s	sys 0.005s
W20260709 13:05:28.579389  9755 leader_election.cc:343] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.547118 10376 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:28.604686 10406 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.605367 10406 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:28.608093 10406 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:28.611167 10059 tablet_bootstrap.cc:654] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.611661 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:28.612645 10057 raft_consensus.cc:359] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.613471 10057 raft_consensus.cc:385] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.613760 10057 raft_consensus.cc:740] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.620503 10058 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:28.624851 10057 consensus_queue.cc:260] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.626405  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:28.627614 10057 ts_tablet_manager.cc:1434] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.001s
I20260709 13:05:28.634765  9829 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:28.642311 10061 tablet_bootstrap.cc:654] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.651970 10057 tablet_bootstrap.cc:492] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:28.647228 10444 raft_consensus.cc:2749] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.671391 10422 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.680495 10058 tablet_bootstrap.cc:654] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.691577 10207 consensus_queue.cc:1048] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:28.695025 10316 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.698084 10316 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:28.700124 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:28.706897  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:28.706888 10008 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:28.725483  9755 leader_election.cc:343] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.733842 10422 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:28.750958 10008 raft_consensus.cc:2468] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:28.755465 10059 tablet_bootstrap.cc:492] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:28.756420 10059 ts_tablet_manager.cc:1403] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.262s	user 0.016s	sys 0.003s
I20260709 13:05:28.758050 10058 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:28.758659 10058 ts_tablet_manager.cc:1403] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.140s	user 0.014s	sys 0.004s
I20260709 13:05:28.760604 10059 raft_consensus.cc:359] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:28.736022  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:28.761869 10059 raft_consensus.cc:385] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.762475 10059 raft_consensus.cc:740] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.763729 10422 consensus_queue.cc:1048] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:28.763661 10059 consensus_queue.cc:260] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:28.767042 10059 ts_tablet_manager.cc:1434] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:28.768567 10059 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:28.769292  9754 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:28.760681  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:28.777024 10316 raft_consensus.cc:2804] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:28.778263  9975 leader_election.cc:343] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.762168  9937 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.778151 10316 raft_consensus.cc:697] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:28.787736 10316 consensus_queue.cc:237] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:28.801870 10057 tablet_bootstrap.cc:654] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.809963 10112 raft_consensus.cc:493] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.811074 10112 raft_consensus.cc:515] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:28.832482  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:28.848402 10112 leader_election.cc:290] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:28.849321 10058 raft_consensus.cc:359] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.850121 10058 raft_consensus.cc:385] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:28.850428 10058 raft_consensus.cc:740] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.853947 10058 consensus_queue.cc:260] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:28.862069 10059 tablet_bootstrap.cc:654] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:28.868978 10058 ts_tablet_manager.cc:1434] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.110s	user 0.009s	sys 0.003s
I20260709 13:05:28.871906  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:28.873450 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:28.877790  9903 leader_election.cc:343] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.890168 10061 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:28.890959 10061 ts_tablet_manager.cc:1403] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.353s	user 0.046s	sys 0.018s
I20260709 13:05:28.915098  9863 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:28.915098  9862 raft_consensus.cc:3060] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:28.919543  9901 leader_election.cc:343] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:28.920431  9901 leader_election.cc:304] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:28.923887 10434 raft_consensus.cc:2749] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:28.924672  9937 raft_consensus.cc:2468] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:28.928429 10057 tablet_bootstrap.cc:492] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:28.929220 10057 ts_tablet_manager.cc:1403] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.285s	user 0.012s	sys 0.018s
I20260709 13:05:28.939878  9683 catalog_manager.cc:5697] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.942853 10058 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:29.059273 10434 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.060037 10059 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:29.059980 10434 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:29.060921 10059 ts_tablet_manager.cc:1403] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.293s	user 0.022s	sys 0.020s
I20260709 13:05:29.106822  9977 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.082948 10061 raft_consensus.cc:359] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.119660 10061 raft_consensus.cc:385] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.121412 10059 raft_consensus.cc:359] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.128283 10059 raft_consensus.cc:385] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.128870 10059 raft_consensus.cc:740] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.100304 10207 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.130678 10059 consensus_queue.cc:260] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.132309 10057 raft_consensus.cc:359] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.141574 10057 raft_consensus.cc:385] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.142493 10057 raft_consensus.cc:740] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.129668 10061 raft_consensus.cc:740] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
W20260709 13:05:29.091321  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 203502us
I20260709 13:05:29.090562 10303 consensus_queue.cc:1048] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:05:29.148933 10057 consensus_queue.cc:260] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.157491 10466 raft_consensus.cc:2804] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:29.158655 10466 raft_consensus.cc:697] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:29.160063 10057 ts_tablet_manager.cc:1434] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.230s	user 0.011s	sys 0.003s
I20260709 13:05:29.161604 10057 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:29.163023 10466 consensus_queue.cc:237] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:29.171069 10057 tablet_bootstrap.cc:654] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:29.181183 10434 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:29.185861 10112 consensus_queue.cc:1048] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.190742 10059 ts_tablet_manager.cc:1434] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.007s
I20260709 13:05:29.193650 10058 tablet_bootstrap.cc:654] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:29.144152 10061 consensus_queue.cc:260] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.227363 10061 ts_tablet_manager.cc:1434] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.335s	user 0.006s	sys 0.016s
I20260709 13:05:29.232805 10059 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:29.144238 10207 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:29.223680  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 161054us
I20260709 13:05:29.254580 10057 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
W20260709 13:05:29.215257  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 120504us
I20260709 13:05:29.255775 10057 ts_tablet_manager.cc:1403] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.094s	user 0.008s	sys 0.018s
W20260709 13:05:29.260147  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 153440us
I20260709 13:05:29.263842 10061 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:29.281221 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:29.301671 10057 raft_consensus.cc:359] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.302753 10057 raft_consensus.cc:385] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.303287 10057 raft_consensus.cc:740] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.305763 10207 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.304144 10057 consensus_queue.cc:260] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.313090 10061 tablet_bootstrap.cc:654] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:29.316322 10057 ts_tablet_manager.cc:1434] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:05:29.318249 10057 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:29.318034  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:29.322988  9903 leader_election.cc:343] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.325347 10473 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.326207 10473 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.328650 10473 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.332783  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:29.367189 10195 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.368221 10195 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:29.356441 10475 log.cc:927] Time spent T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:05:29.389248  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:29.390199  9784 raft_consensus.cc:2468] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:29.391799 10059 tablet_bootstrap.cc:654] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:29.386754  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:29.379639  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:29.368188  9978 leader_election.cc:343] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.410779 10195 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:29.412750  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:29.413905  9933 raft_consensus.cc:2468] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:29.422897  9754 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
W20260709 13:05:29.428908  9904 leader_election.cc:343] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.434333 10195 raft_consensus.cc:2804] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:29.434981 10195 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.435478 10195 raft_consensus.cc:3060] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.454643  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:29.448302  9857 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:29.432782  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 70082.5us
I20260709 13:05:29.472743  9901 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
W20260709 13:05:29.474453  9901 leader_election.cc:343] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.475090  9901 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.476728 10468 raft_consensus.cc:2804] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:29.477200 10468 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.477630 10468 raft_consensus.cc:3060] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.479262 10473 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.483728 10057 tablet_bootstrap.cc:654] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:29.492767  9975 leader_election.cc:343] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.493497  9975 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.498453 10260 raft_consensus.cc:2749] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.524647 10195 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:29.525792  9755 leader_election.cc:343] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.536129 10444 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.536886 10444 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.521950 10437 consensus_queue.cc:1048] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:05:29.542626 10061 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:29.543361 10462 raft_consensus.cc:493] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.543697 10061 ts_tablet_manager.cc:1403] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.291s	user 0.022s	sys 0.006s
I20260709 13:05:29.544005 10462 raft_consensus.cc:515] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.545481 10058 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:29.546185 10058 ts_tablet_manager.cc:1403] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.619s	user 0.026s	sys 0.037s
I20260709 13:05:29.547645 10462 leader_election.cc:290] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:29.548209 10061 raft_consensus.cc:359] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.549021 10061 raft_consensus.cc:385] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.549275 10061 raft_consensus.cc:740] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.549588 10058 raft_consensus.cc:359] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.550297 10058 raft_consensus.cc:385] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.550556 10058 raft_consensus.cc:740] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.551632 10058 consensus_queue.cc:260] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.552681 10061 consensus_queue.cc:260] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.561219 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:29.564050  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:29.583175 10008 raft_consensus.cc:2468] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:29.587044 10059 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:29.588897 10058 ts_tablet_manager.cc:1434] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:05:29.589370 10059 ts_tablet_manager.cc:1403] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.382s	user 0.018s	sys 0.010s
I20260709 13:05:29.592146 10061 ts_tablet_manager.cc:1434] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.004s
I20260709 13:05:29.597839  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:29.600701  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:29.602198 10058 tablet_bootstrap.cc:492] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:29.587340 10465 consensus_queue.cc:1048] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:05:29.619618 10195 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:29.621922  9903 leader_election.cc:304] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:29.622514 10059 raft_consensus.cc:359] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.623332 10059 raft_consensus.cc:385] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.623389 10473 raft_consensus.cc:2804] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:29.623795 10059 raft_consensus.cc:740] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.623924 10473 raft_consensus.cc:493] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.633103 10059 consensus_queue.cc:260] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.630666 10468 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.638531 10061 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:29.641528  9904 leader_election.cc:343] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.636430 10473 raft_consensus.cc:3060] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.664867 10444 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:29.669590 10468 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.670346  9858 raft_consensus.cc:3060] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.684219 10059 ts_tablet_manager.cc:1434] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.004s
I20260709 13:05:29.692700  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:29.697058  9788 raft_consensus.cc:2393] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:29.695539  9683 catalog_manager.cc:5697] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.698288 10059 tablet_bootstrap.cc:492] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:29.713465 10058 tablet_bootstrap.cc:654] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:29.697141  9936 raft_consensus.cc:2393] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:29.685607 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:29.719425  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:29.728883 10057 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:29.729645 10057 ts_tablet_manager.cc:1403] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.412s	user 0.037s	sys 0.048s
I20260709 13:05:29.730259  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:29.736202 10057 raft_consensus.cc:359] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:29.737882  9904 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.746009 10460 raft_consensus.cc:493] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.746675 10460 raft_consensus.cc:515] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.749683 10469 consensus_queue.cc:1048] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:29.751233 10462 raft_consensus.cc:2749] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.752959 10422 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.753603 10460 leader_election.cc:290] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:29.738031  9829 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.759182 10444 raft_consensus.cc:2749] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:29.759403  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:29.762612  9904 leader_election.cc:343] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.737215 10057 raft_consensus.cc:385] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.765837 10057 raft_consensus.cc:740] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.768831 10057 consensus_queue.cc:260] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:29.744144 10059 tablet_bootstrap.cc:654] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:29.778574 10057 ts_tablet_manager.cc:1434] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.005s
I20260709 13:05:29.758558 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:29.781708 10057 tablet_bootstrap.cc:492] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:29.753710 10422 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.791334 10061 tablet_bootstrap.cc:654] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:29.795375  9903 leader_election.cc:343] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.796025  9903 leader_election.cc:304] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:29.791322 10057 tablet_bootstrap.cc:654] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:29.807456  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:29.810062 10460 raft_consensus.cc:2749] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.811151  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:29.812563 10422 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.819542  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:29.830394  9975 leader_election.cc:343] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.836169 10195 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.836979 10195 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.842247 10195 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.856175 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:29.857722 10009 raft_consensus.cc:2468] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:29.858834 10460 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.859342  9978 leader_election.cc:343] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.860219  9978 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.861208  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:29.859660 10460 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.866344 10463 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.867648 10460 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.810045 10473 raft_consensus.cc:515] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.869125  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:29.870734 10473 leader_election.cc:290] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:29.869196  9858 raft_consensus.cc:2468] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:29.879091  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:29.879299  9755 leader_election.cc:343] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:29.882218  9901 leader_election.cc:343] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:29.884177  9904 leader_election.cc:343] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.865760 10061 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:29.885339  9904 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.885993 10061 ts_tablet_manager.cc:1403] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.293s	user 0.015s	sys 0.032s
I20260709 13:05:29.889008  9754 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:29.884889  9755 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: d02e8a054a634be599149db641901736
I20260709 13:05:29.905170 10484 raft_consensus.cc:2804] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:29.904757 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:29.906105 10484 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.906977 10058 tablet_bootstrap.cc:492] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:29.906673 10484 raft_consensus.cc:3060] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.907753 10058 ts_tablet_manager.cc:1403] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.306s	user 0.020s	sys 0.030s
I20260709 13:05:29.917972 10058 raft_consensus.cc:359] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.919329 10480 raft_consensus.cc:2804] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:29.921507 10058 raft_consensus.cc:385] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.924949 10059 tablet_bootstrap.cc:492] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:29.921929 10058 raft_consensus.cc:740] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.925611 10059 ts_tablet_manager.cc:1403] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.232s	user 0.012s	sys 0.017s
I20260709 13:05:29.926594 10058 consensus_queue.cc:260] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.921561  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:29.906594 10009 raft_consensus.cc:3060] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:29.933641 10061 raft_consensus.cc:359] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:29.934607 10061 raft_consensus.cc:385] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.934942 10061 raft_consensus.cc:740] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
W20260709 13:05:29.936101  9904 leader_election.cc:343] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.896229 10473 raft_consensus.cc:2749] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.945580 10480 raft_consensus.cc:697] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
W20260709 13:05:29.946010  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:29.946874 10480 consensus_queue.cc:237] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.955683 10473 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.956494 10473 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:29.959088 10473 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:29.959972 10009 raft_consensus.cc:2468] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:29.963167  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:29.965173 10059 raft_consensus.cc:359] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:29.966149 10059 raft_consensus.cc:385] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:29.966663 10059 raft_consensus.cc:740] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.949008 10061 consensus_queue.cc:260] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:29.971612  9903 leader_election.cc:304] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:29.971678  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:29.979315  9901 leader_election.cc:343] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.981065 10462 raft_consensus.cc:2804] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:29.982039 10462 raft_consensus.cc:697] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:29.987317 10057 tablet_bootstrap.cc:492] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
W20260709 13:05:29.987444  9904 leader_election.cc:343] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.990084  9904 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:29.991914 10434 raft_consensus.cc:2749] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:29.988159 10057 ts_tablet_manager.cc:1403] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.207s	user 0.024s	sys 0.020s
I20260709 13:05:29.972296 10058 ts_tablet_manager.cc:1434] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.001s
I20260709 13:05:29.996803 10059 consensus_queue.cc:260] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.008754 10061 ts_tablet_manager.cc:1434] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.122s	user 0.009s	sys 0.002s
I20260709 13:05:30.013687 10059 ts_tablet_manager.cc:1434] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.087s	user 0.005s	sys 0.006s
I20260709 13:05:30.015816 10484 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:29.983611 10462 consensus_queue.cc:237] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.025955 10058 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:30.034533 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:30.035483 10009 raft_consensus.cc:3060] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.043150 10484 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:30.030968 10057 raft_consensus.cc:359] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.046494 10057 raft_consensus.cc:385] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.044196  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:30.055585 10061 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:30.046981 10057 raft_consensus.cc:740] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.058616 10057 consensus_queue.cc:260] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.073518 10059 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:30.081930 10009 raft_consensus.cc:2468] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:30.088483  9754 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:30.089807 10376 raft_consensus.cc:2804] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:30.090735 10376 raft_consensus.cc:697] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:30.092067 10376 consensus_queue.cc:237] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.097515 10444 raft_consensus.cc:493] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.098963 10444 raft_consensus.cc:515] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.105443  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:05:30.105726  9755 leader_election.cc:343] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.120810 10444 leader_election.cc:290] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:30.124811 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3324f833b7b462d8aa8db06fb4bd74e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:30.126504 10009 raft_consensus.cc:2393] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:30.129431  9829 leader_election.cc:304] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:30.139043 10057 ts_tablet_manager.cc:1434] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.145s	user 0.009s	sys 0.002s
I20260709 13:05:30.152750  9684 catalog_manager.cc:5697] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.146435 10444 raft_consensus.cc:3060] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.166131 10195 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.166795 10195 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.162667 10422 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.168907 10195 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:30.169170 10422 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.170276 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:30.171554 10009 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:30.175120  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:30.176028  9788 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:30.178215  9754 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:30.178329  9975 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:30.178704 10058 tablet_bootstrap.cc:654] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.179355 10195 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:30.179772 10195 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.180065 10195 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.180610 10207 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:30.181177 10207 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.181583 10207 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.172027 10422 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:30.154552 10434 raft_consensus.cc:493] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.191680 10057 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:30.218276 10061 tablet_bootstrap.cc:654] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.190129 10434 raft_consensus.cc:515] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.219811 10195 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.220914 10059 tablet_bootstrap.cc:654] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.222085 10434 leader_election.cc:290] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:30.223839 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:30.189671  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:30.236382 10207 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.239236 10195 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:30.241360 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:30.243042 10009 raft_consensus.cc:2393] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:30.190299  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:30.253075 10207 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:30.257601  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:30.260583  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
W20260709 13:05:30.250571  9903 leader_election.cc:343] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.266844  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:30.269469 10444 raft_consensus.cc:2749] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.270761  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:30.272296  9788 raft_consensus.cc:2393] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:30.272527  9754 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:30.241257  9933 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:30.274286 10465 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.291129  9975 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:30.277123  9685 catalog_manager.cc:5697] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:30.307125  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 88275.7us
I20260709 13:05:30.307858 10463 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.322206 10061 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:30.323285 10061 ts_tablet_manager.cc:1403] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.275s	user 0.012s	sys 0.009s
I20260709 13:05:30.331692 10061 raft_consensus.cc:359] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.332625 10061 raft_consensus.cc:385] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.332933 10061 raft_consensus.cc:740] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.333766 10061 consensus_queue.cc:260] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.338376 10061 ts_tablet_manager.cc:1434] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:05:30.340319 10061 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:30.348704 10061 tablet_bootstrap.cc:654] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:30.353487  9904 leader_election.cc:343] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.354175  9904 leader_election.cc:304] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:30.364684 10057 tablet_bootstrap.cc:654] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.359170 10473 raft_consensus.cc:2749] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.384028 10058 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:30.384927 10058 ts_tablet_manager.cc:1403] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.377s	user 0.018s	sys 0.009s
I20260709 13:05:30.401908 10470 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.406417 10470 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.410075  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:30.410723 10059 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:30.411338 10059 ts_tablet_manager.cc:1403] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.362s	user 0.023s	sys 0.019s
W20260709 13:05:30.413023  9975 leader_election.cc:343] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.414654 10059 raft_consensus.cc:359] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.415500 10059 raft_consensus.cc:385] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.415791 10059 raft_consensus.cc:740] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.418125 10470 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:30.419386 10058 raft_consensus.cc:359] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.420256 10058 raft_consensus.cc:385] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.420914 10058 raft_consensus.cc:740] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.425724  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:30.427481 10058 consensus_queue.cc:260] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.430953 10058 ts_tablet_manager.cc:1434] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
W20260709 13:05:30.431939  9978 leader_election.cc:343] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.432718  9978 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:30.434443 10058 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:30.443888 10470 raft_consensus.cc:2749] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.432317 10059 consensus_queue.cc:260] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.455076 10058 tablet_bootstrap.cc:654] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.459259 10061 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:30.460043 10061 ts_tablet_manager.cc:1403] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.120s	user 0.010s	sys 0.012s
I20260709 13:05:30.462983 10059 ts_tablet_manager.cc:1434] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.002s
I20260709 13:05:30.464536 10059 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:30.466864  9685 catalog_manager.cc:5697] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:30.473524 10061 raft_consensus.cc:359] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.474489 10061 raft_consensus.cc:385] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.474844 10061 raft_consensus.cc:740] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.475883 10061 consensus_queue.cc:260] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.482134 10057 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:30.483507 10057 ts_tablet_manager.cc:1403] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.318s	user 0.008s	sys 0.012s
I20260709 13:05:30.503021 10061 ts_tablet_manager.cc:1434] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:05:30.504685 10061 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:30.506464 10195 consensus_queue.cc:1048] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:30.516868 10061 tablet_bootstrap.cc:654] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.533845 10059 tablet_bootstrap.cc:654] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.487864 10057 raft_consensus.cc:359] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.542085 10058 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:30.551302 10057 raft_consensus.cc:385] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.551151 10504 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.551728 10058 ts_tablet_manager.cc:1403] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.118s	user 0.029s	sys 0.009s
I20260709 13:05:30.552088 10504 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.553241 10465 consensus_queue.cc:1048] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:30.554860 10504 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:30.555538 10057 raft_consensus.cc:740] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.556578 10057 consensus_queue.cc:260] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.558146  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:30.560704 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:30.562422 10470 raft_consensus.cc:493] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.563153 10470 raft_consensus.cc:515] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.566956 10057 ts_tablet_manager.cc:1434] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.083s	user 0.009s	sys 0.000s
I20260709 13:05:30.568684  9829 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:30.568863 10473 consensus_queue.cc:1048] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:30.571908 10057 tablet_bootstrap.cc:492] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:30.572469 10504 raft_consensus.cc:2749] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.571424 10058 raft_consensus.cc:359] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.573136 10058 raft_consensus.cc:385] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.573493 10058 raft_consensus.cc:740] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.574258 10058 consensus_queue.cc:260] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.576838 10058 ts_tablet_manager.cc:1434] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:05:30.578326 10470 leader_election.cc:290] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:30.581900 10434 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.582903 10434 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.584898  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:30.585805  9936 raft_consensus.cc:2468] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:30.587458  9977 leader_election.cc:304] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:30.590344 10058 tablet_bootstrap.cc:492] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:30.591209 10470 raft_consensus.cc:2804] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:30.591665 10470 raft_consensus.cc:493] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.594062  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:30.596043  9978 leader_election.cc:343] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.592067 10470 raft_consensus.cc:3060] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.611918 10376 consensus_queue.cc:1048] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:30.618846 10057 tablet_bootstrap.cc:654] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.624871  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:30.632072  9901 leader_election.cc:343] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.644237 10058 tablet_bootstrap.cc:654] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:30.619863 10468 consensus_queue.cc:1048] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:05:30.656000 10434 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:30.655342  9861 raft_consensus.cc:3060] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.696623  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:30.699353  9904 leader_election.cc:343] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.699954  9904 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:30.731905 10468 raft_consensus.cc:2749] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.755786 10434 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.765594 10434 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.775207  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:30.777532  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:30.780948  9904 leader_election.cc:343] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:30.785615  9901 leader_election.cc:343] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.785598 10422 raft_consensus.cc:493] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.786409  9901 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:30.786629 10422 raft_consensus.cc:515] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.788317 10061 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:30.788492 10468 raft_consensus.cc:2749] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:30.789611 10061 ts_tablet_manager.cc:1403] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.285s	user 0.043s	sys 0.036s
I20260709 13:05:30.790434  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:30.791061 10422 leader_election.cc:290] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:30.791229  9937 raft_consensus.cc:2468] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:30.792786  9977 leader_election.cc:304] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:30.793536 10422 raft_consensus.cc:2804] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:30.793993 10422 raft_consensus.cc:493] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.794431 10422 raft_consensus.cc:3060] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.800005  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:30.808708 10484 consensus_queue.cc:1048] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:30.812884  9975 leader_election.cc:343] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.825189 10059 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:30.826051 10059 ts_tablet_manager.cc:1403] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.362s	user 0.021s	sys 0.022s
I20260709 13:05:30.830849 10059 raft_consensus.cc:359] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.831972 10059 raft_consensus.cc:385] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.844733  9788 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.832304 10059 raft_consensus.cc:740] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.768241 10434 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:30.860759 10470 raft_consensus.cc:515] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:30.856439 10503 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.863142 10503 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:30.865443  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:30.858465 10059 consensus_queue.cc:260] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:30.869915  9978 leader_election.cc:343] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:30.870365 10061 raft_consensus.cc:359] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.871599 10061 raft_consensus.cc:385] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:30.872458 10061 raft_consensus.cc:740] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.882900 10059 ts_tablet_manager.cc:1434] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.056s	user 0.011s	sys 0.000s
I20260709 13:05:30.884297 10061 consensus_queue.cc:260] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:30.905651 10061 ts_tablet_manager.cc:1434] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.116s	user 0.013s	sys 0.000s
I20260709 13:05:30.907236 10470 leader_election.cc:290] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:30.866285 10422 raft_consensus.cc:515] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.921928  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:30.931346  9859 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:30.936620  9755 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:30.904793 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:30.905196 10503 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:30.944741 10437 raft_consensus.cc:2804] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:30.948575 10437 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.949030 10437 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:30.954074 10061 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:30.967741  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:30.991511 10057 tablet_bootstrap.cc:492] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:30.995133 10057 ts_tablet_manager.cc:1403] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.427s	user 0.017s	sys 0.004s
I20260709 13:05:30.995307 10059 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:30.988794 10484 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:31.001930  9937 raft_consensus.cc:3060] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.002619 10484 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:30.974349 10008 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:30.994378 10437 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.049737 10057 raft_consensus.cc:359] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.050957 10057 raft_consensus.cc:385] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.051358 10057 raft_consensus.cc:740] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.052204 10057 consensus_queue.cc:260] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.063046 10057 ts_tablet_manager.cc:1434] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.002s
I20260709 13:05:31.075945  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:31.076963  9933 raft_consensus.cc:3060] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.117799 10437 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:31.123914  9937 raft_consensus.cc:2468] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:31.124557 10484 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:31.125857 10422 leader_election.cc:290] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:31.128126  9977 leader_election.cc:304] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:31.130149 10464 raft_consensus.cc:2804] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:31.130723 10464 raft_consensus.cc:697] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:31.131906 10464 consensus_queue.cc:237] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:31.136303  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:31.147012  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:31.116297  9933 raft_consensus.cc:2468] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:31.152487  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:31.153412  9858 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.155670 10059 tablet_bootstrap.cc:654] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.153012 10061 tablet_bootstrap.cc:654] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.153267  9977 leader_election.cc:304] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:31.166852 10207 raft_consensus.cc:2804] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:31.167630  9975 leader_election.cc:343] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.168313 10207 raft_consensus.cc:697] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:31.171890 10207 consensus_queue.cc:237] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.181726 10058 tablet_bootstrap.cc:492] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:31.182466 10058 ts_tablet_manager.cc:1403] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.594s	user 0.027s	sys 0.000s
I20260709 13:05:31.185359 10463 raft_consensus.cc:493] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.186172 10463 raft_consensus.cc:515] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.196813  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:31.197149  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:31.207402  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 45235.5us
I20260709 13:05:31.213707 10057 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:31.214915 10463 leader_election.cc:290] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:31.217366 10058 raft_consensus.cc:359] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.218502 10058 raft_consensus.cc:385] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.218798 10058 raft_consensus.cc:740] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.219750 10058 consensus_queue.cc:260] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.222666 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:31.223850 10012 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.224656 10058 ts_tablet_manager.cc:1434] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.002s
I20260709 13:05:31.224828 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:31.226311 10058 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:31.226897 10437 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.227706 10437 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:31.237315  9978 leader_election.cc:343] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.244581 10437 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:31.246282  9684 catalog_manager.cc:5697] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:31.255506  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:31.256263  9934 raft_consensus.cc:2468] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:31.256855  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:31.262451  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:31.263504  9937 raft_consensus.cc:2468] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:31.267148 10061 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:31.267868 10061 ts_tablet_manager.cc:1403] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.318s	user 0.018s	sys 0.000s
I20260709 13:05:31.272681 10061 raft_consensus.cc:359] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.273579 10061 raft_consensus.cc:385] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.273910 10061 raft_consensus.cc:740] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.274859 10061 consensus_queue.cc:260] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.277442  9977 leader_election.cc:304] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:31.278517 10061 ts_tablet_manager.cc:1434] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:05:31.279104 10422 raft_consensus.cc:2804] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:31.279656 10422 raft_consensus.cc:493] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.278554 10059 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:31.281119 10059 ts_tablet_manager.cc:1403] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.308s	user 0.021s	sys 0.004s
I20260709 13:05:31.280028 10422 raft_consensus.cc:3060] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.284777 10059 raft_consensus.cc:359] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.285677 10059 raft_consensus.cc:385] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.286010 10059 raft_consensus.cc:740] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.256084  9684 catalog_manager.cc:5697] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.280141 10061 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:31.290844  9755 leader_election.cc:343] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.290007  9754 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:31.293210 10058 tablet_bootstrap.cc:654] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.293532 10012 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:31.297307 10503 raft_consensus.cc:2749] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:31.297132  9858 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:31.301400  9754 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:31.302884 10057 tablet_bootstrap.cc:654] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.306944 10503 raft_consensus.cc:2804] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:31.307572 10503 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.308187 10503 raft_consensus.cc:3060] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.291028 10059 consensus_queue.cc:260] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.311832  9755 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:31.314827 10484 raft_consensus.cc:2804] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:31.315459 10484 raft_consensus.cc:697] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:31.316447 10484 consensus_queue.cc:237] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.322849 10059 ts_tablet_manager.cc:1434] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.002s
I20260709 13:05:31.338289 10061 tablet_bootstrap.cc:654] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.364017 10059 tablet_bootstrap.cc:492] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:31.381729 10422 raft_consensus.cc:515] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.411157 10058 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:31.411928 10058 ts_tablet_manager.cc:1403] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.186s	user 0.010s	sys 0.010s
I20260709 13:05:31.424045 10057 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:31.424782 10057 ts_tablet_manager.cc:1403] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.211s	user 0.018s	sys 0.004s
I20260709 13:05:31.428431 10058 raft_consensus.cc:359] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.379674 10503 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.431067 10057 raft_consensus.cc:359] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.432442 10057 raft_consensus.cc:385] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.432551 10462 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.432847 10057 raft_consensus.cc:740] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.433548 10503 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:31.433882  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:31.434329  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:31.435053 10061 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:31.435672 10061 ts_tablet_manager.cc:1403] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.156s	user 0.019s	sys 0.002s
I20260709 13:05:31.429569 10058 raft_consensus.cc:385] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:31.437901  9978 leader_election.cc:343] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.435037  9937 raft_consensus.cc:3060] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.439558  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:31.441906  9858 raft_consensus.cc:3060] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.433166 10462 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.437928 10058 raft_consensus.cc:740] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.448436 10058 consensus_queue.cc:260] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.458287  9685 catalog_manager.cc:5697] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.462754 10061 raft_consensus.cc:359] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.464104 10061 raft_consensus.cc:385] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.464545  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:31.469918 10061 raft_consensus.cc:740] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.470381  9933 raft_consensus.cc:3060] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.470853 10061 consensus_queue.cc:260] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.471470  9937 raft_consensus.cc:2468] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:31.463876 10057 consensus_queue.cc:260] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.473208  9754 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:31.469298 10058 ts_tablet_manager.cc:1434] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.057s	user 0.011s	sys 0.000s
I20260709 13:05:31.474291 10484 raft_consensus.cc:2804] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:31.477347 10484 raft_consensus.cc:697] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:31.478546 10484 consensus_queue.cc:237] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.453212 10422 leader_election.cc:290] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:31.476640 10061 ts_tablet_manager.cc:1434] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.004s
I20260709 13:05:31.487269 10057 ts_tablet_manager.cc:1434] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.000s
I20260709 13:05:31.489338 10058 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:31.482533 10462 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:31.492641 10061 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:31.505024 10059 tablet_bootstrap.cc:654] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.495779  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:31.512511 10057 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
W20260709 13:05:31.517335  9904 leader_election.cc:343] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.514573 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:31.532437  9858 raft_consensus.cc:2468] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
W20260709 13:05:31.537638  9978 consensus_peers.cc:597] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.540040 10058 tablet_bootstrap.cc:654] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.542883  9933 raft_consensus.cc:2468] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:31.547669  9903 leader_election.cc:343] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.545418 10061 tablet_bootstrap.cc:654] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.550439 10057 tablet_bootstrap.cc:654] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.557808  9903 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:31.565241 10528 raft_consensus.cc:2749] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:31.593792  9977 leader_election.cc:304] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:31.598562 10422 raft_consensus.cc:2804] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:31.599406 10422 raft_consensus.cc:697] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:31.600674  9684 catalog_manager.cc:5697] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:31.615453 10195 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.616452 10195 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.660950 10057 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:31.661765 10057 ts_tablet_manager.cc:1403] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.027s
I20260709 13:05:31.675359 10207 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.622836 10422 consensus_queue.cc:237] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.652525 10480 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.680579 10059 tablet_bootstrap.cc:492] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:31.683473 10195 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:31.684077 10059 ts_tablet_manager.cc:1403] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.320s	user 0.024s	sys 0.012s
I20260709 13:05:31.676110 10207 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.703186 10207 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:31.708698 10057 raft_consensus.cc:359] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.710002 10057 raft_consensus.cc:385] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.710402 10057 raft_consensus.cc:740] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.714061  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:31.715924  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:31.720819 10057 consensus_queue.cc:260] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.732076  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:31.772850 10550 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.695616 10480 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.773655 10550 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:31.775002  9788 raft_consensus.cc:2468] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:31.756294 10061 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:31.776202 10480 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:31.764808 10059 raft_consensus.cc:359] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.777174 10059 raft_consensus.cc:385] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.778448 10059 raft_consensus.cc:740] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.770838 10058 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:31.778517  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:31.777437  9755 leader_election.cc:343] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.776147 10061 ts_tablet_manager.cc:1403] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.287s	user 0.014s	sys 0.015s
I20260709 13:05:31.768163  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:31.786785 10059 consensus_queue.cc:260] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:31.788976 10058 ts_tablet_manager.cc:1403] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.300s	user 0.023s	sys 0.016s
I20260709 13:05:31.790899 10469 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.791616 10469 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.788861  9934 raft_consensus.cc:2468] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:31.777758 10057 ts_tablet_manager.cc:1434] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.000s
I20260709 13:05:31.796093  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:31.792037  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:31.796320 10061 raft_consensus.cc:359] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:31.799219  9975 leader_election.cc:343] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:31.798964  9755 leader_election.cc:343] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:31.802860  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 83029.3us
I20260709 13:05:31.800522 10469 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:31.800243  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:31.807286  9975 leader_election.cc:343] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.799834 10550 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:31.807629 10058 raft_consensus.cc:359] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.808986  9975 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:31.809326 10058 raft_consensus.cc:385] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.809883 10058 raft_consensus.cc:740] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.810323 10550 raft_consensus.cc:2804] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:31.810846 10550 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.811245 10550 raft_consensus.cc:3060] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.812489 10059 ts_tablet_manager.cc:1434] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.128s	user 0.011s	sys 0.000s
I20260709 13:05:31.814080 10059 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:31.814419 10058 consensus_queue.cc:260] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.801327 10061 raft_consensus.cc:385] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:31.817054 10061 raft_consensus.cc:740] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.796367 10057 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:31.826900 10061 consensus_queue.cc:260] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:31.821388 10550 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:31.840507  9978 leader_election.cc:343] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.840754 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:31.856769  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:31.853089 10462 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.858124 10462 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:31.838611 10553 consensus_queue.cc:1048] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:31.857030 10011 raft_consensus.cc:2393] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:31.861724 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:31.857543  9788 raft_consensus.cc:3060] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.863476 10462 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:31.855758 10550 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:31.865228  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:31.865875  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:31.868196  9901 leader_election.cc:343] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:31.869247  9903 leader_election.cc:343] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:31.870091  9903 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:31.872897 10057 tablet_bootstrap.cc:654] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:31.873535 10058 ts_tablet_manager.cc:1434] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.001s
I20260709 13:05:31.874961 10058 tablet_bootstrap.cc:492] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:31.876945 10555 raft_consensus.cc:2749] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:31.885659  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:31.887491 10195 consensus_queue.cc:1048] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:31.853191  9754 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:31.895807 10195 raft_consensus.cc:2804] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:31.896728 10195 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.897209 10195 raft_consensus.cc:3060] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:31.901595 10061 ts_tablet_manager.cc:1434] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.116s	user 0.010s	sys 0.000s
I20260709 13:05:31.999665 10061 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:31.873884  9685 catalog_manager.cc:5697] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:31.893285  9788 raft_consensus.cc:2468] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:31.898227  9978 leader_election.cc:343] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:31.988574  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 135471us
I20260709 13:05:31.990736 10554 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.005590 10554 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.006785 10059 tablet_bootstrap.cc:654] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.009305 10554 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:31.991183 10057 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:32.014873 10057 ts_tablet_manager.cc:1403] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.219s	user 0.025s	sys 0.000s
W20260709 13:05:31.998133 10560 log.cc:927] Time spent T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.103s	user 0.000s	sys 0.009s
I20260709 13:05:31.997026 10195 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.025614 10195 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:32.027472 10556 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.028290 10556 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:32.030699 10556 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:32.036082  9975 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:32.037771 10422 raft_consensus.cc:2804] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:32.038447 10422 raft_consensus.cc:697] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:32.039506 10422 consensus_queue.cc:237] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:32.040786  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:32.041711  9937 raft_consensus.cc:2468] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:32.045900  9754 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:32.053809  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:32.054718  9937 raft_consensus.cc:2468] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:32.067401 10061 tablet_bootstrap.cc:654] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.069804 10057 raft_consensus.cc:359] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:32.070798 10057 raft_consensus.cc:385] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:32.071116 10057 raft_consensus.cc:740] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.077466 10057 consensus_queue.cc:260] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:32.079026 10480 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:32.068176  9978 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:32.016117 10058 tablet_bootstrap.cc:654] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.093927 10195 consensus_queue.cc:1048] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:32.095649  9977 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:32.097886  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 199679us
W20260709 13:05:32.099129  9978 leader_election.cc:343] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:32.106878  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:32.107475  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:32.108166  9936 raft_consensus.cc:3060] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:32.108518 10551 raft_consensus.cc:2749] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:32.109784 10464 raft_consensus.cc:2804] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:32.110435 10464 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.110818 10464 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:32.113233  9859 raft_consensus.cc:3060] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:32.098634 10469 consensus_queue.cc:1048] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:32.120920  9754 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:32.131814  9936 raft_consensus.cc:2468] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:32.132444 10195 raft_consensus.cc:2804] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:32.143162 10195 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.143920 10195 raft_consensus.cc:3060] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:32.139071  9755 leader_election.cc:343] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:32.141575 10057 ts_tablet_manager.cc:1434] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.001s
I20260709 13:05:32.148051  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:32.148936  9933 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
W20260709 13:05:32.183820  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 48919.3us
I20260709 13:05:32.133128  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:32.224363 10057 tablet_bootstrap.cc:492] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:32.199040  9754 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:32.245504 10058 tablet_bootstrap.cc:492] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:32.246273 10058 ts_tablet_manager.cc:1403] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.372s	user 0.017s	sys 0.013s
I20260709 13:05:32.248528 10551 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.249212 10551 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:32.250809 10059 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:32.251477 10059 ts_tablet_manager.cc:1403] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.438s	user 0.012s	sys 0.008s
W20260709 13:05:32.250605  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 51683.2us
I20260709 13:05:32.190867 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:32.280185  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:32.251296 10484 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:32.311455 10551 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:32.325538 10195 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:32.328653  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 203891us
W20260709 13:05:32.325141  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 105337us
I20260709 13:05:32.289166 10504 raft_consensus.cc:493] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:32.342623 10058 raft_consensus.cc:359] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:32.344122 10058 raft_consensus.cc:385] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:32.344662 10058 raft_consensus.cc:740] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.289600 10012 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:32.350361 10058 consensus_queue.cc:260] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:32.348170 10059 raft_consensus.cc:359] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.353595 10059 raft_consensus.cc:385] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:32.354101 10059 raft_consensus.cc:740] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.314270 10484 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.384968 10059 consensus_queue.cc:260] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.390882 10484 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.409693  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:32.411039  9938 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:32.433815  9754 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:32.346905 10504 raft_consensus.cc:515] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:32.445551 10058 ts_tablet_manager.cc:1434] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.199s	user 0.010s	sys 0.008s
I20260709 13:05:32.452226 10059 ts_tablet_manager.cc:1434] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.200s	user 0.010s	sys 0.008s
I20260709 13:05:32.453507 10058 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:32.479386  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:32.489269 10195 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:32.491078 10569 log.cc:927] Time spent T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:32.495324 10464 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:32.439522 10480 consensus_queue.cc:1048] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:05:32.496706 10061 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:32.497565 10061 ts_tablet_manager.cc:1403] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.498s	user 0.044s	sys 0.043s
W20260709 13:05:32.510082  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 66431.9us
W20260709 13:05:32.511262  9755 leader_election.cc:343] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:32.517292  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 83588.5us
I20260709 13:05:32.501808 10376 raft_consensus.cc:2804] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:32.528724 10057 tablet_bootstrap.cc:654] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.539223  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:32.542148  9936 raft_consensus.cc:3060] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:32.545106  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:32.549024 10061 raft_consensus.cc:359] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:32.549957 10061 raft_consensus.cc:385] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:32.550259 10061 raft_consensus.cc:740] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.550985 10061 consensus_queue.cc:260] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:32.531599 10578 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.554875 10578 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:32.557700 10578 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:32.535727 10376 raft_consensus.cc:697] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:32.560446 10376 consensus_queue.cc:237] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.568511  9936 raft_consensus.cc:2468] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:32.582839 10061 ts_tablet_manager.cc:1434] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.002s
I20260709 13:05:32.596252  9754 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:32.609985 10061 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:32.531841 10503 consensus_queue.cc:1048] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.063s
W20260709 13:05:32.614790  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 108760us
W20260709 13:05:32.615311  9755 leader_election.cc:343] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:32.637377 10195 raft_consensus.cc:2804] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:32.638130 10195 raft_consensus.cc:697] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
W20260709 13:05:32.640072  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 44050.5us
I20260709 13:05:32.639027 10195 consensus_queue.cc:237] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.624408 10464 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:32.660940  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:32.674042 10484 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:32.696962 10057 tablet_bootstrap.cc:492] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:32.697835 10057 ts_tablet_manager.cc:1403] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.474s	user 0.019s	sys 0.009s
I20260709 13:05:32.696776 10572 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.703071 10572 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:32.711979 10058 tablet_bootstrap.cc:654] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.713169  9934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:32.717268  9934 raft_consensus.cc:2393] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
W20260709 13:05:32.716977  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 49300.4us
I20260709 13:05:32.717046  9788 raft_consensus.cc:2393] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 2: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
W20260709 13:05:32.727775  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 121075us
W20260709 13:05:32.742311  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 91020.9us
I20260709 13:05:32.748504  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:32.749461  9935 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:32.749873  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
W20260709 13:05:32.751214  9975 leader_election.cc:343] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:32.764066 10057 raft_consensus.cc:359] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:32.765223 10057 raft_consensus.cc:385] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:32.765550 10057 raft_consensus.cc:740] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:32.766361 10057 consensus_queue.cc:260] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:32.767108  9977 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:32.743093  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:32.737957 10572 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:32.790222 10057 ts_tablet_manager.cc:1434] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.092s	user 0.008s	sys 0.001s
I20260709 13:05:32.800655 10589 consensus_queue.cc:1048] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:05:32.791147 10469 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:32.804931 10469 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.805375 10469 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.807665 10059 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:32.816751 10484 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:32.836115 10061 tablet_bootstrap.cc:654] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.847648  9935 raft_consensus.cc:2468] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:32.850706 10059 tablet_bootstrap.cc:654] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:32.851207 10504 leader_election.cc:290] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:32.885703  9978 consensus_peers.cc:597] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:32.855443 10558 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.855166  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 58620.7us
I20260709 13:05:32.816587 10057 tablet_bootstrap.cc:492] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
W20260709 13:05:32.896413  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 152862us
W20260709 13:05:32.887135  9978 leader_election.cc:343] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:32.901976  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 161684us
I20260709 13:05:32.897421  9978 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:32.907241  9788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:32.915242  9977 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:32.928877 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:32.925048  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:32.907465 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82893adc39f24309a241ee3e5ff6611e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:32.908483 10558 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:32.942369 10058 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:32.922106 10550 raft_consensus.cc:2749] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:32.953660 10058 ts_tablet_manager.cc:1403] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.500s	user 0.023s	sys 0.008s
W20260709 13:05:32.961550  9975 leader_election.cc:343] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:32.966833  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 59958.6us
I20260709 13:05:32.943327 10587 raft_consensus.cc:2804] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:32.944701 10469 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:32.977283 10469 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:32.977607  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:32.980751  9935 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.980998 10010 raft_consensus.cc:2393] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 2: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:32.995620  9829 leader_election.cc:304] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:33.003170 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:33.004671 10587 raft_consensus.cc:697] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:33.005476  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:33.005775 10587 consensus_queue.cc:237] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:32.976118 10558 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:33.006330  9827 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:33.000828 10504 raft_consensus.cc:2749] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:33.008842 10567 raft_consensus.cc:3060] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.012883  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:33.014187  9786 raft_consensus.cc:2393] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 2: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:33.018972 10555 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.019760 10555 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:33.021999 10469 consensus_queue.cc:1048] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:33.022948 10555 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:32.991791 10594 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:33.033641  9904 leader_election.cc:343] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.060047 10057 tablet_bootstrap.cc:654] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.030553 10594 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:33.079210  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:33.145264 10600 log.cc:927] Time spent T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
W20260709 13:05:33.198067  9904 leader_election.cc:343] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.202256  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:33.205353  9935 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
W20260709 13:05:33.149914  9903 leader_election.cc:343] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.216701  9903 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:33.219280 10594 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:33.216086 10606 log.cc:927] Time spent T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:05:33.234416  9978 consensus_peers.cc:597] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:33.235054  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 219883us
I20260709 13:05:33.245635 10059 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:33.246330 10059 ts_tablet_manager.cc:1403] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.439s	user 0.034s	sys 0.006s
I20260709 13:05:33.251788  9683 catalog_manager.cc:5697] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.239116  9754 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:33.266018 10059 raft_consensus.cc:359] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:33.269702 10059 raft_consensus.cc:385] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.270529 10059 raft_consensus.cc:740] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.271687 10059 consensus_queue.cc:260] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:33.269762 10594 consensus_queue.cc:1048] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:33.283460 10504 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.284183 10504 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:33.285995 10058 raft_consensus.cc:359] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:33.287179 10058 raft_consensus.cc:385] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.287230 10504 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:33.287547 10058 raft_consensus.cc:740] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
W20260709 13:05:33.289768  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 141537us
I20260709 13:05:33.293092 10558 raft_consensus.cc:2749] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.292210 10061 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:33.296865  9937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
W20260709 13:05:33.288229  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 91457.4us
I20260709 13:05:33.296942 10061 ts_tablet_manager.cc:1403] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.692s	user 0.039s	sys 0.031s
I20260709 13:05:33.297353 10058 consensus_queue.cc:260] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:33.299063  9937 raft_consensus.cc:2393] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 2: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:33.268611 10376 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:33.320189 10376 raft_consensus.cc:697] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:33.278158 10551 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.320984 10551 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.321412 10376 consensus_queue.cc:237] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.323489 10551 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:33.308266 10059 ts_tablet_manager.cc:1434] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.062s	user 0.010s	sys 0.000s
I20260709 13:05:33.288511 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:33.318753 10567 raft_consensus.cc:2749] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.338275 10057 tablet_bootstrap.cc:492] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:33.349782 10057 ts_tablet_manager.cc:1403] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.534s	user 0.045s	sys 0.015s
I20260709 13:05:33.343176  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:33.343676  9977 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:33.356422  9901 leader_election.cc:343] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.357270  9901 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:33.358825 10058 ts_tablet_manager.cc:1434] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.405s	user 0.007s	sys 0.003s
I20260709 13:05:33.364734 10059 tablet_bootstrap.cc:492] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:33.359791 10531 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:33.354153 10012 raft_consensus.cc:2468] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:33.379243 10595 raft_consensus.cc:2749] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:33.403733  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:33.384684 10558 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.423969 10058 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: Bootstrap starting.
I20260709 13:05:33.428813  9932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:33.407240  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:33.449317 10057 raft_consensus.cc:359] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.452661 10057 raft_consensus.cc:385] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.453022 10057 raft_consensus.cc:740] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.413612 10558 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.458276 10558 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:33.459040 10061 raft_consensus.cc:359] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:33.461548  9755 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:33.464915 10582 consensus_queue.cc:1048] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:33.460110 10061 raft_consensus.cc:385] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.468940 10061 raft_consensus.cc:740] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.469743 10061 consensus_queue.cc:260] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.472631 10057 consensus_queue.cc:260] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.474541  9932 raft_consensus.cc:2468] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:33.486091  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:33.487648 10058 tablet_bootstrap.cc:654] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:33.498164  9755 leader_election.cc:343] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.519932 10061 ts_tablet_manager.cc:1434] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.222s	user 0.009s	sys 0.001s
I20260709 13:05:33.525219 10577 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.525878 10577 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:33.526808 10059 tablet_bootstrap.cc:654] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.478698 10462 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.528069 10601 raft_consensus.cc:493] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.545884 10462 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.549247 10462 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:33.521250 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:33.532804 10615 log.cc:927] Time spent T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:05:33.531891 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:33.554344  9783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:33.554605 10011 raft_consensus.cc:2468] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 0.
I20260709 13:05:33.531651  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:33.555898  9786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:33.551514  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:33.528611 10057 ts_tablet_manager.cc:1434] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.178s	user 0.004s	sys 0.005s
W20260709 13:05:33.557446  9901 leader_election.cc:343] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:33.558202  9901 leader_election.cc:343] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.558621 10057 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:33.561920 10061 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:33.528211 10577 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:33.569824  9754 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
W20260709 13:05:33.572486  9904 leader_election.cc:343] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.575011  9904 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:33.577363  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 58489.8us
I20260709 13:05:33.557941  9829 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:33.583578  9904 leader_election.cc:343] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.589478 10057 tablet_bootstrap.cc:654] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.585356 10577 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:33.605216  9978 leader_election.cc:343] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.606348 10061 tablet_bootstrap.cc:654] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:33.546173 10601 raft_consensus.cc:515] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.588645 10594 raft_consensus.cc:2804] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:33.586051 10572 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:33.646965  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 59645.8us
I20260709 13:05:33.649262  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:33.629144 10601 leader_election.cc:290] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:33.641220 10594 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.662456 10594 raft_consensus.cc:3060] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.682276 10059 tablet_bootstrap.cc:492] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:33.688007 10059 ts_tablet_manager.cc:1403] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.330s	user 0.017s	sys 0.004s
I20260709 13:05:33.692690  9977 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:33.693931 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:33.696141 10207 raft_consensus.cc:2804] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:33.696666 10207 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.697038 10207 raft_consensus.cc:3060] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.702343 10058 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: No bootstrap required, opened a new log
I20260709 13:05:33.703408 10058 ts_tablet_manager.cc:1403] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: Time spent bootstrapping tablet: real 0.280s	user 0.054s	sys 0.025s
I20260709 13:05:33.709767 10007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:33.713559 10601 consensus_queue.cc:1048] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:33.724995  9754 leader_election.cc:304] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:33.730404 10207 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.681779  9903 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:33.756465 10207 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:33.767807 10058 raft_consensus.cc:359] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.768999 10058 raft_consensus.cc:385] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.769593 10058 raft_consensus.cc:740] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.771809 10058 consensus_queue.cc:260] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.746109 10550 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.789409 10058 ts_tablet_manager.cc:1434] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: Time spent starting tablet: real 0.086s	user 0.011s	sys 0.000s
I20260709 13:05:33.789579 10059 raft_consensus.cc:359] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.791579 10059 raft_consensus.cc:385] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.791895 10059 raft_consensus.cc:740] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.757119  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:33.792624 10059 consensus_queue.cc:260] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.803413 10059 ts_tablet_manager.cc:1434] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.001s
I20260709 13:05:33.748502 10551 consensus_queue.cc:1048] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:05:33.807098  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
W20260709 13:05:33.784698  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 117502us
I20260709 13:05:33.812824 10574 raft_consensus.cc:2749] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.817418 10619 raft_consensus.cc:2804] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:33.826061 10619 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.826730 10619 raft_consensus.cc:3060] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:33.828833  9978 leader_election.cc:343] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.787015 10550 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:33.820832 10061 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
W20260709 13:05:33.836992  9975 consensus_peers.cc:597] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:33.842159 10550 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:33.837122 10061 ts_tablet_manager.cc:1403] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.276s	user 0.025s	sys 0.016s
I20260709 13:05:33.847173 10061 raft_consensus.cc:359] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:33.838747  9903 leader_election.cc:343] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.848970  9903 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:33.851048 10462 raft_consensus.cc:2749] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.853754 10594 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:33.863879 10061 raft_consensus.cc:385] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:33.865087 10061 raft_consensus.cc:740] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:33.865885 10061 consensus_queue.cc:260] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:33.879694 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:33.884508  9933 raft_consensus.cc:1275] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:33.886155 10011 raft_consensus.cc:3060] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.856236 10594 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:33.890085  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:33.895735 10059 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:33.895912  9861 raft_consensus.cc:3060] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.899484  9864 raft_consensus.cc:3060] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.903533  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:33.915186 10061 ts_tablet_manager.cc:1434] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.004s
W20260709 13:05:33.915912  9978 leader_election.cc:343] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.922622  9789 raft_consensus.cc:3060] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.923063  9685 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33610 (request call id 62) took 1300 ms. Trace:
I20260709 13:05:33.923543  9685 rpcz_store.cc:276] 0709 13:05:32.622612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.628591 (+  5979us) service_pool.cc:224] Handling call
0709 13:05:33.922947 (+1294356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12925,"reported_tablets":4}
W20260709 13:05:33.920697  9755 leader_election.cc:343] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:33.941210 10061 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:33.955858  9683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33634 (request call id 59) took 1066 ms. Trace:
I20260709 13:05:33.957751  9684 catalog_manager.cc:5697] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.970332  9684 catalog_manager.cc:5697] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:33.993839  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:33.994926  9929 raft_consensus.cc:2468] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:34.003712  9977 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:34.005429 10057 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:34.006160 10531 raft_consensus.cc:2804] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:34.006814 10531 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.007155 10531 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:33.913825 10574 consensus_queue.cc:1048] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 13:05:34.031420 10011 raft_consensus.cc:2468] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:34.036571 10554 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.006263 10057 ts_tablet_manager.cc:1403] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.448s	user 0.026s	sys 0.045s
I20260709 13:05:34.050496  9754 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:34.060560 10061 tablet_bootstrap.cc:654] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.062364 10582 raft_consensus.cc:2804] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.065979 10582 raft_consensus.cc:697] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:34.067235 10582 consensus_queue.cc:237] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.044531 10554 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:34.093122 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:34.093479 10554 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:34.093155  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:34.097843  9754 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:34.099560 10594 raft_consensus.cc:2749] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:33.956454  9683 rpcz_store.cc:276] 0709 13:05:32.904207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.908509 (+  4302us) service_pool.cc:224] Handling call
0709 13:05:33.955776 (+1047267us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:33.419400 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:33.420102 (+   702us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:33.420139 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:05:33.420161 (+    22us) tablet.cc:662] Decoding operations
0709 13:05:33.420580 (+   419us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:33.420681 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:33.438053 (+ 17372us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:33.446873 (+  8820us) tablet.cc:701] Row locks acquired
0709 13:05:33.448638 (+  1765us) write_op.cc:260] PREPARE: finished
0709 13:05:33.474421 (+ 25783us) write_op.cc:270] Start()
0709 13:05:33.486176 (+ 11755us) write_op.cc:276] Timestamp: P: 1783602333472187 usec, L: 0
0709 13:05:33.486219 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:05:33.487019 (+   800us) log.cc:844] Serialized 540 byte log entry
0709 13:05:33.715801 (+228782us) op_driver.cc:464] REPLICATION: finished
0709 13:05:33.861161 (+145360us) write_op.cc:301] APPLY: starting
0709 13:05:33.861299 (+   138us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:33.864156 (+  2857us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:33.865160 (+  1004us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:33.876256 (+ 11096us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:33.881546 (+  5290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:33.881606 (+    60us) write_op.cc:312] APPLY: finished
0709 13:05:33.881852 (+   246us) log.cc:844] Serialized 22 byte log entry
0709 13:05:33.886046 (+  4194us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:33.886265 (+   219us) write_op.cc:454] Released schema lock
0709 13:05:33.886321 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:33.886944 (+   623us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:33.886984 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17301,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":145036,"apply.run_cpu_time_us":2835,"apply.run_wall_time_us":25739,"num_ops":1,"prepare.queue_time_us":119451,"prepare.run_cpu_time_us":22117,"prepare.run_wall_time_us":171468,"raft.queue_time_us":113450,"raft.run_cpu_time_us":38573,"raft.run_wall_time_us":86647,"replication_time_us":222071,"thread_start_us":360545,"threads_started":4,"wal-append.queue_time_us":103369}]]}
I20260709 13:05:34.044986 10531 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:34.146191 10057 raft_consensus.cc:359] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.150882 10057 raft_consensus.cc:385] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.151298 10057 raft_consensus.cc:740] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.154066 10057 consensus_queue.cc:260] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.164804 10057 ts_tablet_manager.cc:1434] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.124s	user 0.010s	sys 0.000s
I20260709 13:05:34.193181 10531 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:34.194223  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:34.195972  9978 leader_election.cc:343] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.196915 10059 tablet_bootstrap.cc:654] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.197520 10057 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:34.200966 10619 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:34.213696  9930 raft_consensus.cc:2393] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:34.226518 10007 raft_consensus.cc:1275] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:34.226801 10061 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:34.227402 10061 ts_tablet_manager.cc:1403] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.287s	user 0.030s	sys 0.036s
I20260709 13:05:34.244583 10617 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.245488 10617 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:34.223428  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:34.269749  9929 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.273857 10619 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:34.253122 10589 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.274803 10589 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.277645 10589 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:34.260893 10554 consensus_queue.cc:1048] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:05:34.268064 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:34.285801  9977 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:34.296458  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:34.299278  9904 leader_election.cc:343] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.295892  9684 catalog_manager.cc:5697] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } }
I20260709 13:05:34.260490 10617 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:34.308197 10061 raft_consensus.cc:359] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:34.297950 10587 raft_consensus.cc:2749] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.314714 10061 raft_consensus.cc:385] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.316943 10061 raft_consensus.cc:740] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.320127  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
W20260709 13:05:34.327853  9975 leader_election.cc:343] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.323939 10061 consensus_queue.cc:260] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:34.342000  9789 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.346980  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:34.365593 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:34.368494 10011 raft_consensus.cc:2393] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
W20260709 13:05:34.379137  9901 leader_election.cc:343] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:34.384323  9903 leader_election.cc:343] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.384984  9903 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:34.386673  9903 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:34.390216 10595 raft_consensus.cc:2749] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.391259  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:34.391628 10595 raft_consensus.cc:2749] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.422819 10057 tablet_bootstrap.cc:654] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.457836  9929 raft_consensus.cc:2468] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:34.470211  9977 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:34.473668 10572 raft_consensus.cc:493] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:34.474726 10572 raft_consensus.cc:515] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:34.477080  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 136743us
W20260709 13:05:34.484709  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 61104.5us
I20260709 13:05:34.485607 10572 leader_election.cc:290] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:34.486460 10061 ts_tablet_manager.cc:1434] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.259s	user 0.009s	sys 0.002s
I20260709 13:05:34.487804 10061 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:34.487866 10587 raft_consensus.cc:2804] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:34.488672 10587 raft_consensus.cc:697] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:34.489655 10587 consensus_queue.cc:237] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:34.493372  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 112830us
W20260709 13:05:34.512919  9978 leader_election.cc:343] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:34.513731  9978 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:34.516786 10469 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:34.526674  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:34.529965 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17aba5ba2fe46a49bec25310d428a26" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:34.543426 10059 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:34.569991 10059 ts_tablet_manager.cc:1403] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.694s	user 0.030s	sys 0.036s
W20260709 13:05:34.545993 10643 log.cc:927] Time spent T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
W20260709 13:05:34.546806  9755 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:34.550572  9829 leader_election.cc:304] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:34.563482 10057 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:34.592132 10057 ts_tablet_manager.cc:1403] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.410s	user 0.007s	sys 0.012s
I20260709 13:05:34.564041  9684 catalog_manager.cc:5697] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 0 to 2, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } }
I20260709 13:05:34.543782 10061 tablet_bootstrap.cc:654] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.594086 10572 raft_consensus.cc:2749] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:34.600185 10057 raft_consensus.cc:359] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:34.601933 10059 raft_consensus.cc:359] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:34.609104 10057 raft_consensus.cc:385] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.609225 10059 raft_consensus.cc:385] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.609498 10057 raft_consensus.cc:740] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.609596 10059 raft_consensus.cc:740] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.602332 10582 consensus_queue.cc:1048] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:05:34.610634 10057 consensus_queue.cc:260] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:34.625568 10057 ts_tablet_manager.cc:1434] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:05:34.627079 10057 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: Bootstrap starting.
I20260709 13:05:34.632864  9683 catalog_manager.cc:5697] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.610460 10059 consensus_queue.cc:260] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:34.648646 10057 tablet_bootstrap.cc:654] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.663825 10059 ts_tablet_manager.cc:1434] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.003s
I20260709 13:05:34.674470 10059 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:34.723171  9864 raft_consensus.cc:3060] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:34.741186 10059 tablet_bootstrap.cc:654] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:34.780535 10061 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:34.781473 10061 ts_tablet_manager.cc:1403] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.294s	user 0.019s	sys 0.008s
I20260709 13:05:34.786530 10061 raft_consensus.cc:359] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.787735 10061 raft_consensus.cc:385] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.795006 10061 raft_consensus.cc:740] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.796245 10061 consensus_queue.cc:260] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.801030 10061 ts_tablet_manager.cc:1434] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.001s
I20260709 13:05:34.802963 10061 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:34.863647  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 101713us
I20260709 13:05:34.889426 10059 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:34.891165 10059 ts_tablet_manager.cc:1403] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.217s	user 0.043s	sys 0.023s
W20260709 13:05:34.863212  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 99940.9us
W20260709 13:05:34.916618  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:34.958554 10531 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.962831 10059 raft_consensus.cc:359] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:34.964303 10059 raft_consensus.cc:385] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:34.999907 10059 raft_consensus.cc:740] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:34.989041 10503 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.006181 10503 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.012823 10570 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.013911 10570 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:35.015194 10059 consensus_queue.cc:260] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:35.017904 10570 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:35.035794  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 112964us
I20260709 13:05:35.047900 10061 tablet_bootstrap.cc:654] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:35.056630  9978 consensus_peers.cc:597] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:34.960690 10531 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.094034  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:35.097154  9789 raft_consensus.cc:2468] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:35.101841  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:35.099913 10057 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: No bootstrap required, opened a new log
I20260709 13:05:35.104807 10531 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:35.105702 10057 ts_tablet_manager.cc:1403] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: Time spent bootstrapping tablet: real 0.479s	user 0.017s	sys 0.001s
I20260709 13:05:35.108650 10644 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.109596 10644 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:35.111274 10059 ts_tablet_manager.cc:1434] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.219s	user 0.006s	sys 0.012s
I20260709 13:05:35.117481 10503 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:35.098922 10548 consensus_queue.cc:1048] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:05:35.100718  9684 catalog_manager.cc:5697] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:35.126529  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:35.134785  9930 raft_consensus.cc:2468] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:35.136894 10059 tablet_bootstrap.cc:492] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:35.135408 10644 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:35.145730 10057 raft_consensus.cc:359] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.147578 10057 raft_consensus.cc:385] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.148000 10057 raft_consensus.cc:740] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Initialized, Role: FOLLOWER
W20260709 13:05:35.151592  9978 leader_election.cc:343] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.155300  9975 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:35.161458 10548 raft_consensus.cc:2804] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.162147 10548 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.162673 10548 raft_consensus.cc:3060] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.173813  9754 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:35.176364 10195 raft_consensus.cc:2804] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.177740 10195 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.178284 10195 raft_consensus.cc:3060] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.176318  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:35.179780  9930 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:35.149226 10057 consensus_queue.cc:260] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:35.218722  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 63698us
I20260709 13:05:35.229333 10061 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:35.230538 10061 ts_tablet_manager.cc:1403] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.428s	user 0.012s	sys 0.015s
I20260709 13:05:35.240625  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:35.253059 10057 ts_tablet_manager.cc:1434] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: Time spent starting tablet: real 0.145s	user 0.012s	sys 0.000s
I20260709 13:05:35.255884  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:35.263063  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:35.266837 10195 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.282692 10195 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:35.270828 10548 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:35.285411 10061 raft_consensus.cc:359] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.286070  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:35.302332 10061 raft_consensus.cc:385] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.302533  9931 raft_consensus.cc:3060] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.302817 10061 raft_consensus.cc:740] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.304165 10061 consensus_queue.cc:260] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.286983 10572 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.281992 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:35.295869  9977 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:35.311214 10572 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:35.312433  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 52224.8us
I20260709 13:05:35.313632 10551 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.314164 10551 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.314513 10551 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.314975 10572 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:35.315439 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:35.320701 10061 ts_tablet_manager.cc:1434] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.088s	user 0.012s	sys 0.000s
I20260709 13:05:35.323048  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:35.325618 10061 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:35.326401  9827 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:35.333778 10009 raft_consensus.cc:2393] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:35.336138 10548 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:35.347772  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:35.350415  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:35.352753 10595 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.353611 10595 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:35.336714  9975 leader_election.cc:343] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.359931 10595 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:35.361577 10572 raft_consensus.cc:2749] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.362679  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:35.366899  9904 leader_election.cc:343] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:35.373376  9978 leader_election.cc:343] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.380501 10644 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.381435 10644 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.397027 10061 tablet_bootstrap.cc:654] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.398837  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:35.399770  9933 raft_consensus.cc:2468] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:35.401612  9754 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:35.402693 10484 raft_consensus.cc:2804] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.403192 10484 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.403105  9931 raft_consensus.cc:2468] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
W20260709 13:05:35.407862  9755 leader_election.cc:343] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.403867 10484 raft_consensus.cc:3060] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.413450  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:35.408768 10059 tablet_bootstrap.cc:700] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: Time spent opening tablet: real 0.130s	user 0.001s	sys 0.001s
I20260709 13:05:35.395643 10551 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.404554 10644 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:35.421066  9755 leader_election.cc:343] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.423491  9787 raft_consensus.cc:3060] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.436969  9754 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:35.440742 10548 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.441587 10376 raft_consensus.cc:2804] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.441447 10548 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.441982 10551 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:35.442217 10376 raft_consensus.cc:697] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:35.443944 10548 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:35.450143  9787 raft_consensus.cc:2468] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:35.451797  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:35.453545  9938 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.443529 10376 consensus_queue.cc:237] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.452723 10059 tablet_bootstrap.cc:654] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.422276  9755 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:35.472539 10195 raft_consensus.cc:2749] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:35.472752  9755 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 51744.8us
W20260709 13:05:35.474057  9755 leader_election.cc:343] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.477387  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:35.479149  9931 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:35.480675  9977 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:35.482497 10548 raft_consensus.cc:2804] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.483115 10548 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.483611 10548 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.509676  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:35.528241  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:35.531685  9859 raft_consensus.cc:3060] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.533030  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:35.539844 10061 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:35.541819 10061 ts_tablet_manager.cc:1403] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.219s	user 0.024s	sys 0.005s
I20260709 13:05:35.545089  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:35.527850  9938 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:35.547195 10061 raft_consensus.cc:359] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.548321 10061 raft_consensus.cc:385] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.548709  9977 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:35.549759 10469 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.550408 10469 raft_consensus.cc:697] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:35.551584 10469 consensus_queue.cc:237] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.548734 10061 raft_consensus.cc:740] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.554168 10061 consensus_queue.cc:260] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:35.563227  9755 leader_election.cc:343] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.565467  9975 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:35.566541 10531 raft_consensus.cc:2804] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.567390 10531 raft_consensus.cc:697] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:35.569260 10531 consensus_queue.cc:237] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:35.571571 10059 tablet_bootstrap.cc:492] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:35.577749 10061 ts_tablet_manager.cc:1434] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:05:35.591542 10059 ts_tablet_manager.cc:1403] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.462s	user 0.019s	sys 0.017s
W20260709 13:05:35.599596  9975 leader_election.cc:343] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:35.607388  9975 leader_election.cc:343] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.610774 10061 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:35.657164 10059 raft_consensus.cc:359] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:35.658640 10059 raft_consensus.cc:385] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.659226 10059 raft_consensus.cc:740] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.661011 10059 consensus_queue.cc:260] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:35.697528 10582 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.705649 10582 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.708604 10582 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:35.711336 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:35.712985  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:35.713896  9938 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:35.715691  9754 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:35.717696 10604 raft_consensus.cc:2804] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:35.718458 10604 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.719168 10604 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.724118 10059 ts_tablet_manager.cc:1434] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.130s	user 0.010s	sys 0.000s
I20260709 13:05:35.725818 10059 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:35.730623 10484 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.738386  9785 raft_consensus.cc:2393] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:35.742178 10061 tablet_bootstrap.cc:654] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.748720  9684 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33634 (request call id 61) took 1103 ms. Trace:
I20260709 13:05:35.749328  9684 rpcz_store.cc:276] 0709 13:05:34.645183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.645458 (+   275us) service_pool.cc:224] Handling call
0709 13:05:35.748619 (+1103161us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:35.224928 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:35.225718 (+   790us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:35.225777 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:05:35.225814 (+    37us) tablet.cc:662] Decoding operations
0709 13:05:35.226335 (+   521us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:35.226459 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:35.226516 (+    57us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:35.226859 (+   343us) tablet.cc:701] Row locks acquired
0709 13:05:35.226902 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:05:35.227077 (+   175us) write_op.cc:270] Start()
0709 13:05:35.227246 (+   169us) write_op.cc:276] Timestamp: P: 1783602335227008 usec, L: 0
0709 13:05:35.227296 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:05:35.228327 (+  1031us) log.cc:844] Serialized 540 byte log entry
0709 13:05:35.305501 (+ 77174us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b5400961520. stack: 00007fc830c4f343 00007fc830c4f240 00007fc8305de1d4 00007fc83a31c304 00007fc83a31c299 00007fc833cd8090 00007fc833ce56b8 00007fc833ce52fa 00007fc833d69965 00007fc833d69e09 00007fc8377a4581 00007fc8377a3d58 00007fc8377a89f1 00007fc8377a89a9 00007fc8377a8939 00007fc8377a8901
0709 13:05:35.486030 (+180529us) op_driver.cc:464] REPLICATION: finished
0709 13:05:35.709295 (+223265us) write_op.cc:301] APPLY: starting
0709 13:05:35.723232 (+ 13937us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:35.724207 (+   975us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:35.724254 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:35.726341 (+  2087us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:35.726630 (+   289us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:35.726720 (+    90us) write_op.cc:312] APPLY: finished
0709 13:05:35.727057 (+   337us) log.cc:844] Serialized 22 byte log entry
0709 13:05:35.730239 (+  3182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.730344 (+   105us) write_op.cc:454] Released schema lock
0709 13:05:35.730417 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:35.733468 (+  3051us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:35.733505 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":118362,"apply.run_cpu_time_us":2745,"apply.run_wall_time_us":52761,"num_ops":1,"prepare.queue_time_us":34176,"prepare.run_cpu_time_us":8844,"prepare.run_wall_time_us":114577,"replication_time_us":258493,"spinlock_wait_cycles":3480832,"thread_start_us":94313,"threads_started":1}]]}
I20260709 13:05:35.754928  9901 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:35.766048 10462 raft_consensus.cc:3060] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.781267 10484 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:35.783330  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:35.783330  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:35.786091  9755 leader_election.cc:343] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:35.789707 10604 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.791594 10059 tablet_bootstrap.cc:654] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.794117 10462 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.800544 10548 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.803663 10009 raft_consensus.cc:2393] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:35.810432  9938 raft_consensus.cc:2468] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:35.813004  9754 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:35.814517 10484 raft_consensus.cc:2804] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.815366 10484 raft_consensus.cc:697] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:35.816411 10484 consensus_queue.cc:237] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.818068  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
I20260709 13:05:35.819038  9785 raft_consensus.cc:2393] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 1: Already voted for candidate e8769fa7061d433bb90a9411c71fa0df in this term.
I20260709 13:05:35.822345  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:35.823371  9930 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.823590 10548 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:35.862077 10059 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:35.862803 10059 ts_tablet_manager.cc:1403] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.010s
I20260709 13:05:35.867097 10061 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:35.867918 10061 ts_tablet_manager.cc:1403] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.264s	user 0.018s	sys 0.007s
I20260709 13:05:35.872921 10604 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:35.874244  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:35.876464 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:35.877496 10009 raft_consensus.cc:2393] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e8769fa7061d433bb90a9411c71fa0df in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:35.878769 10059 raft_consensus.cc:359] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.879696 10059 raft_consensus.cc:385] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.879992 10059 raft_consensus.cc:740] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.878839  9754 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:35.882372 10059 consensus_queue.cc:260] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:35.885005 10059 ts_tablet_manager.cc:1434] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:05:35.883157 10061 raft_consensus.cc:359] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.889142 10061 raft_consensus.cc:385] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:35.889557 10061 raft_consensus.cc:740] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:35.889606 10059 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:35.890372 10484 raft_consensus.cc:2749] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:35.892803  9857 raft_consensus.cc:3060] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:35.900627 10550 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.901391 10550 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:35.901517 10059 tablet_bootstrap.cc:654] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:35.906703 10550 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:35.907719  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:35.908075  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:35.908752  9859 raft_consensus.cc:2468] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 0.
I20260709 13:05:35.894440  9685 catalog_manager.cc:5697] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:35.889796  9930 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:35.925200  9977 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:35.926837 10548 raft_consensus.cc:2804] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:35.927445 10548 raft_consensus.cc:697] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:35.928431 10548 consensus_queue.cc:237] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.911669 10061 consensus_queue.cc:260] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:35.908984  9933 raft_consensus.cc:2468] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:35.913569  9978 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:35.934643 10061 ts_tablet_manager.cc:1434] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
I20260709 13:05:35.935266 10531 raft_consensus.cc:2804] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:35.914786 10604 consensus_queue.cc:1048] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:35.935806 10531 raft_consensus.cc:493] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.936276 10531 raft_consensus.cc:3060] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.952061 10061 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:35.976773 10531 raft_consensus.cc:515] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:35.991677 10531 leader_election.cc:290] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:35.997694  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:35.998813  9933 raft_consensus.cc:3060] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:36.001631  9978 consensus_peers.cc:597] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:36.019662 10059 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:36.043648 10059 ts_tablet_manager.cc:1403] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.154s	user 0.020s	sys 0.018s
I20260709 13:05:36.031141  9933 raft_consensus.cc:2468] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:36.033417 10594 consensus_queue.cc:1048] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:05:36.058660 10059 raft_consensus.cc:359] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.061902  9977 leader_election.cc:304] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:36.072084 10059 raft_consensus.cc:385] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.072510 10059 raft_consensus.cc:740] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.040242 10531 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.073356 10531 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.073411 10059 consensus_queue.cc:260] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.076278 10531 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:36.040982  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5673ca542b1745a38fdc5b65d4357b42" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:36.068477  9783 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.077932  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:36.078069  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:36.082048  9978 leader_election.cc:343] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.083309 10059 ts_tablet_manager.cc:1434] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
W20260709 13:05:36.085526  9975 leader_election.cc:343] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.086898  9975 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:36.090608 10548 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.107059 10668 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.108027 10668 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.112154  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:36.113104 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:36.112558 10668 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:36.072619 10550 raft_consensus.cc:2804] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:36.116187 10550 raft_consensus.cc:697] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:36.117632 10550 consensus_queue.cc:237] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:36.122527  9857 raft_consensus.cc:3060] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.124780  9785 raft_consensus.cc:2393] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 1: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:36.011266  9683 catalog_manager.cc:5697] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.103106 10061 tablet_bootstrap.cc:654] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.128927  9827 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:36.130604 10668 raft_consensus.cc:3060] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.107003 10059 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:36.128973  9683 catalog_manager.cc:5697] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.155359 10376 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.157968 10531 consensus_queue.cc:1048] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.156126 10376 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:36.161566 10548 consensus_queue.cc:1048] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.165447  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:36.201282  9933 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:36.198500 10668 raft_consensus.cc:2749] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:36.238958  9734 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:36.245694  9754 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:36.253722 10604 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:36.254211 10604 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.254623 10604 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.274410  9857 raft_consensus.cc:2468] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:36.248651 10376 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:36.276023 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:36.249652 10550 consensus_queue.cc:1048] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.289701 10059 tablet_bootstrap.cc:654] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.289911 10604 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:36.299224  9783 raft_consensus.cc:1240] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Rejecting Update request from peer 5e087ae2ce3f417093313edd26b8b3ee for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:05:36.308529  9685 catalog_manager.cc:5697] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.324998 10061 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: No bootstrap required, opened a new log
I20260709 13:05:36.325757 10061 ts_tablet_manager.cc:1403] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.374s	user 0.016s	sys 0.000s
I20260709 13:05:36.336459 10604 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:36.342857 10582 consensus_queue.cc:1048] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.344901 10061 raft_consensus.cc:359] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.345883 10061 raft_consensus.cc:385] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.346203 10061 raft_consensus.cc:740] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.347132 10061 consensus_queue.cc:260] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.351859 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:36.352856 10551 consensus_queue.cc:1059] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.353530 10061 ts_tablet_manager.cc:1434] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:05:36.354349 10550 raft_consensus.cc:3055] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:36.354789 10550 raft_consensus.cc:740] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:36.355865 10550 consensus_queue.cc:260] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:36.357722 10550 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:36.387843  9755 consensus_peers.cc:597] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:36.403858  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:36.442363  9754 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:36.443765 10644 raft_consensus.cc:2749] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:36.446869 10059 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:36.448035 10059 ts_tablet_manager.cc:1403] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.341s	user 0.017s	sys 0.006s
I20260709 13:05:36.452487 10059 raft_consensus.cc:359] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:36.470861 10604 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.471900 10604 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.475040 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:36.474210 10604 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:36.476079 10008 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 0.
I20260709 13:05:36.478075 10059 raft_consensus.cc:385] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.478902 10059 raft_consensus.cc:740] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.479844  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:36.480162  9754 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:36.480373 10059 consensus_queue.cc:260] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:36.489408 10531 consensus_queue.cc:1048] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.491569 10376 raft_consensus.cc:2804] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:36.492018 10376 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.492422 10376 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.497545 10668 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.498435 10668 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:36.501474 10668 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:36.501703  9755 leader_election.cc:343] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.504905  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:36.511945  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8769fa7061d433bb90a9411c71fa0df" is_pre_election: true
I20260709 13:05:36.513579 10550 consensus_queue.cc:1048] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.542183 10059 ts_tablet_manager.cc:1434] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 13:05:36.523358  9829 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:36.552098 10668 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:36.543862 10059 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:36.544982  9685 catalog_manager.cc:5697] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 2, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:36.565958  9685 catalog_manager.cc:5697] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 1 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.574260  9938 raft_consensus.cc:1275] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:36.584813 10570 consensus_queue.cc:1048] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.630824 10376 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.645202 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:36.646060 10008 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.659554 10376 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:36.661455  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
W20260709 13:05:36.663335  9755 leader_election.cc:343] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:36.644057 10059 tablet_bootstrap.cc:654] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.672199 10008 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 1.
I20260709 13:05:36.696967  9857 raft_consensus.cc:1275] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:36.699077 10548 consensus_queue.cc:1048] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:36.705216  9754 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:36.707026 10376 raft_consensus.cc:2804] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:36.707791 10376 raft_consensus.cc:697] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:36.709002 10376 consensus_queue.cc:237] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.754675 10059 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:36.755422 10059 ts_tablet_manager.cc:1403] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.212s	user 0.019s	sys 0.021s
I20260709 13:05:36.762666 10059 raft_consensus.cc:359] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.763644 10059 raft_consensus.cc:385] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.765079 10059 raft_consensus.cc:740] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.765944 10059 consensus_queue.cc:260] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:36.768638 10059 ts_tablet_manager.cc:1434] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:05:36.769837 10059 tablet_bootstrap.cc:492] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d: Bootstrap starting.
I20260709 13:05:36.787429  9683 catalog_manager.cc:5697] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 0 to 1, leader changed from <none> to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 1 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:36.791826 10059 tablet_bootstrap.cc:654] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:36.828047  9864 raft_consensus.cc:3060] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:36.863613 10059 tablet_bootstrap.cc:492] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d: No bootstrap required, opened a new log
I20260709 13:05:36.877322 10059 ts_tablet_manager.cc:1403] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d: Time spent bootstrapping tablet: real 0.108s	user 0.014s	sys 0.006s
I20260709 13:05:36.884516 10059 raft_consensus.cc:359] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:36.916781 10059 raft_consensus.cc:385] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:36.917130 10059 raft_consensus.cc:740] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Initialized, Role: FOLLOWER
I20260709 13:05:36.917945 10059 consensus_queue.cc:260] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:36.920924 10059 ts_tablet_manager.cc:1434] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:05:36.971508   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:36.981802  9975 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:05:36.996655  9975 consensus_peers.cc:597] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.019032  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.033352  9975 consensus_peers.cc:597] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.053696  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.198686  9856 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.231684 10668 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.232307 10668 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:37.235512  9975 consensus_peers.cc:597] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.236325  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.237536 10668 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:37.241540 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:37.248940 10008 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:37.259174  9829 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:37.229871 10604 consensus_queue.cc:1048] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:05:37.266081 10376 consensus_queue.cc:1048] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:37.269451 10668 raft_consensus.cc:2804] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:37.300699  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.303442  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.286365 10668 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.301901 10699 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.313505 10699 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:05:37.306314  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.306928 10668 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:37.322356  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 92688.1us
W20260709 13:05:37.320084  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.373596  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.379873 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:37.375962  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:37.379252 10699 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:37.400038  9829 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
W20260709 13:05:37.393891  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.403051 10700 raft_consensus.cc:2749] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:37.439335  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 314356us
W20260709 13:05:37.440251  9975 consensus_peers.cc:597] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.454145  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 69286.1us
I20260709 13:05:37.489909  9859 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:37.490111 10668 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:37.547207  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.571609  9856 raft_consensus.cc:1240] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Rejecting Update request from peer e8769fa7061d433bb90a9411c71fa0df for earlier term 1. Current term is 2. Ops: [1.1-1.1]
W20260709 13:05:37.629470  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 69905.7us
W20260709 13:05:37.651609  9827 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.663419 10668 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:37.689836  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.690478  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 57186.2us
I20260709 13:05:37.699120   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:37.700489   431 tablet_replica.cc:333] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.701238 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:37.701635   431 raft_consensus.cc:2243] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:37.692412  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.714444  9901 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.715238  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.715771  9901 consensus_peers.cc:597] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.703874   431 raft_consensus.cc:2272] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.719811   431 tablet_replica.cc:333] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.728924   431 raft_consensus.cc:2243] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.730474   431 raft_consensus.cc:2272] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:37.732455  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:37.741973  9827 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:37.742267  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.742707  9827 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:37.744915 10700 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:37.758889  9975 consensus_peers.cc:597] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:37.763981   431 tablet_replica.cc:333] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.765116   431 raft_consensus.cc:2243] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.765671   431 raft_consensus.cc:2272] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.788501 10712 consensus_queue.cc:1059] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:37.789857 10594 raft_consensus.cc:3055] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:37.790345 10594 raft_consensus.cc:740] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:37.791360 10594 consensus_queue.cc:260] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:37.793291   431 tablet_replica.cc:333] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.792934 10594 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:37.794646   431 raft_consensus.cc:2243] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.795483   431 raft_consensus.cc:2272] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.813292 10565 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:37.814078 10565 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:37.909605  9754 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 102379us
I20260709 13:05:37.913448   431 tablet_replica.cc:333] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.914583   431 raft_consensus.cc:2243] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.909142 10565 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:37.928473  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 124939us
W20260709 13:05:37.909142  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 103237us
W20260709 13:05:37.950987  9901 leader_election.cc:336] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:37.916064   431 raft_consensus.cc:2272] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.956753   431 tablet_replica.cc:333] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.958873 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:37.959929 10008 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:37.969099  9903 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:37.971072 10706 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:37.971578 10706 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:37.971984 10706 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:37.973371   431 raft_consensus.cc:2243] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:37.974947   431 raft_consensus.cc:2272] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.987071   431 tablet_replica.cc:333] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:37.988842   431 raft_consensus.cc:2243] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:37.989697   431 raft_consensus.cc:2272] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:37.999107   431 tablet_replica.cc:333] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.000836   431 raft_consensus.cc:2243] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.002411   431 raft_consensus.cc:2272] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.006685   431 tablet_replica.cc:333] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.007794   431 raft_consensus.cc:2243] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.008251 10706 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.009230   431 raft_consensus.cc:2272] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.012228 10698 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.022838 10698 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:38.028256 10698 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:38.017184 10712 raft_consensus.cc:493] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.029080 10712 raft_consensus.cc:515] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:38.031769 10712 leader_election.cc:290] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:38.032667 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.035034   431 tablet_replica.cc:333] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.034709  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.036641 10698 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.037237 10698 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.036227   431 raft_consensus.cc:2243] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.042197   431 raft_consensus.cc:2272] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.048863 10604 raft_consensus.cc:493] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:38.049571 10604 raft_consensus.cc:515] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:38.040114 10698 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.052304 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:38.052996   431 tablet_replica.cc:333] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
W20260709 13:05:38.059065  9901 leader_election.cc:336] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.056612 10706 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:38.054041 10604 leader_election.cc:290] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:38.060787  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.061100  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.065531  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.066587 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.062799   431 raft_consensus.cc:2243] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.062798 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47be166313094f0fbe17bde042f05dfe" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.068675   431 raft_consensus.cc:2272] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.069926  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d775d5d3993743689257e4509a059770" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.071753  9755 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:38.071955 10008 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:38.072569   431 tablet_replica.cc:333] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.073568  9903 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:38.073678 10712 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.074594 10706 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.075960 10706 raft_consensus.cc:697] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:38.077169  9754 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.077003 10706 consensus_queue.cc:237] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.078509 10712 raft_consensus.cc:2749] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.081780  9755 leader_election.cc:304] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:38.083810 10712 raft_consensus.cc:2749] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:38.085810  9754 leader_election.cc:304] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:38.073757   431 raft_consensus.cc:2243] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.088589 10594 raft_consensus.cc:2749] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.089215   431 raft_consensus.cc:2272] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.114768   431 tablet_replica.cc:333] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.123312  9685 catalog_manager.cc:5697] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.125694 10698 raft_consensus.cc:493] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.126583 10698 raft_consensus.cc:515] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.132972 10698 leader_election.cc:290] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.133567 10698 raft_consensus.cc:493] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:38.137499  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.139312 10604 raft_consensus.cc:493] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.134213 10698 raft_consensus.cc:515] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:38.140229 10604 raft_consensus.cc:515] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.143136 10698 leader_election.cc:290] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:38.143589 10604 leader_election.cc:290] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.116051   431 raft_consensus.cc:2243] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.147581   431 raft_consensus.cc:2272] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.147122  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.153895  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b253ddcc7d6046c6abdd94aec212612c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.154909   431 tablet_replica.cc:333] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.158257   431 raft_consensus.cc:2243] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.159181   431 raft_consensus.cc:2272] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.161688 10604 raft_consensus.cc:493] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:38.162313 10604 raft_consensus.cc:515] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.163676  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.168156 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "210c05e6675c4ba78ca28841a5315a07" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.165417  9754 leader_election.cc:304] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:38.169411 10698 raft_consensus.cc:2749] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.170266 10604 leader_election.cc:290] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.169693  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.174321   431 tablet_replica.cc:333] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.175863   431 raft_consensus.cc:2243] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.176738   431 raft_consensus.cc:2272] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.181653   431 tablet_replica.cc:333] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.182863 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f8b3f8e9ff4e1ab10c104b1a3c2da6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.189237   431 raft_consensus.cc:2243] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.192852   431 raft_consensus.cc:2272] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.197306   431 tablet_replica.cc:333] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.198547   431 raft_consensus.cc:2243] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.199301   431 raft_consensus.cc:2272] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.202419  9754 leader_election.cc:304] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
W20260709 13:05:38.203425  9754 raft_consensus.cc:2686] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:05:38.207619   431 tablet_replica.cc:333] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.209152   431 raft_consensus.cc:2243] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.207790 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.209746  9754 leader_election.cc:304] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.210654 10604 raft_consensus.cc:2749] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.215396   431 raft_consensus.cc:2272] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.217270 10604 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:38.218144 10604 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:38.220374   431 tablet_replica.cc:333] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.222314  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.221799 10604 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:38.229405  9754 leader_election.cc:304] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.230908  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.241328 10604 raft_consensus.cc:2749] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.243520 10604 raft_consensus.cc:493] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.244393 10604 raft_consensus.cc:515] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.251714 10604 leader_election.cc:290] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.253923  9754 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:38.260782 10698 raft_consensus.cc:2749] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:38.262634 10604 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.263646 10604 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.269922   431 raft_consensus.cc:2243] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.276023 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.278729  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.278939   431 raft_consensus.cc:2272] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.284775 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.274619  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.287602  9754 leader_election.cc:304] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.277109 10604 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.300526 10698 raft_consensus.cc:2749] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.304858   431 tablet_replica.cc:333] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.306268   431 raft_consensus.cc:2243] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.307019   431 raft_consensus.cc:2272] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.308911  9754 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.322933 10604 raft_consensus.cc:2749] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.327988   431 tablet_replica.cc:333] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.333124 10594 raft_consensus.cc:493] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.333995 10594 raft_consensus.cc:515] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.340085 10604 raft_consensus.cc:493] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:38.342221  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.340922 10604 raft_consensus.cc:515] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:38.345989 10604 leader_election.cc:290] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.350052 10594 leader_election.cc:290] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.351047  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.370286 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.356185 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9625fd7e9ed046158552e5252f1ee3db" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.365259   431 raft_consensus.cc:2243] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.373311   431 raft_consensus.cc:2272] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.377907   431 tablet_replica.cc:333] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.378994   431 raft_consensus.cc:2243] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.381067   431 raft_consensus.cc:2272] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.384660 10594 raft_consensus.cc:493] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.385703 10594 raft_consensus.cc:515] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:38.389008  9754 leader_election.cc:304] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.390545 10604 raft_consensus.cc:2749] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.391130  9754 leader_election.cc:304] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.392175 10604 raft_consensus.cc:2749] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.392971 10594 leader_election.cc:290] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:38.393347 10604 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:38.394769 10604 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:38.397550 10604 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:38.393795 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.401216  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.404955 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.407078  9754 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:38.408254 10594 raft_consensus.cc:2749] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.410687   431 tablet_replica.cc:333] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.418883   431 raft_consensus.cc:2243] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.418642  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.419765   431 raft_consensus.cc:2272] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.423627   431 tablet_replica.cc:333] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.425539  9755 leader_election.cc:304] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:38.428534 10594 raft_consensus.cc:2749] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.444762   431 raft_consensus.cc:2243] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.448206   431 raft_consensus.cc:2272] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.454089   431 tablet_replica.cc:333] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.455103   431 raft_consensus.cc:2243] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.464921 10594 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:38.465831 10594 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:38.468426 10594 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:38.463953   431 raft_consensus.cc:2272] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.470258 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:38.476205   431 tablet_replica.cc:333] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.477025 10012 raft_consensus.cc:1275] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:38.477793   431 raft_consensus.cc:2243] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.478650   431 raft_consensus.cc:2272] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.480396 10565 consensus_queue.cc:1048] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:38.476897  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:38.484457   431 tablet_replica.cc:333] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.485595   431 raft_consensus.cc:2243] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.487430   431 raft_consensus.cc:2272] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.487300  9755 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:38.488799 10594 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:38.495446   431 tablet_replica.cc:333] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.496860   431 raft_consensus.cc:2243] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.497635   431 raft_consensus.cc:2272] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.502624   431 tablet_replica.cc:333] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.503592   431 raft_consensus.cc:2243] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.504984   431 raft_consensus.cc:2272] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.509389   431 tablet_replica.cc:333] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.529672   431 raft_consensus.cc:2243] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.531241   431 raft_consensus.cc:2272] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:38.538144  9901 consensus_peers.cc:597] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:38.541268   431 tablet_replica.cc:333] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.542623   431 raft_consensus.cc:2243] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.543264   431 raft_consensus.cc:2272] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.549356   431 tablet_replica.cc:333] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.550426   431 raft_consensus.cc:2243] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.551878   431 raft_consensus.cc:2272] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.555769   431 tablet_replica.cc:333] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.557428   431 raft_consensus.cc:2243] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.558321   431 raft_consensus.cc:2272] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.564103   431 tablet_replica.cc:333] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.565217   431 raft_consensus.cc:2243] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.565868   431 raft_consensus.cc:2272] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.570199   431 tablet_replica.cc:333] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.571475   431 raft_consensus.cc:2243] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.573172   431 raft_consensus.cc:2272] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.579401   431 tablet_replica.cc:333] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.580731   431 raft_consensus.cc:2243] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.582316   431 raft_consensus.cc:2272] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.588482   431 tablet_replica.cc:333] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.606707   431 raft_consensus.cc:2243] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.609066   431 raft_consensus.cc:2272] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.612759   431 tablet_replica.cc:333] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.614027   431 raft_consensus.cc:2243] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.626425   431 raft_consensus.cc:2272] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.630808   431 tablet_replica.cc:333] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.632082   431 raft_consensus.cc:2243] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.632990   431 raft_consensus.cc:2272] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.637297   431 tablet_replica.cc:333] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.642979   431 raft_consensus.cc:2243] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.644941   431 raft_consensus.cc:2272] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.650610   431 tablet_replica.cc:333] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.651865   431 raft_consensus.cc:2243] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:38.653235   431 raft_consensus.cc:2272] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.656644   431 tablet_replica.cc:333] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.657672   431 raft_consensus.cc:2243] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.658610   431 raft_consensus.cc:2272] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.663146   431 tablet_replica.cc:333] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.664300   431 raft_consensus.cc:2243] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.665958   431 raft_consensus.cc:2272] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.674265   431 tablet_replica.cc:333] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.675763   431 raft_consensus.cc:2243] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.676615   431 raft_consensus.cc:2272] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.713375   431 tablet_replica.cc:333] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: stopping tablet replica
I20260709 13:05:38.714700   431 raft_consensus.cc:2243] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:38.715675   431 raft_consensus.cc:2272] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:38.876962 10565 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:38.877784 10565 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:38.881862 10699 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:38.882738 10699 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:38.889134 10699 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:38.890395  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:38.891345  9933 raft_consensus.cc:2468] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:38.894385  9829 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:38.898850 10699 raft_consensus.cc:2804] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:38.899538 10699 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:38.900185 10699 raft_consensus.cc:3060] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.910665 10565 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:38.913092  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:38.922909  9901 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.923026 10699 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:38.925105  9857 raft_consensus.cc:2393] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:38.928653  9904 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:38.930794 10706 raft_consensus.cc:3060] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.938961  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:38.940313 10699 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:38.947942  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 56102.8us
W20260709 13:05:38.958770  9827 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:05:38.972549  9827 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:05:38.973641 10706 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:38.985144  9933 raft_consensus.cc:2468] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:38.987766  9829 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:38.989490 10699 raft_consensus.cc:2804] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.997393 10699 raft_consensus.cc:697] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:38.998924 10699 consensus_queue.cc:237] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.025501  9685 catalog_manager.cc:5697] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.087122   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:05:39.083006 10565 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.088958 10565 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.092243 10565 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.094286  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:39.129305  9901 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.130134  9901 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.137547 10565 raft_consensus.cc:2749] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.159754 10699 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.160890 10699 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.161247 10707 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.162012 10707 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.166951 10707 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.169965  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:39.170959  9933 raft_consensus.cc:2468] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:39.172629  9829 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:39.173624 10729 raft_consensus.cc:2804] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.174042 10729 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.174135 10699 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.174520 10729 raft_consensus.cc:3060] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.183183  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.184628  9859 raft_consensus.cc:2393] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
W20260709 13:05:39.189034  9975 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.189621  9975 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.189754 10699 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.190601 10699 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.191293 10729 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.196483 10548 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.198887 10699 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.202595 10729 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.207821 10548 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.209570  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb3e93354824df58589f250b6264d54" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:39.224961  9938 raft_consensus.cc:3060] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.259894 10704 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.260777 10704 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.265997 10705 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.266799 10705 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:39.196717  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:05:39.213460  9827 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.272686  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 59278.1us
I20260709 13:05:39.274631 10565 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.275542 10565 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.277129 10699 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.277786 10699 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.278946 10729 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.279731 10729 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.280674 10699 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.281301 10699 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.281780 10699 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.282537 10729 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.284658 10699 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:39.285250  9975 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.303208  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:39.304919  9930 raft_consensus.cc:2468] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:39.306473 10565 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.307989  9829 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:39.312309 10699 raft_consensus.cc:2804] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.312968 10699 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.313448 10699 raft_consensus.cc:3060] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.308435  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.301805  9938 raft_consensus.cc:2468] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:39.330862  9829 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
W20260709 13:05:39.336315  9901 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.337725 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.333796 10729 raft_consensus.cc:2804] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.341380 10729 raft_consensus.cc:697] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:39.333895 10704 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.355683 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.335934 10705 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.364743  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:05:39.368472  9975 leader_election.cc:336] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.369455 10729 consensus_queue.cc:237] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.375067  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.384682  9977 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:39.390974  9827 leader_election.cc:336] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.391453 10707 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.391577  9827 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:39.392803  9827 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.398275  9978 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.400543 10739 raft_consensus.cc:2749] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.417006 10740 raft_consensus.cc:2749] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.422485 10699 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.427685  9859 raft_consensus.cc:2393] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:39.429035 10699 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.430852  9904 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.431845  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:39.432545  9930 raft_consensus.cc:3060] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.433223 10462 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.433799 10462 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:39.436198 10462 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.437233 10565 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.439483 10565 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.444041 10740 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.444794 10740 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:39.434628  9827 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.449139  9827 leader_election.cc:336] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.449584  9827 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.450462 10699 raft_consensus.cc:2749] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.448166 10740 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:05:39.451519  9827 leader_election.cc:336] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.451902  9827 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.453048 10740 raft_consensus.cc:2749] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:39.455984  9827 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.458721  9827 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.462330  9901 leader_election.cc:336] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.463864  9901 leader_election.cc:336] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.467427 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.468477  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.479588  9859 raft_consensus.cc:2468] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:39.473510 10565 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.481890 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.483104 10011 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:39.483604  9829 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.484918 10729 raft_consensus.cc:2749] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.485064  9903 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.486543 10741 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.486848 10548 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.487035 10741 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.487646 10548 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.486742  9904 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.491181 10565 raft_consensus.cc:2804] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.491626 10565 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.492040 10565 raft_consensus.cc:3060] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.502558 10548 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.502730  9685 catalog_manager.cc:5697] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.488602 10741 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.504958 10729 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.505817 10729 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.500823  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.509323 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.509603  9856 raft_consensus.cc:2468] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:39.510160 10011 raft_consensus.cc:2468] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:39.511904  9829 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:39.511586 10565 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.512840  9978 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:39.512953 10740 raft_consensus.cc:2804] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.513582 10740 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.514017 10740 raft_consensus.cc:3060] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.515014  9975 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.516196  9827 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.517103 10729 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.518131 10745 raft_consensus.cc:2804] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.518623 10745 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.519112 10745 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.519811  9975 consensus_peers.cc:597] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.526902 10707 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.530151 10707 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.527944 10729 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.529364  9931 raft_consensus.cc:1275] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.532773 10729 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.534188 10699 consensus_queue.cc:1048] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:05:39.535192 10548 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.547286 10548 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.536795 10707 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.551501 10548 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.542867 10462 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.552645 10462 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.555083 10462 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:39.557744  9827 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.558367  9975 leader_election.cc:336] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.540319  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.562508  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.563314  9856 raft_consensus.cc:2468] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:39.542448  9901 leader_election.cc:336] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.566605  9978 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.537338 10729 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.539955  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:39.568673  9931 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:39.568874  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.569630  9859 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:39.570715 10704 raft_consensus.cc:2804] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.571128 10704 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.571486 10704 raft_consensus.cc:3060] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.572746  9827 consensus_peers.cc:597] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:39.533139  9930 raft_consensus.cc:2468] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:39.579758 10706 raft_consensus.cc:2749] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.584684  9904 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:39.585296 10565 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.586066 10706 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.584676  9829 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.590775 10706 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.595438  9829 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:39.594879  9975 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.598508  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:39.599706  9859 raft_consensus.cc:3060] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.593314 10729 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.602701 10729 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.603425 10729 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.609856 10743 raft_consensus.cc:2804] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.610806 10743 raft_consensus.cc:697] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:39.592801 10706 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.594846 10745 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:39.626927  9901 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.544445 10740 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.612130 10743 consensus_queue.cc:237] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.629000 10746 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.631400 10746 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.633387 10741 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:39.633639 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d6f1cfb3ffd4a418cafdda5af0b219d" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:39.634641 10011 raft_consensus.cc:3060] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.635393 10746 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.635780 10741 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.636749  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:39.637483  9856 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.637845 10746 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.638037 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:39.639111 10012 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.638571 10746 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.642940  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.643544  9859 raft_consensus.cc:2468] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:39.632774 10745 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.598491  9975 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.655355 10548 raft_consensus.cc:2749] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.648659 10741 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
W20260709 13:05:39.657517  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 62715.2us
I20260709 13:05:39.652275 10704 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.651855 10699 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.665094 10699 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.666086 10011 raft_consensus.cc:2468] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:39.667773 10699 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.670212 10742 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.670956 10742 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.673627  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:39.676796 10742 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.670228 10746 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.678611  9859 raft_consensus.cc:3060] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.698951  9827 leader_election.cc:336] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.679622 10704 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.680464 10741 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.700457 10699 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.701121 10699 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.681514 10740 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.702739 10740 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.703290 10740 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.703956 10699 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.705577 10741 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:39.673804  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.684185 10729 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:39.708380  9975 consensus_peers.cc:597] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.705410  9856 raft_consensus.cc:2468] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:39.695052  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.709682 10729 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.686182 10706 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.718593 10706 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.687585 10012 raft_consensus.cc:2468] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:05:39.715696  9864 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:39.723927  9901 consensus_peers.cc:597] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.724462  9978 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
W20260709 13:05:39.724622  9901 leader_election.cc:336] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.725225  9901 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.728006  9859 raft_consensus.cc:2468] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:39.728806 10707 raft_consensus.cc:2804] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.715180 10704 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.729307 10707 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.729725 10707 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.729525 10704 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.737390 10704 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.726284  9903 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:39.741031 10741 raft_consensus.cc:2804] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:39.742050 10741 raft_consensus.cc:697] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 3 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:39.735145  9685 catalog_manager.cc:5697] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:39.742908  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:39.743546  9975 consensus_peers.cc:597] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:39.744201  9975 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.743117 10741 consensus_queue.cc:237] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:39.745980  9827 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.711387 10740 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.748611 10707 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.750049 10706 raft_consensus.cc:2749] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.741228  9978 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
W20260709 13:05:39.753402  9901 leader_election.cc:336] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.754168 10707 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.754379 10704 raft_consensus.cc:2804] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.755249 10704 raft_consensus.cc:697] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:39.756381 10704 consensus_queue.cc:237] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:39.760527  9975 leader_election.cc:336] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.763337 10729 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
W20260709 13:05:39.763669  9901 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.764371  9901 leader_election.cc:336] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.764055 10729 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:39.764470  9975 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.767146 10729 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:39.754196  9978 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.768373 10707 raft_consensus.cc:2804] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.768910 10707 raft_consensus.cc:697] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:39.770951 10707 consensus_queue.cc:237] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:39.784849  9827 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.784965  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:39.785815  9827 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.785923  9859 raft_consensus.cc:2468] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:39.786391  9827 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:39.792909  9904 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.794445 10706 raft_consensus.cc:2804] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.795074 10706 raft_consensus.cc:697] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:39.796159 10706 consensus_queue.cc:237] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.803973  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:39.816224  9685 catalog_manager.cc:5697] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 2 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.828521 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.829764 10012 raft_consensus.cc:2393] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 2: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:05:39.833099   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 13:05:39.833168  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:39.828132 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.835551  9938 raft_consensus.cc:2393] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 2: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:39.836408 10011 raft_consensus.cc:2410] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:05:39.839758 10706 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.840456 10706 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.842665 10706 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:39.845542 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:39.846261 10011 raft_consensus.cc:2468] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:39.846837  9829 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.848287 10749 raft_consensus.cc:2749] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.856532 10750 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.855383  9685 catalog_manager.cc:5697] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.857275 10750 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:39.764885  9901 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.875447  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 110570us
I20260709 13:05:39.881928  9829 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.892590 10749 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.893285 10749 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.900686 10740 raft_consensus.cc:2749] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:39.896041 10749 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:39.903532  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:39.908655  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.913504  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.914402  9864 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:39.915032  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:39.915910  9859 raft_consensus.cc:2393] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 2: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
I20260709 13:05:39.916939  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:39.917721  9856 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.913714 10706 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.920325 10706 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.923702 10706 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:39.926710  9975 leader_election.cc:336] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.927711  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:39.928123  9975 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.930552 10756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:39.946491  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.933557 10749 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.969521  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:39.970710  9859 raft_consensus.cc:2468] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:39.962389 10750 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.948071 10742 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.962683 10749 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.977329 10742 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:39.979833 10749 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:39.981469  9864 raft_consensus.cc:2468] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
I20260709 13:05:39.954264 10740 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.984965 10740 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:39.987036  9978 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:39.989351 10745 raft_consensus.cc:2804] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:39.989984 10745 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:39.990392 10745 raft_consensus.cc:3060] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:39.995850  9901 leader_election.cc:336] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:39.996920  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:39.997103  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:39.997586  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:39.964841  9829 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:40.005851  9901 leader_election.cc:336] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.006685  9901 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.006934  9829 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.008118 10740 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:40.007102 10757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:40.006060 10743 raft_consensus.cc:2804] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.009990  9829 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.012267  9829 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:40.016037 10743 raft_consensus.cc:697] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:40.017431 10743 consensus_queue.cc:237] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.016198 10729 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:40.023452  9975 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.028873 10729 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.029560 10729 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.030298  9856 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:40.032157 10740 raft_consensus.cc:2749] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.031963 10729 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:40.033355 10750 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.033942 10750 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.031963 10749 raft_consensus.cc:2804] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.036041 10749 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.036373 10750 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:40.036767 10749 raft_consensus.cc:3060] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.036655  9685 catalog_manager.cc:5697] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:40.039260  9827 leader_election.cc:336] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.048236  9930 raft_consensus.cc:1275] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.048462 10742 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:40.061509  9904 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.073019 10707 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.074106 10707 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:40.080407 10759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:40.078083  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.085182  9904 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:40.087129  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 43498.4us
I20260709 13:05:40.090351  9978 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.097579  9904 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.100507  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:40.102252 10462 raft_consensus.cc:2749] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.101887  9864 raft_consensus.cc:2468] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
I20260709 13:05:40.103710 10548 raft_consensus.cc:2804] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.104318 10548 raft_consensus.cc:697] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:40.080646 10707 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:40.087615 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:40.105762 10548 consensus_queue.cc:237] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.093245 10565 raft_consensus.cc:2749] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:40.107962 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:40.087805 10706 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.105676 10746 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.092433 10750 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.122944  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:40.138123 10706 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.143622 10750 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.126838 10011 raft_consensus.cc:2468] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:40.153901 10706 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:40.137508 10008 raft_consensus.cc:2468] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:40.154124  9863 raft_consensus.cc:2468] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 1.
W20260709 13:05:40.154606  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.137797 10746 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.155208 10746 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:40.162714  9975 consensus_peers.cc:597] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:40.163718  9975 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.165742  9829 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
W20260709 13:05:40.170863  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 69033.6us
I20260709 13:05:40.170078  9829 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:40.173128 10743 raft_consensus.cc:2804] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.173732 10743 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.174289 10743 raft_consensus.cc:3060] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.180222  9904 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.178776   431 server_base.cc:1061] running on GCE node
I20260709 13:05:40.171795 10740 raft_consensus.cc:2804] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.189749 10746 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.214565 10706 raft_consensus.cc:2804] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.215415 10706 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.215528 10746 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:40.216812 10749 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.221848 10749 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:40.225313 10750 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:40.190376 10740 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.231817 10740 raft_consensus.cc:3060] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.216055 10706 raft_consensus.cc:3060] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:40.238569  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 73913.5us
I20260709 13:05:40.248986 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:40.250074 10008 raft_consensus.cc:2468] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
W20260709 13:05:40.250041  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.251578   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:40.252002   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:40.252272 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82999b9832894f40bb6dbe02e2feb1df" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:40.253082 10011 raft_consensus.cc:3060] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.254742  9829 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
W20260709 13:05:40.251541  9901 leader_election.cc:336] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.268712  9827 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.252249   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602340252231 us; error 0 us; skew 500 ppm
W20260709 13:05:40.269728  9827 leader_election.cc:336] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.270869  9827 consensus_peers.cc:597] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:40.271571  9827 leader_election.cc:336] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.272809  9827 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.219765 10745 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.270926 10729 raft_consensus.cc:2804] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.286549 10729 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.287060 10729 raft_consensus.cc:3060] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.279289   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:40.285181  9685 catalog_manager.cc:5697] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:40.288321  9827 consensus_peers.cc:597] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:40.289117  9827 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.290047  9827 leader_election.cc:336] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.290597  9827 leader_election.cc:336] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.291085  9827 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.285722 10745 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:40.297026  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 206807us
I20260709 13:05:40.303469  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:40.304201  9857 raft_consensus.cc:3060] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:40.321879  9901 leader_election.cc:336] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.339084  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.344286 10008 raft_consensus.cc:1275] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:40.347043 10770 consensus_queue.cc:1048] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:40.357704  9975 consensus_peers.cc:597] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.357460 10729 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.368243 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:40.369104 10012 raft_consensus.cc:3060] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.343717 10740 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.373183 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1131589c5bd9480d838a7967131195ca" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:40.336149 10743 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.374096 10009 raft_consensus.cc:3060] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:40.374433  9827 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.377244  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:40.377529  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:40.381685  9827 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.385391  9975 leader_election.cc:336] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.386852 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d9519bce7451aaee1e04623f7fddb" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
W20260709 13:05:40.387156  9827 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:40.388620  9975 consensus_peers.cc:597] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.392272 10006 raft_consensus.cc:3060] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.395741 10740 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:40.397015 10729 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:40.387249 10011 raft_consensus.cc:2468] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
W20260709 13:05:40.401554  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 221425us
I20260709 13:05:40.402846  9904 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.407224  9829 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.408481 10012 raft_consensus.cc:2468] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:40.410259  9829 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.375808 10743 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:40.422847 10706 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.424582 10750 raft_consensus.cc:2804] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:40.425037  9975 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.425313 10750 raft_consensus.cc:697] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:40.427109 10750 consensus_queue.cc:237] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:40.425654  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:40.428773  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:40.414347 10772 raft_consensus.cc:2804] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.430527 10772 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.431156 10772 raft_consensus.cc:3060] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.417171  9857 raft_consensus.cc:2468] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:05:40.434212 10729 raft_consensus.cc:2804] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.435236 10729 raft_consensus.cc:697] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:40.437232  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:40.437438  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:40.438220  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:40.439064  9931 raft_consensus.cc:2468] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:40.438889 10729 consensus_queue.cc:237] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.440932  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:40.439064  9938 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:40.441885  9933 raft_consensus.cc:2468] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 1.
I20260709 13:05:40.445112 10006 raft_consensus.cc:2468] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:40.446439  9932 raft_consensus.cc:1275] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.449057 10009 raft_consensus.cc:2468] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:40.455468  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:40.450820  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:40.456682  9857 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:40.461874  9855 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.451421 10740 consensus_queue.cc:1048] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:05:40.467547  9978 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.470781  9829 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.473531 10740 raft_consensus.cc:2804] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.476492 10774 raft_consensus.cc:2764] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:05:40.477504 10740 raft_consensus.cc:697] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:40.478790 10740 consensus_queue.cc:237] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.483311  9978 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:40.485544  9901 consensus_peers.cc:597] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:40.485929  9827 consensus_peers.cc:597] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.486639  9829 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.490837  9829 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.492041  9829 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.492448 10749 raft_consensus.cc:2804] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.487169 10766 raft_consensus.cc:2764] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:05:40.497843 10750 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:40.496168 10743 raft_consensus.cc:2804] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.500826 10743 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.501343 10743 raft_consensus.cc:3060] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:40.502657  9901 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:40.507763  9978 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.513401   431 webserver.cc:533] Webserver started at http://127.0.107.193:44477/ using document root <none> and password file <none>
I20260709 13:05:40.516185  9904 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.523626 10462 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:40.524395 10462 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.525537 10462 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.498411 10750 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.527447 10750 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.525190  9857 raft_consensus.cc:1275] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.513407 10766 raft_consensus.cc:2804] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.559490  9855 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:05:40.547704   431 fs_manager.cc:362] Metadata directory not provided
W20260709 13:05:40.562615  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 70667.3us
I20260709 13:05:40.564747  9829 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.538146  9685 catalog_manager.cc:5697] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.562512   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:40.577196 10749 raft_consensus.cc:697] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:40.577001  9686 catalog_manager.cc:5697] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.588774  9686 catalog_manager.cc:5697] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.568990 10766 raft_consensus.cc:697] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:40.603494 10462 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.578749 10749 consensus_queue.cc:237] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:40.601877 10783 raft_consensus.cc:2804] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:40.613665 10783 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:05:40.614261 10783 raft_consensus.cc:3060] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.625752 10462 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:40.635039  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 55607.8us
I20260709 13:05:40.637748  9864 raft_consensus.cc:1275] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.606467 10706 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:40.651120  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:40.651916  9859 raft_consensus.cc:3060] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.635654 10729 consensus_queue.cc:1048] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:05:40.660425  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:40.661216  9904 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:40.671420 10462 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.671999 10462 raft_consensus.cc:697] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
W20260709 13:05:40.672894 10777 log.cc:927] Time spent T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.163s	user 0.000s	sys 0.002s
I20260709 13:05:40.652971  9855 raft_consensus.cc:1275] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.675273 10462 consensus_queue.cc:237] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:40.679889  9901 leader_election.cc:336] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.684557 10766 consensus_queue.cc:237] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:40.690515  9901 leader_election.cc:336] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:40.672590 10770 consensus_queue.cc:1048] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:40.645443 10767 consensus_queue.cc:1048] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:05:40.720392  9859 raft_consensus.cc:2468] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
W20260709 13:05:40.726944  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 118264us
W20260709 13:05:40.732023  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 87589.8us
I20260709 13:05:40.752300 10009 raft_consensus.cc:1275] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:40.790454  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 60569.5us
I20260709 13:05:40.796142 10786 consensus_queue.cc:1048] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
W20260709 13:05:40.799870  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 64753us
I20260709 13:05:40.792711  9904 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:05:40.824257  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 76838.4us
W20260709 13:05:40.826048  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 89682.6us
I20260709 13:05:40.836310 10462 raft_consensus.cc:2804] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.838030 10462 raft_consensus.cc:697] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:40.868305 10548 consensus_queue.cc:1048] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:40.856962 10462 consensus_queue.cc:237] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.872604   431 fs_manager.cc:714] Time spent opening directory manager: real 0.250s	user 0.018s	sys 0.007s
I20260709 13:05:40.904608 10772 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:40.918038  9930 raft_consensus.cc:2393] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 2: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
W20260709 13:05:40.945282 10787 log.cc:927] Time spent T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:05:40.907392 10743 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:40.968706  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 58953.3us
I20260709 13:05:40.997584 10772 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:41.043809 10797 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.044829 10797 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:41.047493 10797 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:41.051002 10006 raft_consensus.cc:1275] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.051954 10009 raft_consensus.cc:1275] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.058182 10009 raft_consensus.cc:1275] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.062623 10807 consensus_queue.cc:1048] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.063421 10785 consensus_queue.cc:1048] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.065502 10785 consensus_queue.cc:1048] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.014757 10783 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:41.073065 10783 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:41.074652 10750 consensus_meta.cc:287] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: Time spent flushing consensus metadata: real 0.547s	user 0.016s	sys 0.000s
I20260709 13:05:41.075263 10750 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:41.078053 10750 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:41.079689  9863 raft_consensus.cc:2393] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 3: Already voted for candidate 24c6a28138294e46bf6b3d5ba324046d in this term.
W20260709 13:05:41.129392  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 170316us
I20260709 13:05:41.129921 10812 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:41.140618   431 fs_manager.cc:730] Time spent opening block manager: real 0.170s	user 0.010s	sys 0.016s
I20260709 13:05:41.149253  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:41.150285  9861 raft_consensus.cc:3060] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.141247   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "e8769fa7061d433bb90a9411c71fa0df"
format_stamp: "Formatted at 2026-07-09 13:05:15 on dist-test-slave-ftjc"
I20260709 13:05:41.177489   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:41.192571  9859 raft_consensus.cc:1275] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:41.219451 10795 log.cc:927] Time spent T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: Append to log took a long time: real 0.276s	user 0.000s	sys 0.008s
I20260709 13:05:41.225720  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:41.226963  9857 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
W20260709 13:05:41.267794  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 113115us
I20260709 13:05:41.300942  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:41.302264  9938 raft_consensus.cc:2393] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 24c6a28138294e46bf6b3d5ba324046d in current term 3: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:05:41.309916  9904 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:41.334517  9829 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:41.339021  9857 raft_consensus.cc:1275] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.334978  9978 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:41.365844 10761 consensus_queue.cc:1048] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:05:41.378779 10772 consensus_queue.cc:1048] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.379868  9861 raft_consensus.cc:2468] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
W20260709 13:05:41.379245  9827 consensus_peers.cc:597] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.380510 10820 raft_consensus.cc:2749] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:41.381559 10822 log.cc:927] Time spent T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:05:41.367745 10766 raft_consensus.cc:2804] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:41.384063 10743 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:41.401243  9827 leader_election.cc:336] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:41.410946  9827 consensus_peers.cc:597] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.411713  9827 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:41.412091  9827 leader_election.cc:336] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:41.412319  9827 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:41.413897  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bdd04f0ff44f2e87f76f36a7c5e540" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:41.414902  9930 raft_consensus.cc:3060] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.415151 10750 raft_consensus.cc:2749] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.410168 10766 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.416682 10766 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.403920 10779 raft_consensus.cc:2749] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:41.430038  9975 consensus_peers.cc:597] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.430684  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 97260.7us
I20260709 13:05:41.431793  9857 raft_consensus.cc:1275] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.436785  9904 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:41.449893  9932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:41.450790  9932 raft_consensus.cc:3060] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:41.454046  9827 consensus_peers.cc:597] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.454908  9827 consensus_peers.cc:597] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.450531 10796 consensus_queue.cc:1048] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.474315 10008 raft_consensus.cc:1275] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:41.476202  9901 leader_election.cc:336] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:05:41.489506  9975 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:41.511772  9686 catalog_manager.cc:5697] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 2 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.478111 10770 raft_consensus.cc:2804] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.516255  9930 raft_consensus.cc:2468] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:41.521790  9829 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:41.517237  9686 catalog_manager.cc:5697] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } }
I20260709 13:05:41.542934  9686 catalog_manager.cc:5697] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } }
I20260709 13:05:41.554658 10802 raft_consensus.cc:2804] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.555454 10802 raft_consensus.cc:697] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
W20260709 13:05:41.561280  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 44480.6us
I20260709 13:05:41.488464 10786 consensus_queue.cc:1048] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
W20260709 13:05:41.577975  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 60840us
I20260709 13:05:41.523491  9932 raft_consensus.cc:2468] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:41.550123 10770 raft_consensus.cc:697] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:41.621958  9829 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:41.627316 10749 raft_consensus.cc:2804] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.633714 10770 consensus_queue.cc:237] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:41.693001 10837 log.cc:927] Time spent T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:05:41.626874 10802 consensus_queue.cc:237] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:41.719051  9901 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.679323 10749 raft_consensus.cc:697] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
W20260709 13:05:41.721822  9901 consensus_peers.cc:597] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:41.731666  9975 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:41.750113  9901 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:41.763118 10749 consensus_queue.cc:237] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:41.766741   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:41.789479   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:41.776647 10766 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:41.797417 10766 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:41.827730  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:41.828642  9857 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.848712  9683 catalog_manager.cc:5697] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.851095  9683 catalog_manager.cc:5697] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.821334   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:41.903191  9857 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:05:41.919826  9857 raft_consensus.cc:1275] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.932997 10746 consensus_queue.cc:1048] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:41.936654  9975 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:41.955123  9685 catalog_manager.cc:5697] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.962134  9978 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:41.969442 10742 raft_consensus.cc:2804] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.970068 10742 raft_consensus.cc:697] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:41.981209 10742 consensus_queue.cc:237] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:41.983969  9975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111) [suppressed 312 similar messages]
W20260709 13:05:41.995350  9827 consensus_peers.cc:597] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:42.039079  9683 catalog_manager.cc:5697] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.072665 10856 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:05:42.125819  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:42.136162 10802 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.137017 10802 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:42.140672 10802 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:42.144626  9930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:42.144344  9685 catalog_manager.cc:5697] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.145454  9930 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:42.148685  9685 catalog_manager.cc:5697] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.151079  9829 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:42.152768 10802 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.153232 10802 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.153578 10802 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.171420  9930 raft_consensus.cc:1275] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:42.175294 10786 consensus_queue.cc:1048] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:42.190894  9827 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:42.190241  9683 catalog_manager.cc:5697] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 1 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.195472 10802 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:42.206110  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:42.206925  9931 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.207665 10802 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:05:42.230837  9975 consensus_peers.cc:597] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:42.247743  9931 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 3.
I20260709 13:05:42.250837  9829 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
W20260709 13:05:42.254447  9827 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:42.264788  9857 raft_consensus.cc:1275] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:42.257364 10802 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.274348 10462 consensus_queue.cc:1048] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:42.287006  9827 consensus_peers.cc:597] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.340169  9901 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.344518 10802 raft_consensus.cc:697] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:42.348322 10802 consensus_queue.cc:237] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:42.429910  9857 raft_consensus.cc:1275] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:42.465205  9901 consensus_peers.cc:597] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:42.473618 10707 consensus_queue.cc:1048] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:42.483441  9827 consensus_peers.cc:597] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:42.489073  9935 raft_consensus.cc:1275] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:42.491858 10749 consensus_queue.cc:1048] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:42.497594  9683 catalog_manager.cc:5697] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:42.536418  9975 consensus_peers.cc:597] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:42.549338  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.554126  9975 consensus_peers.cc:597] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.587693  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.623906  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.624683  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.712646  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.785429  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.816104  9975 consensus_peers.cc:597] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.850582  9975 consensus_peers.cc:597] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.853484  9901 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.859011  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:42.952433  9827 consensus_peers.cc:597] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:43.016625  9827 consensus_peers.cc:597] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.018328  9930 raft_consensus.cc:1275] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:43.019749  9975 consensus_peers.cc:597] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:43.021271 10729 consensus_queue.cc:1048] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:43.023461  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.047209  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.074862  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:43.091311 10856 ts_tablet_manager.cc:548] Loading tablet metadata (30/45 complete)
W20260709 13:05:43.124797  9975 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.132055 10765 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.132881 10765 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:43.135349 10765 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:43.141105  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:43.145449  9858 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:05:43.147526  9904 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:43.148926 10462 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:43.149437 10462 raft_consensus.cc:493] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.149772 10462 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:43.171692  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.178333  9901 leader_election.cc:336] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:43.182045 10462 raft_consensus.cc:515] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:43.187438 10462 leader_election.cc:290] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:43.192560  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a693233cb65a40089d314773895f84a0" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:43.193495  9861 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:43.204978  9861 raft_consensus.cc:2468] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 4.
W20260709 13:05:43.207487  9901 leader_election.cc:336] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:05:43.212841  9904 leader_election.cc:304] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:43.215031 10809 raft_consensus.cc:2804] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:43.215713 10809 raft_consensus.cc:697] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 4 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:43.216881 10809 consensus_queue.cc:237] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:43.258878  9975 consensus_peers.cc:597] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.304879   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:05:43.305531   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 1.342s	user 0.018s	sys 0.009s
I20260709 13:05:43.305958   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
W20260709 13:05:43.305017  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:43.311820  9683 catalog_manager.cc:5697] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 2 to 4, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.320905 10856 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:43.348024  9901 consensus_peers.cc:597] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.387444  9827 consensus_peers.cc:597] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.401778 10856 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:43.403080 10856 tablet_bootstrap.cc:492] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:43.403999 10856 ts_tablet_manager.cc:1403] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.083s	user 0.016s	sys 0.004s
I20260709 13:05:43.422115 10856 raft_consensus.cc:359] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:43.423039 10856 raft_consensus.cc:740] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.424081 10856 consensus_queue.cc:260] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:43.434676 10856 ts_tablet_manager.cc:1434] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.030s	user 0.013s	sys 0.003s
I20260709 13:05:43.436703 10856 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:43.454031  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:43.463104 10856 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:43.464762 10856 tablet_bootstrap.cc:492] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:43.465772 10856 ts_tablet_manager.cc:1403] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.011s
I20260709 13:05:43.469102 10856 raft_consensus.cc:359] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:43.469892 10856 raft_consensus.cc:740] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.470778 10856 consensus_queue.cc:260] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:43.473096 10856 ts_tablet_manager.cc:1434] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:43.473961 10856 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:43.499697 10856 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:43.500941 10856 tablet_bootstrap.cc:492] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:43.501731 10856 ts_tablet_manager.cc:1403] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.009s
I20260709 13:05:43.505050 10856 raft_consensus.cc:359] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:43.505818 10856 raft_consensus.cc:740] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.506843 10856 consensus_queue.cc:260] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:43.515409  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.522037  9975 consensus_peers.cc:597] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.529851 10856 ts_tablet_manager.cc:1434] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.003s
W20260709 13:05:43.530520  9827 consensus_peers.cc:597] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.532033 10856 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:43.577513 10856 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:43.578929 10856 tablet_bootstrap.cc:492] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:43.580097 10856 ts_tablet_manager.cc:1403] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.011s
I20260709 13:05:43.589915 10856 raft_consensus.cc:359] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:43.590814 10856 raft_consensus.cc:740] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.591677 10856 consensus_queue.cc:260] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:43.594156 10856 ts_tablet_manager.cc:1434] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:05:43.595394 10856 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:43.610028  9827 consensus_peers.cc:597] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.623313  9827 consensus_peers.cc:597] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.641019  9901 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.654500 10856 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:43.656168 10856 tablet_bootstrap.cc:492] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:43.673992 10856 ts_tablet_manager.cc:1403] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.007s
I20260709 13:05:43.684954 10856 raft_consensus.cc:359] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:43.686226 10856 raft_consensus.cc:740] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.687271 10856 consensus_queue.cc:260] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 3.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:43.721669 10856 ts_tablet_manager.cc:1434] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:05:43.731156 10856 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:43.738446  9858 raft_consensus.cc:1275] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:43.742090 10765 consensus_queue.cc:1048] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:43.791116  9901 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:43.810045 10856 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:43.811471 10856 tablet_bootstrap.cc:492] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:43.812425 10856 ts_tablet_manager.cc:1403] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.082s	user 0.029s	sys 0.004s
I20260709 13:05:43.816025 10856 raft_consensus.cc:359] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:43.816758  9827 consensus_peers.cc:597] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:43.817222 10856 raft_consensus.cc:740] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:43.818378 10856 consensus_queue.cc:260] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:43.821063 10856 ts_tablet_manager.cc:1434] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:43.822561 10856 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:43.823988  9975 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:43.962381  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 50708.4us
W20260709 13:05:43.974730  9901 consensus_peers.cc:597] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:43.975456  9901 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:44.035441 10856 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.037142 10856 tablet_bootstrap.cc:492] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.041463 10856 ts_tablet_manager.cc:1403] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.219s	user 0.029s	sys 0.029s
W20260709 13:05:44.047168  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 77058.7us
I20260709 13:05:44.060776 10856 raft_consensus.cc:359] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.061843 10856 raft_consensus.cc:740] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.062815 10856 consensus_queue.cc:260] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:44.071830  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 47954.6us
I20260709 13:05:44.081887 10856 ts_tablet_manager.cc:1434] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.000s
I20260709 13:05:44.083487 10856 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:44.095259  9901 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:44.163573  9827 consensus_peers.cc:597] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:44.178603 10856 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.179971 10856 tablet_bootstrap.cc:492] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.180903 10856 ts_tablet_manager.cc:1403] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.098s	user 0.025s	sys 0.003s
I20260709 13:05:44.184294 10856 raft_consensus.cc:359] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.185150 10856 raft_consensus.cc:740] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.186519 10856 consensus_queue.cc:260] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.189355 10856 ts_tablet_manager.cc:1434] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:05:44.190392 10856 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:44.237298 10856 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.238786 10856 tablet_bootstrap.cc:492] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.240005 10856 ts_tablet_manager.cc:1403] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.009s
I20260709 13:05:44.244690 10856 raft_consensus.cc:359] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.263765   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:05:44.264436   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.958s	user 0.402s	sys 0.007s
I20260709 13:05:44.286998 10856 raft_consensus.cc:740] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.288444 10856 consensus_queue.cc:260] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.291181 10856 ts_tablet_manager.cc:1434] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:05:44.299659 10856 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:44.392661 10856 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.394203 10856 tablet_bootstrap.cc:492] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.395044 10856 ts_tablet_manager.cc:1403] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.096s	user 0.013s	sys 0.018s
I20260709 13:05:44.423409 10856 raft_consensus.cc:359] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.424664 10856 raft_consensus.cc:740] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.425565 10856 consensus_queue.cc:260] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.432214 10856 ts_tablet_manager.cc:1434] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:05:44.434609 10856 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:44.471678 10856 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.474251 10856 tablet_bootstrap.cc:492] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.475584 10856 ts_tablet_manager.cc:1403] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.006s
I20260709 13:05:44.479090 10856 raft_consensus.cc:359] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.480293 10856 raft_consensus.cc:740] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.481639 10856 consensus_queue.cc:260] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.485255 10856 ts_tablet_manager.cc:1434] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.007s
I20260709 13:05:44.486557 10856 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:44.536417 10856 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.537727 10856 tablet_bootstrap.cc:492] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.538625 10856 ts_tablet_manager.cc:1403] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.012s
I20260709 13:05:44.542752 10856 raft_consensus.cc:359] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.544752 10856 raft_consensus.cc:740] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.545845 10856 consensus_queue.cc:260] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.553786 10856 ts_tablet_manager.cc:1434] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.004s
I20260709 13:05:44.555262 10856 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:44.566722  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:44.567628  9975 consensus_peers.cc:597] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:44.596534  9827 consensus_peers.cc:597] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:44.601050 10856 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.603472 10856 tablet_bootstrap.cc:492] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.605029 10856 ts_tablet_manager.cc:1403] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.050s	user 0.024s	sys 0.003s
I20260709 13:05:44.611768 10856 raft_consensus.cc:359] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.613168 10856 raft_consensus.cc:740] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.625649 10856 consensus_queue.cc:260] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.636034 10856 ts_tablet_manager.cc:1434] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:05:44.640379 10856 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:44.670512 10856 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.671906 10856 tablet_bootstrap.cc:492] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.672802 10856 ts_tablet_manager.cc:1403] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.003s
I20260709 13:05:44.675976 10856 raft_consensus.cc:359] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.679201 10856 raft_consensus.cc:740] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.681993 10856 consensus_queue.cc:260] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:44.691326 10856 ts_tablet_manager.cc:1434] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.004s
I20260709 13:05:44.692885 10856 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:44.713299 10872 raft_consensus.cc:493] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.714027 10872 raft_consensus.cc:515] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:44.745376 10872 leader_election.cc:290] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:44.818360 10856 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:44.826550 10856 tablet_bootstrap.cc:492] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:44.827889 10856 ts_tablet_manager.cc:1403] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.135s	user 0.021s	sys 0.004s
W20260709 13:05:44.849942  9975 consensus_peers.cc:597] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:44.891465 10872 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.892202 10872 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.895100 10872 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:44.942713  9901 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:44.949587 10856 raft_consensus.cc:359] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.950937 10856 raft_consensus.cc:740] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:44.952072 10856 consensus_queue.cc:260] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:44.955161 10856 ts_tablet_manager.cc:1434] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.127s	user 0.004s	sys 0.006s
I20260709 13:05:44.960021 10856 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:44.990885  9975 consensus_peers.cc:597] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:45.002472  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.034240  9827 consensus_peers.cc:597] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:45.084640  9901 consensus_peers.cc:597] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.111341  9827 consensus_peers.cc:597] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:45.136325 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.139005 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.146925  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.152428 10831 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:45.155326 10872 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:45.172494  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.180373  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 330448us
I20260709 13:05:45.193986 10872 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.194765 10872 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:45.199255 10872 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:45.211534 10856 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:45.300410 10856 tablet_bootstrap.cc:492] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:45.289520  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:45.331928 10948 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.338085 10948 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:45.345165 10948 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:45.346208 10948 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.347692 10948 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:45.353282 10948 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:45.354030 10872 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.355027 10872 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:45.380296 10856 ts_tablet_manager.cc:1403] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.421s	user 0.034s	sys 0.004s
W20260709 13:05:45.395607  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 90689.9us
W20260709 13:05:45.396783  9827 consensus_peers.cc:597] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:45.398638  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:45.456420 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.456892 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.458997 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
W20260709 13:05:45.463982  9975 consensus_peers.cc:597] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.436201 10872 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:45.468916  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.481123  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.485384 10856 raft_consensus.cc:359] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:45.487341 10856 raft_consensus.cc:740] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
W20260709 13:05:45.496304  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 143978us
W20260709 13:05:45.521622  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.546389  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:45.555352  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:45.562067  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.582238  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:45.585888 10841 leader_election.cc:304] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:45.587499 10872 raft_consensus.cc:2749] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:45.590199  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.598241  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.591325  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 45343.2us
W20260709 13:05:45.593345  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 44239.5us
I20260709 13:05:45.536195 10856 consensus_queue.cc:260] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:45.637952 10856 ts_tablet_manager.cc:1434] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.257s	user 0.012s	sys 0.003s
W20260709 13:05:45.602478  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.678642 10856 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:45.683117  9901 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.685276 10841 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:45.692994 10872 raft_consensus.cc:2749] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:45.698295  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.706578 10872 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.708088 10872 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:45.713894  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.717118 10872 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:45.718209 10841 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:45.719530 10872 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:45.772281  9975 consensus_peers.cc:597] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.719877 10841 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:45.775689 10872 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:45.778259 10841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 58488.1us
I20260709 13:05:45.784578 10841 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:45.785799 10872 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:45.788242  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:45.791715 10841 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:45.792816 10872 raft_consensus.cc:2749] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:45.810057  9827 consensus_peers.cc:597] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:45.831123 10872 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.831861  9827 consensus_peers.cc:597] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:45.831964 10872 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:45.838101  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.841885 10872 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:45.841904 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.853034 10831 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:45.854451 10948 raft_consensus.cc:2749] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:45.866595 10856 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:45.877648 10856 tablet_bootstrap.cc:492] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:45.883345 10856 ts_tablet_manager.cc:1403] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.244s	user 0.027s	sys 0.007s
I20260709 13:05:45.887333 10856 raft_consensus.cc:359] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:45.888129 10856 raft_consensus.cc:740] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:45.888973 10856 consensus_queue.cc:260] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:45.892952  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.893664  9975 consensus_peers.cc:597] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:45.909034 10856 ts_tablet_manager.cc:1434] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.005s
I20260709 13:05:45.910295 10948 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.911115 10948 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:45.914706  9901 consensus_peers.cc:597] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:45.915787  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.918107 10872 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.920398 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.918987 10872 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:45.924219 10872 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:45.926416 10831 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:45.926533  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.927706 10872 raft_consensus.cc:2749] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:45.928227 10948 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:45.926432 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:45.930449 10831 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:45.931130 10856 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:45.931875 10872 raft_consensus.cc:2749] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:45.933173 10872 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.933887 10872 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:45.944103  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:45.954670  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:45.955297 10872 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:45.957671 10831 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:45.958848 10948 raft_consensus.cc:2749] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:45.975986  9827 consensus_peers.cc:597] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.997758  9975 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.998502  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:45.999117  9975 consensus_peers.cc:597] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:45.999794  9975 consensus_peers.cc:597] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.004124 10948 raft_consensus.cc:493] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.005048 10948 raft_consensus.cc:515] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.008073 10872 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.008773 10948 leader_election.cc:290] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:46.009231 10872 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.009469  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:46.011705 10872 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:46.013661  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:46.013546  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:46.017130 10856 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.017650  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:46.023427 10841 leader_election.cc:304] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:46.026754 10872 raft_consensus.cc:2749] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.023696 10831 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:46.028957 10872 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.021332 10856 tablet_bootstrap.cc:492] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:46.030848 10856 ts_tablet_manager.cc:1403] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.100s	user 0.022s	sys 0.000s
I20260709 13:05:46.035758 10856 raft_consensus.cc:359] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:46.036689 10856 raft_consensus.cc:740] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.052642 10856 consensus_queue.cc:260] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:46.081068 10872 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.082050 10872 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.085858  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:46.087378  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:46.089603 10841 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:46.090492 10948 raft_consensus.cc:2749] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.090888 10872 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:46.078450 10856 ts_tablet_manager.cc:1434] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.006s
I20260709 13:05:46.095945 10856 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:46.113272  9901 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:46.156934  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:46.172390  9827 consensus_peers.cc:597] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.217864 10856 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.219547 10856 tablet_bootstrap.cc:492] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:46.220569 10856 ts_tablet_manager.cc:1403] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.125s	user 0.026s	sys 0.004s
I20260709 13:05:46.225224 10856 raft_consensus.cc:359] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.226305 10856 raft_consensus.cc:740] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.227411 10856 consensus_queue.cc:260] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.232448 10856 ts_tablet_manager.cc:1434] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:05:46.241994 10856 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:46.342542 10856 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:46.340499  9827 consensus_peers.cc:597] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.350962 10856 tablet_bootstrap.cc:492] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
W20260709 13:05:46.351461  9901 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.357872 10856 ts_tablet_manager.cc:1403] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.116s	user 0.025s	sys 0.005s
W20260709 13:05:46.380379  9901 consensus_peers.cc:597] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:46.403859  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:46.392751 10856 raft_consensus.cc:359] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:46.407495 10856 raft_consensus.cc:740] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.413305 10856 consensus_queue.cc:260] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:46.423945 10856 ts_tablet_manager.cc:1434] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.001s
W20260709 13:05:46.434029  9975 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.425921 10856 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:46.468233  9901 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:46.497432  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:46.502755  9901 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.547912 10856 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.557643 10856 tablet_bootstrap.cc:492] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:46.564512 10856 ts_tablet_manager.cc:1403] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.139s	user 0.020s	sys 0.007s
I20260709 13:05:46.572010 10856 raft_consensus.cc:359] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:46.573189 10856 raft_consensus.cc:740] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.575064 10856 consensus_queue.cc:260] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:46.596031 10856 ts_tablet_manager.cc:1434] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:05:46.597795 10856 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:46.645619  9827 consensus_peers.cc:597] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.694672 10856 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.696211 10856 tablet_bootstrap.cc:492] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:46.697181 10856 ts_tablet_manager.cc:1403] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.100s	user 0.026s	sys 0.000s
I20260709 13:05:46.701635 10856 raft_consensus.cc:359] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.702574 10856 raft_consensus.cc:740] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.703459 10856 consensus_queue.cc:260] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.709044 10856 ts_tablet_manager.cc:1434] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:05:46.710225 10856 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:46.738096 10856 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.739425 10856 tablet_bootstrap.cc:492] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:46.740784 10856 ts_tablet_manager.cc:1403] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.008s
I20260709 13:05:46.745347 10856 raft_consensus.cc:359] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:46.746495 10856 raft_consensus.cc:740] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.747426 10856 consensus_queue.cc:260] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:46.750617 10856 ts_tablet_manager.cc:1434] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:46.751716 10856 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:46.755869  9827 consensus_peers.cc:597] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:46.763448 10872 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.769652 10872 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:46.773444 10872 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:46.773990  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:46.775122 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:46.778429 10831 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:46.781770 10872 raft_consensus.cc:2749] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:46.812273 10856 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:46.822392 10856 tablet_bootstrap.cc:492] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:46.823638 10856 ts_tablet_manager.cc:1403] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.010s
I20260709 13:05:46.832928 10856 raft_consensus.cc:359] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.833930 10856 raft_consensus.cc:740] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:46.846686 10856 consensus_queue.cc:260] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:46.853583 10856 ts_tablet_manager.cc:1434] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.005s
I20260709 13:05:46.854898 10856 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:46.870640 10990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:32801 every 8 connection(s)
I20260709 13:05:46.872766   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:32801
I20260709 13:05:47.039535 10856 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:47.071561 10856 tablet_bootstrap.cc:492] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:47.072764 10856 ts_tablet_manager.cc:1403] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.218s	user 0.021s	sys 0.034s
I20260709 13:05:47.077358 10856 raft_consensus.cc:359] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:47.078444 10856 raft_consensus.cc:740] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.079397 10856 consensus_queue.cc:260] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:47.084523  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 79383.8us
I20260709 13:05:47.102074 10856 ts_tablet_manager.cc:1434] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.006s
W20260709 13:05:47.113487  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 99720.4us
I20260709 13:05:47.126653 10957 raft_consensus.cc:3060] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.136555 10856 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:47.149375 10872 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.150243 10872 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:47.176002 10959 raft_consensus.cc:3060] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.186437  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:47.190582 11009 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.167821 10872 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:47.198657 11009 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:47.200618  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 85487.3us
I20260709 13:05:47.229056 10955 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.232271 10952 raft_consensus.cc:3060] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.232662 10954 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.235903 10872 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.236691 10872 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:47.241284  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:47.289680 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:47.315668 10872 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:05:47.354475  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 60400.4us
I20260709 13:05:47.377743 10992 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37093
I20260709 13:05:47.378372 10992 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:47.379585 10992 heartbeater.cc:507] Master 127.0.107.254:37093 requested a full tablet report, sending...
I20260709 13:05:47.393374 11009 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:47.397464 10937 raft_consensus.cc:3060] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.398675 10831 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
W20260709 13:05:47.401736  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 80634.3us
I20260709 13:05:47.401619 10953 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.418448  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:47.430176  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:47.440544 11008 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.417402 11009 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.445760 11009 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:05:47.497108 11014 log.cc:927] Time spent T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df: Append to log took a long time: real 0.152s	user 0.005s	sys 0.000s
I20260709 13:05:47.502966 10943 raft_consensus.cc:3060] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.504179 10872 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.507203 10872 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:47.509951 10872 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:47.510528 10872 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.511036 10872 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:47.513629 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:47.513929 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:47.515391  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:47.516021 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:47.525336 10831 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:05:47.568105  9863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
W20260709 13:05:47.577253  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 45740.6us
I20260709 13:05:47.578153 11009 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:47.579192 10952 raft_consensus.cc:3060] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:47.583199 10872 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:47.586939 10872 raft_consensus.cc:2749] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.628612 10856 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:47.675631  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 59504.1us
I20260709 13:05:47.640165 10942 raft_consensus.cc:3060] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:47.685708  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 129350us
I20260709 13:05:47.662130 10957 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.724107 10856 tablet_bootstrap.cc:492] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:47.739477 10856 ts_tablet_manager.cc:1403] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.607s	user 0.033s	sys 0.018s
W20260709 13:05:47.764330  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 151319us
I20260709 13:05:47.715322 10956 raft_consensus.cc:3060] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:47.782044 10831 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 242570us
I20260709 13:05:47.805066 10856 raft_consensus.cc:359] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:47.807147 10856 raft_consensus.cc:740] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.812951 10856 consensus_queue.cc:260] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:47.813649 11008 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:47.822944 11008 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:47.850126 11013 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:05:47.850896 11013 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:47.852622 10831 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:47.855822 10856 ts_tablet_manager.cc:1434] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.116s	user 0.006s	sys 0.003s
I20260709 13:05:47.857043 10856 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:47.858532 11013 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:47.859678 11009 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:05:47.860325 11009 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:05:47.866807  9827 consensus_peers.cc:597] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:47.873519  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:47.863310 11009 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:47.895977 11008 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:05:47.899004  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 335848us
I20260709 13:05:47.900640 10841 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:47.924265 11032 raft_consensus.cc:2749] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:47.940722 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:47.949776 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:05:47.950369  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:05:47.964273 11030 consensus_queue.cc:1050] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:05:47.969733 11009 raft_consensus.cc:2749] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:47.983588  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:05:47.991044  9827 consensus_peers.cc:597] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:48.008818  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:48.054582 10856 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.055852 10856 tablet_bootstrap.cc:492] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.057562 10856 ts_tablet_manager.cc:1403] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.201s	user 0.027s	sys 0.003s
I20260709 13:05:48.060623 10856 raft_consensus.cc:359] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:48.061431 10856 raft_consensus.cc:740] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.062316 10856 consensus_queue.cc:260] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:48.062927  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:48.063208  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:48.066244  9686 ts_manager.cc:194] Re-registered known tserver with Master: e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:05:48.068785  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:48.069303  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:48.071156  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:05:48.081163 10856 ts_tablet_manager.cc:1434] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:05:48.085731 10856 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:48.087558 10841 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:48.090705 11009 raft_consensus.cc:2749] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:48.113557  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:48.106498 10841 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:48.115600 11009 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:48.117080 10841 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:48.118043 11009 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:48.119560 10841 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:05:48.120540 11009 raft_consensus.cc:2749] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:48.125015 10841 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:05:48.132606 11009 raft_consensus.cc:2749] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:48.194103  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:48.235886 10856 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.237206 10856 tablet_bootstrap.cc:492] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.238137 10856 ts_tablet_manager.cc:1403] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.015s
I20260709 13:05:48.254010 10856 raft_consensus.cc:359] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.254864 10856 raft_consensus.cc:740] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.258016 10856 consensus_queue.cc:260] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.272805 10856 ts_tablet_manager.cc:1434] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:05:48.275343 10856 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:48.291817  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:48.324306 10856 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.325907 10856 tablet_bootstrap.cc:492] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.326920 10856 ts_tablet_manager.cc:1403] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.003s
I20260709 13:05:48.334630 10856 raft_consensus.cc:359] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:48.335229  9686 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56888
I20260709 13:05:48.335537 10856 raft_consensus.cc:740] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.338328 10856 consensus_queue.cc:260] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:48.342025 10856 ts_tablet_manager.cc:1434] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:05:48.346830 10856 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:48.349068 10992 heartbeater.cc:499] Master 127.0.107.254:37093 was elected leader, sending a full tablet report...
W20260709 13:05:48.421932  9827 consensus_peers.cc:597] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:05:48.528967  9827 consensus_peers.cc:597] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:48.539693 10856 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.541088 10856 tablet_bootstrap.cc:492] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.541915 10856 ts_tablet_manager.cc:1403] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.195s	user 0.024s	sys 0.005s
I20260709 13:05:48.563364 10856 raft_consensus.cc:359] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:48.564254 10856 raft_consensus.cc:740] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.565215 10856 consensus_queue.cc:260] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:48.568331 10856 ts_tablet_manager.cc:1434] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:05:48.569373 10856 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:48.571360  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 66651.3us
W20260709 13:05:48.598198  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:05:48.614048 10955 raft_consensus.cc:3060] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:48.636672 10856 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.639340 10856 tablet_bootstrap.cc:492] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.640372 10856 ts_tablet_manager.cc:1403] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.071s	user 0.033s	sys 0.014s
I20260709 13:05:48.643725 10856 raft_consensus.cc:359] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.644585 10856 raft_consensus.cc:740] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.652776 10856 consensus_queue.cc:260] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.659377 10856 ts_tablet_manager.cc:1434] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:05:48.660490 10856 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:48.705694 10856 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.706789 10856 tablet_bootstrap.cc:492] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.707551 10856 ts_tablet_manager.cc:1403] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.017s
I20260709 13:05:48.711102 10856 raft_consensus.cc:359] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:48.711867 10856 raft_consensus.cc:740] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.712766 10856 consensus_queue.cc:260] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:48.715410 10856 ts_tablet_manager.cc:1434] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:05:48.716341 10856 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:48.800791 10856 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.802040 10856 tablet_bootstrap.cc:492] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.802942 10856 ts_tablet_manager.cc:1403] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.087s	user 0.031s	sys 0.029s
I20260709 13:05:48.806245 10856 raft_consensus.cc:359] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.807027 10856 raft_consensus.cc:740] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.807806 10856 consensus_queue.cc:260] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.811832 10856 ts_tablet_manager.cc:1434] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:05:48.814445 10856 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:48.887388 10856 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.889837 10856 tablet_bootstrap.cc:492] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.890749 10856 ts_tablet_manager.cc:1403] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.009s
I20260709 13:05:48.895610 10856 raft_consensus.cc:359] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.896469 10856 raft_consensus.cc:740] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.897251 10856 consensus_queue.cc:260] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:48.900624 10856 ts_tablet_manager.cc:1434] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:05:48.929778 10856 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:48.930356 10955 raft_consensus.cc:3060] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:48.981124  9827 consensus_peers.cc:597] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:48.992488 10856 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:48.993839 10856 tablet_bootstrap.cc:492] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:48.994657 10856 ts_tablet_manager.cc:1403] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.018s
I20260709 13:05:48.997978 10856 raft_consensus.cc:359] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:48.998816 10856 raft_consensus.cc:740] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.999629 10856 consensus_queue.cc:260] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.004966 10856 ts_tablet_manager.cc:1434] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 13:05:49.006292 10856 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:49.062645  9901 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:49.089082 10955 raft_consensus.cc:3060] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.095847 10856 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.097173 10856 tablet_bootstrap.cc:492] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:49.097995 10856 ts_tablet_manager.cc:1403] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.092s	user 0.020s	sys 0.015s
I20260709 13:05:49.110620 10856 raft_consensus.cc:359] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.111508 10856 raft_consensus.cc:740] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.112330 10856 consensus_queue.cc:260] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.114971 10856 ts_tablet_manager.cc:1434] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:05:49.116003 10856 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:49.149245 10856 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.151646 10856 tablet_bootstrap.cc:492] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:49.152529 10856 ts_tablet_manager.cc:1403] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.004s
I20260709 13:05:49.155524 10856 raft_consensus.cc:359] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:49.156523 10856 raft_consensus.cc:740] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.165747 10856 consensus_queue.cc:260] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:49.169476 10856 ts_tablet_manager.cc:1434] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:05:49.170548 10856 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:49.263762 10856 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.265493 10856 tablet_bootstrap.cc:492] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:49.266407 10856 ts_tablet_manager.cc:1403] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.096s	user 0.027s	sys 0.012s
I20260709 13:05:49.269796 10856 raft_consensus.cc:359] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:49.270761 10856 raft_consensus.cc:740] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.271795 10856 consensus_queue.cc:260] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:49.277887 10955 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.286952 10856 ts_tablet_manager.cc:1434] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:05:49.301064 10856 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:49.363493 10856 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.364889 10856 tablet_bootstrap.cc:492] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:49.366168 10856 ts_tablet_manager.cc:1403] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.012s
I20260709 13:05:49.374222 10955 raft_consensus.cc:3060] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.402704 10856 raft_consensus.cc:359] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:49.405732 10856 raft_consensus.cc:740] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.407171 10856 consensus_queue.cc:260] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:49.414676 10856 ts_tablet_manager.cc:1434] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.002s
I20260709 13:05:49.415766 10856 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:49.532325 10856 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.533614 10856 tablet_bootstrap.cc:492] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:49.534523 10856 ts_tablet_manager.cc:1403] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.119s	user 0.035s	sys 0.040s
I20260709 13:05:49.540868 10856 raft_consensus.cc:359] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.555925 10856 raft_consensus.cc:740] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.550771 10955 raft_consensus.cc:3060] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.557029 10856 consensus_queue.cc:260] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.561714 10856 ts_tablet_manager.cc:1434] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:05:49.562719 10856 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:49.568898 10942 raft_consensus.cc:3060] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:49.635708 10856 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:49.648979 10856 tablet_bootstrap.cc:492] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:49.650239 10856 ts_tablet_manager.cc:1403] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.016s
I20260709 13:05:49.655238 10856 raft_consensus.cc:359] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.658612 10856 raft_consensus.cc:740] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.664659 10856 consensus_queue.cc:260] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:49.673463 10856 ts_tablet_manager.cc:1434] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:05:49.674589 10856 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:49.772936  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 47497.1us
W20260709 13:05:49.974589  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 197553us
W20260709 13:05:50.149139  9827 consensus_peers.cc:597] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:50.159886 10856 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.161688 10856 tablet_bootstrap.cc:492] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:50.163518 10856 ts_tablet_manager.cc:1403] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.489s	user 0.031s	sys 0.007s
I20260709 13:05:50.168591 10856 raft_consensus.cc:359] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:50.169724 10856 raft_consensus.cc:740] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.171511 10856 consensus_queue.cc:260] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:50.302316 10856 ts_tablet_manager.cc:1434] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.138s	user 0.003s	sys 0.004s
I20260709 13:05:50.322250 10856 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
W20260709 13:05:50.340310  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 73735.5us
W20260709 13:05:50.385569  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 54082.2us
I20260709 13:05:50.423640 10856 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.425200 10856 tablet_bootstrap.cc:492] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:50.426116 10856 ts_tablet_manager.cc:1403] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.104s	user 0.032s	sys 0.004s
I20260709 13:05:50.429633 10856 raft_consensus.cc:359] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:50.430559 10856 raft_consensus.cc:740] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.431517 10856 consensus_queue.cc:260] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:50.434497 10856 ts_tablet_manager.cc:1434] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:50.464742 10856 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:50.501911 10856 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.503221 10856 tablet_bootstrap.cc:492] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:50.504045 10856 ts_tablet_manager.cc:1403] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.008s
I20260709 13:05:50.507176 10856 raft_consensus.cc:359] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:50.508013 10856 raft_consensus.cc:740] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.508986 10856 consensus_queue.cc:260] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:50.531702 10856 ts_tablet_manager.cc:1434] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:05:50.543264 10856 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Bootstrap starting.
I20260709 13:05:50.581710 10856 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:50.583456 10856 tablet_bootstrap.cc:492] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Bootstrap complete.
I20260709 13:05:50.584754 10856 ts_tablet_manager.cc:1403] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Time spent bootstrapping tablet: real 0.042s	user 0.025s	sys 0.000s
I20260709 13:05:50.595830 10856 raft_consensus.cc:359] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:50.597026 10856 raft_consensus.cc:740] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Initialized, Role: FOLLOWER
I20260709 13:05:50.598094 10856 consensus_queue.cc:260] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:50.613655 10856 ts_tablet_manager.cc:1434] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:05:50.678923 10943 raft_consensus.cc:3060] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.723891 10955 raft_consensus.cc:3060] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.875463 10942 raft_consensus.cc:3060] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.950289  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08001f5f00 after lost signal to thread 434
W20260709 13:05:50.952373  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08001ccf80 after lost signal to thread 9721
W20260709 13:05:50.954617  9885 debug-util.cc:398] Leaking SignalData structure 0x7b0800298980 after lost signal to thread 9888
W20260709 13:05:50.991852  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8560 after lost signal to thread 9962
W20260709 13:05:50.994765  9885 debug-util.cc:398] Leaking SignalData structure 0x7b0800139740 after lost signal to thread 10036
W20260709 13:05:50.997263  9885 debug-util.cc:398] Leaking SignalData structure 0x7b080023cbc0 after lost signal to thread 10990
W20260709 13:05:51.005744  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 99981.6us
W20260709 13:05:51.256384 10955 consensus_meta.cc:287] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df: Time spent flushing consensus metadata: real 0.523s	user 0.015s	sys 0.000s
W20260709 13:05:51.247184 10943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.082s	sys 0.144s
W20260709 13:05:51.272537 10943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.083s	sys 0.144s
W20260709 13:05:51.292312  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 115119us
W20260709 13:05:51.346282  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 106733us
W20260709 13:05:51.346395  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 165145us
W20260709 13:05:51.366683  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 68718.5us
W20260709 13:05:51.420578  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 163578us
W20260709 13:05:52.162743  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 46967us
W20260709 13:05:52.636700  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 66035.8us
W20260709 13:05:52.916112  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 48708.5us
W20260709 13:05:53.492473  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 44621us
W20260709 13:05:53.577104  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 88606us
W20260709 13:05:56.290460  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 72065.2us
W20260709 13:05:56.731180  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 63120.4us
I20260709 13:05:57.007200   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:57.209283  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 57366.2us
W20260709 13:05:57.461836  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 47680.3us
W20260709 13:05:57.489485  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 74915.4us
I20260709 13:05:57.514210 10012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:57.515404 10012 raft_consensus.cc:606] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:57.528725 10012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:57.531513 10012 raft_consensus.cc:606] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:57.535151 10011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:57.535817 10011 raft_consensus.cc:606] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:57.559760 11125 raft_consensus.cc:993] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.560396 11125 raft_consensus.cc:1081] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.563001 10937 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:05:57.563906 10937 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.564685 10937 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.591959 11125 raft_consensus.cc:993] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.592789 11125 raft_consensus.cc:1081] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.595710 10953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:05:57.596618 10953 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.596979 10953 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:57.613471 10937 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:57.618906 10937 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:57.617321 10953 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:57.620888  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:57.622821  9936 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.621493 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:57.628989 10011 raft_consensus.cc:3055] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:57.629402 10011 raft_consensus.cc:740] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:57.630040  9935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c"
dest_uuid: "d02e8a054a634be599149db641901736"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46044
I20260709 13:05:57.630743  9935 raft_consensus.cc:606] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:57.630486 10011 consensus_queue.cc:260] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:57.634389 10012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:57.635309 10012 raft_consensus.cc:606] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Received request to transfer leadership to TS 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:57.633906 10011 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.639688 10012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8a86072500f497585b83ac124a8a562"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:57.640358 10012 raft_consensus.cc:606] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:57.642347   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:57.644940   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:57.659475 11140 raft_consensus.cc:993] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.660146 11140 raft_consensus.cc:1081] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.660735 10953 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:57.661578 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:57.662348 10008 raft_consensus.cc:3055] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:57.662647 10008 raft_consensus.cc:740] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:57.663528 10008 consensus_queue.cc:260] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:57.665603  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404c2d54fc4e4b6c8a892de6e4988671" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:57.666690  9864 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:57.666893 10008 raft_consensus.cc:3060] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:57.679126 10953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8a86072500f497585b83ac124a8a562"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:05:57.686918 10953 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.695537 10953 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.710731  9864 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 4.
I20260709 13:05:57.697242  9936 raft_consensus.cc:2468] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:57.714124 10831 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:57.712009 10011 raft_consensus.cc:2468] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:57.718334 10841 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:57.723474 10953 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:57.732539 10953 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:57.744536 11145 raft_consensus.cc:2804] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:57.746338 11144 raft_consensus.cc:2804] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:57.750074 10008 raft_consensus.cc:2468] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 4.
I20260709 13:05:57.758951 11145 raft_consensus.cc:697] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:57.760298 11145 consensus_queue.cc:237] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:05:57.767246 10831 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 53331.7us
I20260709 13:05:57.782539 11140 raft_consensus.cc:993] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:57.783169 11140 raft_consensus.cc:1081] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Signalling peer 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:57.785866  9860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:52528
I20260709 13:05:57.786489  9860 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.787119  9860 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.797210  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:57.798071  9936 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.802258 11144 raft_consensus.cc:697] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 4 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:57.797639  9860 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:57.803702 11144 consensus_queue.cc:237] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:57.807092  9936 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:57.807875  9860 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:57.809609 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:57.824717 10008 raft_consensus.cc:3055] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:57.825115 10008 raft_consensus.cc:740] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:57.825946 10008 consensus_queue.cc:260] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:57.827411 10008 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.810719 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:57.811167 10831 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:57.828680 10011 raft_consensus.cc:3055] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.829032 10011 raft_consensus.cc:740] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:57.829162 11144 raft_consensus.cc:2804] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:57.829785 11144 raft_consensus.cc:697] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:57.829914 10011 consensus_queue.cc:260] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:57.830797 11144 consensus_queue.cc:237] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:57.831859 10011 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.843118  9686 catalog_manager.cc:5697] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.845415  9686 catalog_manager.cc:5697] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 3 to 4, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 4 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.811218  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:57.856544  9936 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.888715 10011 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 3.
I20260709 13:05:57.891403 10008 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:57.907117  9829 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:57.914474 11139 raft_consensus.cc:2804] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:57.915099 11139 raft_consensus.cc:697] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:57.916088 11139 consensus_queue.cc:237] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:57.910969  9936 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 3.
I20260709 13:05:57.911808 10886 raft_consensus.cc:993] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:57.953020 10886 raft_consensus.cc:1081] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:57.956369  9864 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:52528
I20260709 13:05:57.957204  9864 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.957895  9864 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:57.988833 11111 raft_consensus.cc:993] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.990509 11122 raft_consensus.cc:1081] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:57.999351 10937 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46174
I20260709 13:05:58.002218 10937 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.003041 10937 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.020022  9864 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:58.033203  9685 catalog_manager.cc:5697] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 2 to 3, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.044149  9864 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:58.055248  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:58.055670 10011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:58.056509 10011 raft_consensus.cc:3055] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.056872 10011 raft_consensus.cc:740] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:58.055810  9936 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.058082 10011 consensus_queue.cc:260] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:58.060400 10011 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.067232 10937 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:58.067862  9936 raft_consensus.cc:2468] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:58.085247 10937 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:58.087806 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:58.088915 10008 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.102763  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:58.103811  9935 raft_consensus.cc:3055] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.104171  9935 raft_consensus.cc:740] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:58.121379  9685 catalog_manager.cc:5697] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.129546  9829 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:05:58.130559  9935 consensus_queue.cc:260] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:58.135739 10011 raft_consensus.cc:2468] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:58.144724 11137 raft_consensus.cc:2804] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.145684 11137 raft_consensus.cc:697] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:58.147307 11137 consensus_queue.cc:237] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:58.159565  9936 raft_consensus.cc:1275] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.164734 11144 consensus_queue.cc:1048] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.164775 10008 raft_consensus.cc:2468] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:58.162845  9935 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.167726 10012 raft_consensus.cc:1240] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Rejecting Update request from peer d02e8a054a634be599149db641901736 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:58.174120 10831 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:58.178927 11144 raft_consensus.cc:2804] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.179487 11144 raft_consensus.cc:697] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:58.180953 11144 consensus_queue.cc:237] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:58.201282  9935 raft_consensus.cc:2468] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:58.202596 10012 raft_consensus.cc:1275] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:58.205303 11148 consensus_queue.cc:1048] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:58.230473  9936 raft_consensus.cc:1275] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.238770  9685 catalog_manager.cc:5697] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.262799 11148 consensus_queue.cc:1048] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.269358  9686 catalog_manager.cc:5697] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from d02e8a054a634be599149db641901736 (127.0.107.195) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.281052 10012 raft_consensus.cc:1275] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.309175 11145 consensus_queue.cc:1048] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.359827  9858 raft_consensus.cc:1275] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:58.365563 11145 consensus_queue.cc:1048] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:58.386703 10012 raft_consensus.cc:1275] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.389410 11145 consensus_queue.cc:1048] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.501605  9935 raft_consensus.cc:1275] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:58.504947 11157 consensus_queue.cc:1048] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.510210 10009 raft_consensus.cc:1275] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:58.527554 11157 consensus_queue.cc:1048] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:58.638325  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 67174.8us
W20260709 13:05:58.644021  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 83100.5us
I20260709 13:05:58.687170   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:58.755357 10012 raft_consensus.cc:1275] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.767936 10012 raft_consensus.cc:1275] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:58.780472 11172 log.cc:927] Time spent T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.124s	user 0.000s	sys 0.002s
I20260709 13:05:58.776870 11148 consensus_queue.cc:1048] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
W20260709 13:05:58.810459  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 156641us
I20260709 13:05:58.829826  9935 raft_consensus.cc:1275] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:58.863535  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 53482.7us
I20260709 13:05:58.876600 11157 consensus_queue.cc:1048] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:58.937727 11165 log.cc:927] Time spent T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.325s	user 0.006s	sys 0.000s
W20260709 13:05:58.971885  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 43940us
I20260709 13:05:58.975752   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:58.991142  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 55149.1us
I20260709 13:05:59.000458  9935 raft_consensus.cc:1275] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.002458 11181 consensus_queue.cc:1048] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.020737 11144 consensus_queue.cc:1048] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.200871 10008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:59.201503 10008 raft_consensus.cc:606] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:59.209554 10008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:59.210176 10008 raft_consensus.cc:606] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Received request to transfer leadership to TS 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:59.218951  9933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3695e4783714136ad332a175348e9fc"
dest_uuid: "d02e8a054a634be599149db641901736"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46044
I20260709 13:05:59.219725  9933 raft_consensus.cc:606] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:59.223160 10008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:59.224110 10008 raft_consensus.cc:606] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:05:59.230801 10011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:59.231640 10011 raft_consensus.cc:606] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:59.269438 10008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:05:59.275841 10008 raft_consensus.cc:606] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:05:59.288925 11140 raft_consensus.cc:993] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.290565 11198 raft_consensus.cc:1081] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.292943 10955 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:05:59.293704 10955 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.294164 10955 raft_consensus.cc:3060] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:59.315588  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 46979.7us
I20260709 13:05:59.321429 10955 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:59.331167 10955 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:59.338523  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:59.340698  9855 raft_consensus.cc:3060] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.348845 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5dd44a42946450e936b89718864c5e3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:59.349727 10008 raft_consensus.cc:3055] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:59.350097 10008 raft_consensus.cc:740] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:59.351137 10008 consensus_queue.cc:260] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:59.360437 10008 raft_consensus.cc:3060] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.365473   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:59.366639 11187 raft_consensus.cc:993] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.367151 11187 raft_consensus.cc:1081] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.367368   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:59.369910 10955 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:05:59.371198 10955 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.372325 10955 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.378758 10008 raft_consensus.cc:2468] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:59.380772 10831 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:59.382021 11148 raft_consensus.cc:2804] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:59.382906 11148 raft_consensus.cc:697] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:59.386788 11148 consensus_queue.cc:237] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:59.419039  9855 raft_consensus.cc:2468] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:59.431553  9863 raft_consensus.cc:1240] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Rejecting Update request from peer 5e087ae2ce3f417093313edd26b8b3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:59.440078 10886 raft_consensus.cc:993] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.440970 11130 raft_consensus.cc:1081] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.447357 10953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:05:59.448206 10953 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.448602 10953 raft_consensus.cc:3060] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.474936 11180 raft_consensus.cc:993] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.475579 11180 raft_consensus.cc:1081] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:05:59.495528  9685 catalog_manager.cc:5697] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.522634 10955 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:59.525261 10955 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:59.521761 10953 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:59.530858  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:59.532574  9935 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.533136 10937 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3695e4783714136ad332a175348e9fc"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46174
I20260709 13:05:59.533393 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:59.533824 10953 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:05:59.534341 10008 raft_consensus.cc:3055] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:59.534789 10008 raft_consensus.cc:740] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:59.538072  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:05:59.538817  9864 raft_consensus.cc:3060] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.535746 10008 consensus_queue.cc:260] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:59.534181 10937 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.546962 10937 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.551245 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c68f14a359d54249b7cab91d0e63d7b0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:59.551963 10006 raft_consensus.cc:3055] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:59.552244 10006 raft_consensus.cc:740] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:59.553082 10006 consensus_queue.cc:260] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:59.553596 10008 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.554451 10006 raft_consensus.cc:3060] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.566804  9935 raft_consensus.cc:2468] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:59.567497 10006 raft_consensus.cc:2468] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:59.571990 10008 raft_consensus.cc:2468] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:59.578742  9864 raft_consensus.cc:2468] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:05:59.580222 10831 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:59.583732 10841 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:59.586689 10937 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:59.589042 10937 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:59.595798  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:59.596213 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:59.596571  9933 raft_consensus.cc:3055] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:59.596828 10006 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.597265  9933 raft_consensus.cc:740] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:05:59.598213  9933 consensus_queue.cc:260] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:59.594465 11148 raft_consensus.cc:2804] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:59.599362  9933 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.600178 11148 raft_consensus.cc:697] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:59.596325 11145 raft_consensus.cc:2804] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:59.606313 11145 raft_consensus.cc:697] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:59.609313  9933 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:05:59.601590 11148 consensus_queue.cc:237] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:59.607606 11145 consensus_queue.cc:237] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:05:59.612882 10831 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:05:59.605422 10006 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:05:59.613749 11144 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:59.664427 11130 raft_consensus.cc:993] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:59.664934 11130 raft_consensus.cc:1081] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Signalling peer 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:59.675493  9855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:52528
I20260709 13:05:59.676613  9855 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.677217  9855 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.701748  9855 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:59.699038 11144 raft_consensus.cc:697] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:05:59.703341 11130 raft_consensus.cc:993] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:59.718292 11187 raft_consensus.cc:1081] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:05:59.718590 11144 consensus_queue.cc:237] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:05:59.722169  9855 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:05:59.722467  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:59.723340  9933 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.725400 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:59.726063 10008 raft_consensus.cc:3055] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:59.726333 10008 raft_consensus.cc:740] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:59.727136 10008 consensus_queue.cc:260] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:59.728307 10008 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.733366  9863 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:52528
I20260709 13:05:59.734205  9863 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.734635  9863 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:59.740736 11210 log.cc:927] Time spent T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
I20260709 13:05:59.746204 10008 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 3.
I20260709 13:05:59.747879  9829 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:59.749488 11137 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:59.749979 11137 raft_consensus.cc:697] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:59.750844 11137 consensus_queue.cc:237] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:05:59.762547  9933 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 3.
I20260709 13:05:59.768195  9863 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:59.771386  9863 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:05:59.772258 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:05:59.773313 10008 raft_consensus.cc:3055] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:59.773737 10008 raft_consensus.cc:740] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:05:59.774811 10008 consensus_queue.cc:260] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:59.780073  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:05:59.781040  9933 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.781376 10008 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.789618 10008 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:59.792865  9829 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:05:59.793844 11137 raft_consensus.cc:2804] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:59.796486  9687 catalog_manager.cc:5697] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 2 to 3, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 3 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.805333 11137 raft_consensus.cc:697] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:05:59.812951 11137 consensus_queue.cc:237] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:05:59.824654  9685 catalog_manager.cc:5697] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:59.828986  9685 catalog_manager.cc:5697] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.847115  9933 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:05:59.831223  9685 catalog_manager.cc:5697] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 2 to 3, leader changed from d02e8a054a634be599149db641901736 (127.0.107.195) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 3 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:59.852735  9687 catalog_manager.cc:5697] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:59.866987 10011 raft_consensus.cc:1275] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.882012 11145 consensus_queue.cc:1048] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.944589  9863 raft_consensus.cc:1275] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.955065 11148 consensus_queue.cc:1048] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:00.015813  9933 raft_consensus.cc:1275] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.018867 11144 consensus_queue.cc:1048] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:00.076519 10011 raft_consensus.cc:1275] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.082608 11144 consensus_queue.cc:1048] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:00.095368  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 52657us
I20260709 13:06:00.117137 10011 raft_consensus.cc:1275] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.135321 11209 consensus_queue.cc:1048] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:00.160827 10008 raft_consensus.cc:1275] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:00.180904  9935 raft_consensus.cc:1275] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:00.194808 11196 consensus_queue.cc:1048] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:00.198542  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 51799.2us
I20260709 13:06:00.196177 11185 consensus_queue.cc:1048] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:00.230926  9864 raft_consensus.cc:1275] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.235257 11144 consensus_queue.cc:1048] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:00.261571  9935 raft_consensus.cc:1275] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:00.267324 11144 consensus_queue.cc:1048] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:00.271461  9933 raft_consensus.cc:1275] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.274785 11185 consensus_queue.cc:1048] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:00.409817   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:00.557789  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 43751.9us
W20260709 13:06:00.560294  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 53764.6us
I20260709 13:06:00.570061 10006 raft_consensus.cc:1275] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:00.584898 10006 raft_consensus.cc:1275] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:06:00.596477  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 60484.5us
I20260709 13:06:00.618467   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:00.633915 11209 consensus_queue.cc:1048] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:00.628372 11189 consensus_queue.cc:1048] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:06:00.678406 10831 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 66436.6us
W20260709 13:06:00.688584  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 199732us
W20260709 13:06:00.696861  9904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 183940us
I20260709 13:06:00.813153 10006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "d02e8a054a634be599149db641901736"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:06:00.813975 10006 raft_consensus.cc:606] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS d02e8a054a634be599149db641901736
I20260709 13:06:00.821069 10006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:06:00.831053 10006 raft_consensus.cc:606] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:00.835259 11130 raft_consensus.cc:993] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower d02e8a054a634be599149db641901736 to start an election
I20260709 13:06:00.836310 11245 raft_consensus.cc:1081] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer d02e8a054a634be599149db641901736 to start an election
I20260709 13:06:00.839877  9936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d"
dest_uuid: "d02e8a054a634be599149db641901736"
 from {username='slave'} at 127.0.0.1:40634
I20260709 13:06:00.840495  9936 raft_consensus.cc:493] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:00.840848  9936 raft_consensus.cc:3060] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:00.882661  9936 raft_consensus.cc:515] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:00.901491  9936 leader_election.cc:290] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:00.998747 11245 raft_consensus.cc:993] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:00.999267 11245 raft_consensus.cc:1081] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:01.014592 10953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:06:01.015208 10953 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:01.015563 10953 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.016288 10009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:01.017280 10009 raft_consensus.cc:3055] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:01.017616 10009 raft_consensus.cc:740] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:01.018509 10009 consensus_queue.cc:260] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:01.022966 10953 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:01.024861 10009 raft_consensus.cc:3060] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.028990 10953 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:01.040849  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:01.041899  9938 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.043704 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:01.044454 10006 raft_consensus.cc:3055] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:01.044770 10006 raft_consensus.cc:740] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:01.045573 10006 consensus_queue.cc:260] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:01.094765 10009 raft_consensus.cc:2468] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:01.097013 10006 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:01.100708  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 63498.8us
I20260709 13:06:01.102819  9938 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
W20260709 13:06:01.104031  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 48598.9us
I20260709 13:06:01.107134 10831 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:06:01.112869 11144 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:01.113474 11144 raft_consensus.cc:697] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:06:01.114606 11144 consensus_queue.cc:237] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:01.142513 10006 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:06:01.153168  9903 leader_election.cc:304] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:01.166949 11246 raft_consensus.cc:2804] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:01.168392 11246 raft_consensus.cc:697] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:01.170451 11246 consensus_queue.cc:237] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:01.188449  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cfdee35c2f0491da78cbc51ed63ba5d" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:01.189430  9860 raft_consensus.cc:3060] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.224174  9860 raft_consensus.cc:2468] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:01.271045  9687 catalog_manager.cc:5697] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:01.282253  9686 catalog_manager.cc:5697] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.363400  9858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4716f93906794e73b45089786b4a75f0"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:54588
I20260709 13:06:01.364403  9858 raft_consensus.cc:606] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:01.370410 10010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:06:01.371182 10010 raft_consensus.cc:606] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:01.375232 10010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:06:01.375995 10010 raft_consensus.cc:606] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Received request to transfer leadership to TS 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:06:01.377460   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:01.380232   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:01.464720 11198 raft_consensus.cc:993] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:06:01.465902 11140 raft_consensus.cc:1081] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer 24c6a28138294e46bf6b3d5ba324046d to start an election
I20260709 13:06:01.474916  9858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
 from {username='slave'} at 127.0.0.1:52528
I20260709 13:06:01.475628  9858 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:01.475946  9858 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.511579  9858 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:01.514225  9858 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:01.524907 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:01.525552 10008 raft_consensus.cc:3055] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:01.525838 10008 raft_consensus.cc:740] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:01.526508 10008 consensus_queue.cc:260] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:01.527499  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "24c6a28138294e46bf6b3d5ba324046d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:01.527660  9931 raft_consensus.cc:1275] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:01.527808 10008 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.535058 10008 raft_consensus.cc:2468] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:06:01.547432 11145 consensus_queue.cc:1048] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:01.550475  9829 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:06:01.557179 11137 raft_consensus.cc:2804] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:01.557677 11137 raft_consensus.cc:697] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Becoming Leader. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:06:01.558612 11137 consensus_queue.cc:237] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:01.569839  9936 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.632165 10008 raft_consensus.cc:1275] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:01.634058 11209 consensus_queue.cc:1048] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:01.646584  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 51833.7us
I20260709 13:06:01.664145  9860 raft_consensus.cc:1275] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:01.735636  9936 raft_consensus.cc:2468] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24c6a28138294e46bf6b3d5ba324046d in term 2.
I20260709 13:06:01.736816 11194 consensus_queue.cc:1048] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:01.754549  9686 catalog_manager.cc:5697] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194). New cstate: current_term: 2 leader_uuid: "24c6a28138294e46bf6b3d5ba324046d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:01.803815 10008 raft_consensus.cc:1275] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:01.806207 11194 consensus_queue.cc:1048] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:01.840161 11137 raft_consensus.cc:993] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:01.840903 11137 raft_consensus.cc:1081] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:01.845404 10956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4716f93906794e73b45089786b4a75f0"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46188
I20260709 13:06:01.846244 10956 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:01.846683 10956 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:01.867610 11245 raft_consensus.cc:993] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:01.869483 11240 raft_consensus.cc:1081] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:01.872843 10959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:06:01.873664 10959 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:01.874128 10959 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.905011 10959 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:01.908998  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:01.909803  9936 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.910069 10956 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:01.912880 10956 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:01.913937 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:01.914613 10008 raft_consensus.cc:3055] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:01.915038 10008 raft_consensus.cc:740] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:01.914712  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:01.915876  9858 raft_consensus.cc:3055] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:01.916157 10008 consensus_queue.cc:260] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:01.917505 10008 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:01.918205  9858 raft_consensus.cc:740] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:06:01.919350  9858 consensus_queue.cc:260] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:01.920889  9858 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:01.923031  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4716f93906794e73b45089786b4a75f0" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:01.923810  9933 raft_consensus.cc:3060] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:01.926625 10008 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:06:01.927647 10959 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:01.931627 10831 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:06:01.932853 11209 raft_consensus.cc:2804] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:01.933507 11209 raft_consensus.cc:697] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:06:01.934741 11209 consensus_queue.cc:237] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:01.937518  9858 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 4.
I20260709 13:06:01.946007 10841 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:06:01.947398  9936 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 2.
I20260709 13:06:01.948467  9933 raft_consensus.cc:2468] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 4.
I20260709 13:06:01.948565 11144 raft_consensus.cc:2804] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:02.016768 11144 raft_consensus.cc:697] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 4 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:06:02.017970 11144 consensus_queue.cc:237] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:02.038604 10010 raft_consensus.cc:1275] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.045102  9938 raft_consensus.cc:1275] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer 24c6a28138294e46bf6b3d5ba324046d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.048659 11137 consensus_queue.cc:1048] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:02.049602  9686 catalog_manager.cc:5697] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 1 to 2, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 2 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.050375 11137 consensus_queue.cc:1048] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.080024  9686 catalog_manager.cc:5697] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 3 to 4, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 4 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:02.363224  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 110835us
I20260709 13:06:02.381819   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:02.385574  9933 raft_consensus.cc:1275] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.387130 11273 consensus_queue.cc:1048] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:02.395996 10010 raft_consensus.cc:1275] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.400223 11273 consensus_queue.cc:1048] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:02.660197  9855 raft_consensus.cc:1275] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:02.667572   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:02.678577 11273 consensus_queue.cc:1048] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:02.725462 11280 log.cc:927] Time spent T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: Append to log took a long time: real 0.051s	user 0.000s	sys 0.002s
I20260709 13:06:02.765074  9933 raft_consensus.cc:1275] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:02.780678  9933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6"
dest_uuid: "d02e8a054a634be599149db641901736"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46044
I20260709 13:06:02.781430  9933 raft_consensus.cc:606] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:02.793532 10010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:06:02.794409 10010 raft_consensus.cc:606] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:02.796432 11273 consensus_queue.cc:1048] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:02.814456  9855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c"
dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
mode: GRACEFUL
new_leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
 from {username='slave'} at 127.0.0.1:54588
I20260709 13:06:02.815332  9855 raft_consensus.cc:606] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Received request to transfer leadership to TS 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:02.817649   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:02.820086   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:02.899518 11285 raft_consensus.cc:993] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:02.900239 11285 raft_consensus.cc:1081] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:02.918707 10956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:06:02.919538 10956 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:02.920042 10956 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:02.925940 11279 raft_consensus.cc:993] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: : Instructing follower 5e087ae2ce3f417093313edd26b8b3ee to start an election
I20260709 13:06:02.926549 11279 raft_consensus.cc:1081] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Signalling peer 5e087ae2ce3f417093313edd26b8b3ee to start an election
I20260709 13:06:02.985765 10956 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:02.990149 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:02.990944 10010 raft_consensus.cc:3055] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:02.991304 10010 raft_consensus.cc:740] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:02.992137 10010 consensus_queue.cc:260] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:02.993655 10010 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.012138 10008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
 from {username='slave'} at 127.0.0.1:32920
I20260709 13:06:03.013286 10008 raft_consensus.cc:493] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.013728 10008 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.020293 10956 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:03.025447  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22c6e691c1264b8e8de49c1df4cd8bb5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:03.027704  9858 raft_consensus.cc:3060] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.053345 10008 raft_consensus.cc:515] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:03.053392  9858 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:06:03.076210 10841 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:06:03.077503 11283 raft_consensus.cc:2804] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:03.078090 11283 raft_consensus.cc:697] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:06:03.079139 11283 consensus_queue.cc:237] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:03.059971 10010 raft_consensus.cc:2468] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:06:03.092620 10008 leader_election.cc:290] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:03.095340  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:03.098806  9858 raft_consensus.cc:3055] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:03.099227  9858 raft_consensus.cc:740] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24c6a28138294e46bf6b3d5ba324046d, State: Running, Role: LEADER
I20260709 13:06:03.103554  9858 consensus_queue.cc:260] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:03.104950  9858 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.103567  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9ea20d35c3b4150ba41cec32b7c9e0c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:03.108126  9938 raft_consensus.cc:3060] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.162860  9858 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 4.
I20260709 13:06:03.181802 11236 raft_consensus.cc:993] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:03.176149  9978 leader_election.cc:304] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee; no voters: 
I20260709 13:06:03.196451 11198 raft_consensus.cc:2804] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:03.197326 11198 raft_consensus.cc:697] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 4 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:03.198386 11198 consensus_queue.cc:237] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:03.184120 11236 raft_consensus.cc:1081] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
W20260709 13:06:03.228446 10831 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 60527.4us
I20260709 13:06:03.234442 10953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46174
I20260709 13:06:03.235450 10953 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.235996 10953 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.237653  9938 raft_consensus.cc:2468] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 4.
I20260709 13:06:03.273775  9686 catalog_manager.cc:5697] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 3 to 4, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 4 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:03.282357  9687 catalog_manager.cc:5697] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 2 to 3, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 3 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.317699 10953 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:03.333470  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:03.334436  9933 raft_consensus.cc:3055] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:03.334803  9933 raft_consensus.cc:740] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:03.335693  9933 consensus_queue.cc:260] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:03.341737  9933 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.332252 10953 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:03.374706  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:03.375666  9864 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.407640  9864 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:06:03.415974 10841 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:06:03.417340 11293 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:03.417987 11293 raft_consensus.cc:697] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:06:03.419374 11293 consensus_queue.cc:237] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:03.414556  9933 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:06:03.477494  9686 catalog_manager.cc:5697] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 2 to 3, leader changed from d02e8a054a634be599149db641901736 (127.0.107.195) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 3 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.599139  9858 raft_consensus.cc:1275] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:03.604001 11293 consensus_queue.cc:1048] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:03.683732 10010 raft_consensus.cc:1275] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:03.702111  9933 raft_consensus.cc:1275] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:03.704408 11240 consensus_queue.cc:1048] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:03.717276 11293 consensus_queue.cc:1048] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:03.738632  9860 raft_consensus.cc:1275] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:03.742086 11198 consensus_queue.cc:1048] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:03.821691   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:03.919911   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:03.928658  9931 raft_consensus.cc:1275] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:03.932610 11292 consensus_queue.cc:1048] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:03.973371  9864 raft_consensus.cc:1275] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:03.976276 11293 consensus_queue.cc:1048] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:04.018368 10010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3"
dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
mode: GRACEFUL
new_leader_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:35110
I20260709 13:06:04.021839 10010 raft_consensus.cc:606] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Received request to transfer leadership to TS e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:04.023690   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:06:04.027047   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:04.385835 11198 raft_consensus.cc:993] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: : Instructing follower e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:04.386566 11198 raft_consensus.cc:1081] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Signalling peer e8769fa7061d433bb90a9411c71fa0df to start an election
I20260709 13:06:04.389607 10956 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3"
dest_uuid: "e8769fa7061d433bb90a9411c71fa0df"
 from {username='slave'} at 127.0.0.1:46158
I20260709 13:06:04.390479 10956 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.391368 10956 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:04.424621 10831 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 55231.4us
I20260709 13:06:04.449128 10956 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:04.475775 10956 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:04.515161  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:04.516399  9858 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:04.549563 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04298f45db464a09bd0d2b82492e61b3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:04.550730 10012 raft_consensus.cc:3055] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:04.551071 10012 raft_consensus.cc:740] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:04.554726 10012 consensus_queue.cc:260] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:04.559332 10012 raft_consensus.cc:3060] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:04.605831  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 126299us
I20260709 13:06:04.632054  9858 raft_consensus.cc:2468] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:06:04.645931 10012 raft_consensus.cc:2468] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8769fa7061d433bb90a9411c71fa0df in term 3.
I20260709 13:06:04.668965 10831 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df; no voters: 
I20260709 13:06:04.670117 11284 raft_consensus.cc:2804] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:04.670881 11284 raft_consensus.cc:697] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Becoming Leader. State: Replica: e8769fa7061d433bb90a9411c71fa0df, State: Running, Role: LEADER
I20260709 13:06:04.672061 11284 consensus_queue.cc:237] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:06:04.957849  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 88073us
W20260709 13:06:04.977720  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 348814us
W20260709 13:06:04.977028  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 111721us
W20260709 13:06:05.083623  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 124416us
I20260709 13:06:05.115579   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:05.191959 10012 raft_consensus.cc:1275] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:06:05.210855  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 125291us
I20260709 13:06:05.245291 11294 consensus_queue.cc:1048] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:05.252177  9686 catalog_manager.cc:5697] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df reported cstate change: term changed from 2 to 3, leader changed from 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196) to e8769fa7061d433bb90a9411c71fa0df (127.0.107.193). New cstate: current_term: 3 leader_uuid: "e8769fa7061d433bb90a9411c71fa0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:05.375198 10831 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 50884.4us
W20260709 13:06:05.385308  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 88043.8us
I20260709 13:06:05.466495  9864 raft_consensus.cc:1275] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Refusing update from remote peer e8769fa7061d433bb90a9411c71fa0df: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:06:05.467554  9975 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 74501.7us
W20260709 13:06:05.469600  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 77974us
W20260709 13:06:05.548539  9903 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38131, user_credentials={real_user=slave}} blocked reactor thread for 46253.6us
I20260709 13:06:05.562415 11293 consensus_queue.cc:1048] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:06.153757   431 tablet_server.cc:179] TabletServer@127.0.107.193:32801 shutting down...
W20260709 13:06:06.189934  9901 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 421 similar messages]
W20260709 13:06:06.259480  9901 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 69742.8us
W20260709 13:06:06.316201  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:32801, user_credentials={real_user=slave}} blocked reactor thread for 59514.1us
W20260709 13:06:06.322830  9827 consensus_peers.cc:597] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.325634  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.338162  9901 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.339500  9975 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.340999  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.342108  9827 consensus_peers.cc:597] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.346627  9827 consensus_peers.cc:597] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.377390  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.398258  9975 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.400767  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.403021  9901 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.423195  9901 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.441085  9827 consensus_peers.cc:597] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.472913  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.475591  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.478492  9901 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.499047  9827 consensus_peers.cc:597] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.519081  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.518935  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.519840   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:06.524624   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.530238   431 raft_consensus.cc:2243] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:06.537130  9901 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.545343  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.565487  9827 consensus_peers.cc:597] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.573421  9827 consensus_peers.cc:597] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.580137  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.543448   431 raft_consensus.cc:2272] T 357a7ca5934d4fb6958de3cca90f5a36 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.590058   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.591741   431 raft_consensus.cc:2243] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.592628   431 raft_consensus.cc:2272] T d775d5d3993743689257e4509a059770 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.598752   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.600270   431 raft_consensus.cc:2243] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.601121   431 raft_consensus.cc:2272] T e18d9519bce7451aaee1e04623f7fddb P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.604292  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.605913   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.607643   431 raft_consensus.cc:2243] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.608409   431 raft_consensus.cc:2272] T 9625fd7e9ed046158552e5252f1ee3db P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.614171   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.615677   431 raft_consensus.cc:2243] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.616554   431 raft_consensus.cc:2272] T a693233cb65a40089d314773895f84a0 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.621227   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.622682   431 raft_consensus.cc:2243] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:06.623839  9827 consensus_peers.cc:597] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.625696   431 raft_consensus.cc:2272] T 210c05e6675c4ba78ca28841a5315a07 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.631011  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.636454   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.638814   431 raft_consensus.cc:2243] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.639714   431 raft_consensus.cc:2272] T 1131589c5bd9480d838a7967131195ca P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.653159   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.654470   431 raft_consensus.cc:2243] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.655504   431 raft_consensus.cc:2272] T 82999b9832894f40bb6dbe02e2feb1df P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.673846   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.675334   431 raft_consensus.cc:2243] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.677713   431 raft_consensus.cc:2272] T 0f50ff8860ba431d9ee421ff76612df6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.686131  9901 consensus_peers.cc:597] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.690531   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.692112   431 raft_consensus.cc:2243] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.693038   431 raft_consensus.cc:2272] T 47be166313094f0fbe17bde042f05dfe P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.698619   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.700008   431 raft_consensus.cc:2243] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.700829   431 raft_consensus.cc:2272] T 9cb3e93354824df58589f250b6264d54 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.705389   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.706522   431 raft_consensus.cc:2243] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.710911   431 raft_consensus.cc:2272] T 7d6f1cfb3ffd4a418cafdda5af0b219d P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.715471   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.720196   431 raft_consensus.cc:2243] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.721019   431 raft_consensus.cc:2272] T f0bdd04f0ff44f2e87f76f36a7c5e540 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.725939   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:06.726622  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.729439   431 raft_consensus.cc:2243] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.730484   431 raft_consensus.cc:2272] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:06.741884  9978 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:32919, user_credentials={real_user=slave}} blocked reactor thread for 45388us
I20260709 13:06:06.826723   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:06.836380   431 raft_consensus.cc:2243] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:06.837284   431 raft_consensus.cc:2272] T b253ddcc7d6046c6abdd94aec212612c P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:06.932760   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:06.945385  9827 consensus_peers.cc:597] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:06.946094  9827 consensus_peers.cc:597] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:06.986084   431 raft_consensus.cc:2243] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:06.987730   431 raft_consensus.cc:2272] T e62c671eeefb49758edb7a7273fc6a2c P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:07.006524  9977 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37607, user_credentials={real_user=slave}} blocked reactor thread for 59894.8us
I20260709 13:06:07.114322 11328 raft_consensus.cc:493] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.115330 11328 raft_consensus.cc:515] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.119815 11328 leader_election.cc:290] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.122355  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.293520 11292 raft_consensus.cc:493] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.294435 11292 raft_consensus.cc:515] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.365610 11336 raft_consensus.cc:493] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.368671 11292 leader_election.cc:290] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.432422 11292 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:07.433364 11292 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.416057   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.432287 11336 raft_consensus.cc:515] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:07.438526   431 raft_consensus.cc:2243] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.440788 11292 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.441715 11292 raft_consensus.cc:493] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.442637 11292 raft_consensus.cc:515] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.446072 11292 leader_election.cc:290] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:07.440243   431 raft_consensus.cc:2272] T c68f14a359d54249b7cab91d0e63d7b0 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.493134  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bab397fa40041c281ce616134672828" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.440650 11336 leader_election.cc:290] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:07.498683 11336 raft_consensus.cc:493] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.499840 11336 raft_consensus.cc:515] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.520880  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.524629 11336 leader_election.cc:290] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.526940   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.528414   431 raft_consensus.cc:2243] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:07.625402 11353 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:07.626260 11353 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.628713 11353 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:07.669196  9931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.669584  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.670773  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.680305  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a2a7189703643899a38fc9b896d3c78" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.693682   431 raft_consensus.cc:2272] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.725281 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c416362f795e46e28a60937c9ef5164f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.727027 10010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.731784 11347 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:07.732450 11347 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.741343  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.742069  9938 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:06:07.742853 11336 raft_consensus.cc:493] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.743580 11336 raft_consensus.cc:515] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:07.746066 11351 raft_consensus.cc:493] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.746685 11351 raft_consensus.cc:515] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:07.760177 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.760915 10006 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:07.761119 11351 leader_election.cc:290] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:07.775350 11347 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:07.776710 11292 raft_consensus.cc:493] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.778087 11355 raft_consensus.cc:493] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.791105 11292 raft_consensus.cc:515] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:07.791369 11355 raft_consensus.cc:515] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.793524 11292 leader_election.cc:290] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:07.787034  9975 leader_election.cc:336] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:07.793751 11328 raft_consensus.cc:493] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.789877  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5fa7348f6384183abeb9619f1fd0d75" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.790133  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf82e92a7f845f4a7c658d0783ea4a5" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.790381  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69af1a39e3934236883aaa506f999dee" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.794832  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.789741   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.793677 11355 leader_election.cc:290] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.795233 11328 raft_consensus.cc:515] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.796008 11351 raft_consensus.cc:493] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.796728 11351 raft_consensus.cc:515] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.796113   431 raft_consensus.cc:2243] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.796170 11355 raft_consensus.cc:493] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.798957   431 raft_consensus.cc:2272] T 158c95c12c924f2ea6752771f4a48f0f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.799293 11348 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:07.798892 11355 raft_consensus.cc:515] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.797889 11328 leader_election.cc:290] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.798808 11351 leader_election.cc:290] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:07.803828 11355 leader_election.cc:290] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:07.799885 11348 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.816432 11336 leader_election.cc:290] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:07.818495   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.819734   431 raft_consensus.cc:2243] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:07.820576   431 raft_consensus.cc:2272] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.820245 11348 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.794035 11292 raft_consensus.cc:493] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e087ae2ce3f417093313edd26b8b3ee)
I20260709 13:06:07.829630 10841 leader_election.cc:304] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.827522 11292 raft_consensus.cc:515] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.830543 11328 raft_consensus.cc:2749] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:07.831564 10841 leader_election.cc:304] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.832031 11292 leader_election.cc:290] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:07.834278 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a08704c3c3c4030bb48886795597fe1" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.836864 11351 raft_consensus.cc:2749] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:07.837515   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.838215   431 raft_consensus.cc:2243] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.839015 10841 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.841840  9977 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:07.842823 11357 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:07.843178 11357 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:07.843508 11357 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.846009  9903 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:07.846894 11334 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:07.847245 11334 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:07.847612 11334 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.853004 10831 leader_election.cc:304] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
W20260709 13:06:07.853897 10831 raft_consensus.cc:2686] T c416362f795e46e28a60937c9ef5164f P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.855459   431 raft_consensus.cc:2272] T b3695e4783714136ad332a175348e9fc P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.859110   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.859898 11292 raft_consensus.cc:2749] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:07.861212   431 raft_consensus.cc:2243] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:07.861449 10823 consensus_peers.cc:183] SignalRequest failed: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.861892   431 raft_consensus.cc:2272] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.860044 10841 leader_election.cc:304] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
W20260709 13:06:07.863746 10841 raft_consensus.cc:2686] T 6bf82e92a7f845f4a7c658d0783ea4a5 P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.864720 10841 leader_election.cc:304] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.864853 11292 raft_consensus.cc:493] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d02e8a054a634be599149db641901736)
I20260709 13:06:07.865495   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:07.865638 10841 raft_consensus.cc:2686] T 69af1a39e3934236883aaa506f999dee P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.865540 11292 raft_consensus.cc:515] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.866585   431 raft_consensus.cc:2243] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.866655 10841 leader_election.cc:304] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:07.868753 11359 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:07.869434 11359 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:07.872586 11359 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:07.868180 11292 leader_election.cc:290] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:07.874819  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.876482   431 raft_consensus.cc:2272] T 04298f45db464a09bd0d2b82492e61b3 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.880215   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.888463 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.889089   431 raft_consensus.cc:2243] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:07.889885   431 raft_consensus.cc:2272] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.880658 11292 raft_consensus.cc:2749] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:07.891625  9938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.891989  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.891523 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12cd0294fa614c61b200754ffbbf7601" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.893229  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd7ef372f79649dda9b18919bfc39298" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.895339 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.880347 11357 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.882139  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1078585b45d74186be92d54ac68f787a" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.874215 11334 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:07.899749 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.883670  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625608f2b39c495d9f5b471dcabde2de" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.884267  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.883880  9864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79129676db004c7bbe29ab7e9ec1fbe3" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.901953 11334 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:07.887117  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.904474 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:07.897475   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:07.908126  9901 leader_election.cc:336] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:07.906303  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2d6cf525014f80a8f95687135fff6f" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.906553  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.909765 10006 raft_consensus.cc:2393] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 3: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:06:07.907696 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eab4bb2719246f2bf5af8bbccae641d" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.897727  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16d91cae140a4615a015d3d34ac878c4" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:06:07.906497  9975 leader_election.cc:336] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:07.913405   431 raft_consensus.cc:2243] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.904709 11357 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:07.914321   431 raft_consensus.cc:2272] T 4e31a807457a433b9c28f7a67c98095c P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.915460  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:07.916322  9935 raft_consensus.cc:2393] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e087ae2ce3f417093313edd26b8b3ee in current term 3: Already voted for candidate d02e8a054a634be599149db641901736 in this term.
I20260709 13:06:07.918695  9977 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:07.920603 11357 raft_consensus.cc:2749] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:07.921957 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.925877  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca2ecb398d14bd48a7c452af841da65" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.938143   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.939199 11359 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:07.939929   431 raft_consensus.cc:2243] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.939874 11359 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:07.940888   431 raft_consensus.cc:2272] T f8a86072500f497585b83ac124a8a562 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.942360 11359 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:07.943908  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:07.946023   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.946540 10831 leader_election.cc:304] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.947044   431 raft_consensus.cc:2243] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:07.947217 10831 raft_consensus.cc:2686] T 79129676db004c7bbe29ab7e9ec1fbe3 P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.947782   431 raft_consensus.cc:2272] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.948376 10831 leader_election.cc:304] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
W20260709 13:06:07.949045 10831 raft_consensus.cc:2686] T bd7ef372f79649dda9b18919bfc39298 P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.952442   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.953845   431 raft_consensus.cc:2243] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:07.954553   431 raft_consensus.cc:2272] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.958817   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.960989  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12836e61c6d4427a5d883c4bdf5b5b6" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:07.967108 10831 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.968088 11359 raft_consensus.cc:2749] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:07.969897 10831 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:07.972681 11359 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:07.979847 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "877ba104634d4068a2f26aa62d860b7b" candidate_uuid: "e8769fa7061d433bb90a9411c71fa0df" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:07.967319   431 raft_consensus.cc:2243] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.981621   431 raft_consensus.cc:2272] T 404c2d54fc4e4b6c8a892de6e4988671 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.985570   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.985922 10831 leader_election.cc:304] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:07.986569   431 raft_consensus.cc:2243] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:07.986863 11359 raft_consensus.cc:2749] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:07.987349   431 raft_consensus.cc:2272] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:07.991644 10831 leader_election.cc:304] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:06:07.992605 11359 raft_consensus.cc:2749] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:07.990787   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:07.993988 10831 leader_election.cc:304] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:06:07.994163   431 raft_consensus.cc:2243] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:07.995360 11359 raft_consensus.cc:2749] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:07.996667 10831 leader_election.cc:304] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
W20260709 13:06:07.997822 10831 raft_consensus.cc:2686] T 625608f2b39c495d9f5b471dcabde2de P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:07.999830 10831 leader_election.cc:304] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:06:07.999948   431 raft_consensus.cc:2272] T 4716f93906794e73b45089786b4a75f0 P e8769fa7061d433bb90a9411c71fa0df [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.000629 10831 raft_consensus.cc:2686] T 7eab4bb2719246f2bf5af8bbccae641d P e8769fa7061d433bb90a9411c71fa0df: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:08.002108 10831 leader_election.cc:304] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736
I20260709 13:06:08.002892 11359 raft_consensus.cc:2749] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:08.003644 10831 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8769fa7061d433bb90a9411c71fa0df; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:08.004226   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.004530 11359 raft_consensus.cc:2749] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:08.005498   431 raft_consensus.cc:2243] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.006237   431 raft_consensus.cc:2272] T 4a2a7189703643899a38fc9b896d3c78 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.008522  9901 leader_election.cc:336] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:08.009092  9901 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:08.009678   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.009910 11334 raft_consensus.cc:2749] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:06:08.010808   431 raft_consensus.cc:2243] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.011437   431 raft_consensus.cc:2272] T 4a08704c3c3c4030bb48886795597fe1 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.014706   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.015611   431 raft_consensus.cc:2243] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.016301   431 raft_consensus.cc:2272] T 12cd0294fa614c61b200754ffbbf7601 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.019397   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.020032   431 raft_consensus.cc:2243] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.020527   431 raft_consensus.cc:2272] T 1078585b45d74186be92d54ac68f787a P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.022891   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.023494   431 raft_consensus.cc:2243] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.023958   431 raft_consensus.cc:2272] T 6bab397fa40041c281ce616134672828 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.026943   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.027570   431 raft_consensus.cc:2243] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.028214   431 raft_consensus.cc:2272] T 877ba104634d4068a2f26aa62d860b7b P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.031369   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.032536   431 raft_consensus.cc:2243] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.033963   431 raft_consensus.cc:2272] T 22c6e691c1264b8e8de49c1df4cd8bb5 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.036996   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.037673   431 raft_consensus.cc:2243] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.038190   431 raft_consensus.cc:2272] T 0c2d6cf525014f80a8f95687135fff6f P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.041181   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.041987   431 raft_consensus.cc:2243] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.042639   431 raft_consensus.cc:2272] T 16d91cae140a4615a015d3d34ac878c4 P e8769fa7061d433bb90a9411c71fa0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.046178   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.046859   431 raft_consensus.cc:2243] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.048019   431 raft_consensus.cc:2272] T f5dd44a42946450e936b89718864c5e3 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.051910   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.052636   431 raft_consensus.cc:2243] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.053211   431 raft_consensus.cc:2272] T fca2ecb398d14bd48a7c452af841da65 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.054740 11334 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.055601 11334 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:08.056139   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.056911   431 raft_consensus.cc:2243] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.057998   431 raft_consensus.cc:2272] T 1ad9129f493c417eaa0d52805e5b1144 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.059578  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d" is_pre_election: true
I20260709 13:06:08.060437  9862 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:06:08.061609   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:08.062392  9904 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:08.062407   431 raft_consensus.cc:2243] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.063377 11236 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:08.063782 11236 raft_consensus.cc:493] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.064273 11236 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:08.065210 11334 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:08.063621   431 raft_consensus.cc:2272] T e12836e61c6d4427a5d883c4bdf5b5b6 P e8769fa7061d433bb90a9411c71fa0df [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.070180   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:06:08.070487  9901 leader_election.cc:336] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:08.071007   431 raft_consensus.cc:2243] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.071825   431 raft_consensus.cc:2272] T b5fa7348f6384183abeb9619f1fd0d75 P e8769fa7061d433bb90a9411c71fa0df [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.074606 11236 raft_consensus.cc:515] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:08.077086 11236 leader_election.cc:290] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: Requested vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:08.081463  9901 leader_election.cc:336] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:08.084821  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f50ff8860ba431d9ee421ff76612df6" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "24c6a28138294e46bf6b3d5ba324046d"
I20260709 13:06:08.085554  9862 raft_consensus.cc:3060] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:08.093748  9862 raft_consensus.cc:2468] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 4.
I20260709 13:06:08.094923  9904 leader_election.cc:304] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:08.096148 11236 raft_consensus.cc:2804] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:08.097600 11236 raft_consensus.cc:697] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 4 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:08.098560 11236 consensus_queue.cc:237] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:08.124164  9686 catalog_manager.cc:5697] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 3 to 4, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.193112   431 tablet_server.cc:196] TabletServer@127.0.107.193:32801 shutdown complete.
I20260709 13:06:08.241225 11347 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.242094 11347 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:08.248788  9935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:08.262387 11347 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:06:08.291643  9975 leader_election.cc:336] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:08.308403  9975 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:08.311453 11349 raft_consensus.cc:2749] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:08.340179   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:08.345786  9978 consensus_peers.cc:597] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.362826  9904 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.369191  9978 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.388779  9904 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.407394  9978 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.411736   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:08.412933   431 tablet_replica.cc:333] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.413964   431 raft_consensus.cc:2243] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.415104   431 raft_consensus.cc:2272] T fca2ecb398d14bd48a7c452af841da65 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.418298   431 tablet_replica.cc:333] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.419247   431 raft_consensus.cc:2243] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.419916   431 raft_consensus.cc:2272] T c68f14a359d54249b7cab91d0e63d7b0 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.423091   431 tablet_replica.cc:333] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.423928   431 raft_consensus.cc:2243] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.425210   431 raft_consensus.cc:2272] T 9cb3e93354824df58589f250b6264d54 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.433110   431 tablet_replica.cc:333] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.434593   431 raft_consensus.cc:2243] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.435482   431 raft_consensus.cc:2272] T 79129676db004c7bbe29ab7e9ec1fbe3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.438990   431 tablet_replica.cc:333] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.440039   431 raft_consensus.cc:2243] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.440864   431 raft_consensus.cc:2272] T bd7ef372f79649dda9b18919bfc39298 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.441802  9978 consensus_peers.cc:597] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.444427   431 tablet_replica.cc:333] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.445533   431 raft_consensus.cc:2243] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.446223   431 raft_consensus.cc:2272] T 82893adc39f24309a241ee3e5ff6611e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.449769   431 tablet_replica.cc:333] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.450865   431 raft_consensus.cc:2243] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.451663   431 raft_consensus.cc:2272] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.455372   431 tablet_replica.cc:333] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.456487   431 raft_consensus.cc:2243] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.457083   431 raft_consensus.cc:2272] T 04298f45db464a09bd0d2b82492e61b3 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.459967 11344 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.460413   431 tablet_replica.cc:333] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.460652 11344 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:08.461551   431 raft_consensus.cc:2243] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:08.461624  9978 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.462796   431 raft_consensus.cc:2272] T 1801426ac1e84ebaadda531ec03958bd P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.463001 11344 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:08.463652 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:08.464428 10008 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:08.465811  9903 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:08.466738 11344 raft_consensus.cc:2804] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:08.466779   431 tablet_replica.cc:333] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.467242 11344 raft_consensus.cc:493] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.468015   431 raft_consensus.cc:2243] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.468885   431 raft_consensus.cc:2272] T 4125c95bf84c45538728679b308f6aca P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.471429  9901 leader_election.cc:336] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:08.472211   431 tablet_replica.cc:333] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.467654 11344 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:08.473344   431 raft_consensus.cc:2243] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.474242   431 raft_consensus.cc:2272] T 4a2a7189703643899a38fc9b896d3c78 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.477614   431 tablet_replica.cc:333] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.478683   431 raft_consensus.cc:2243] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.479861   431 raft_consensus.cc:2272] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.483836   431 tablet_replica.cc:333] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.484769   431 raft_consensus.cc:2243] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.485441   431 raft_consensus.cc:2272] T b17aba5ba2fe46a49bec25310d428a26 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.488708   431 tablet_replica.cc:333] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
W20260709 13:06:08.489081  9904 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.489526   431 raft_consensus.cc:2243] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.490162   431 raft_consensus.cc:2272] T 6bab397fa40041c281ce616134672828 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.493551   431 tablet_replica.cc:333] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.494758   431 raft_consensus.cc:2243] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.494923 11236 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.495460   431 raft_consensus.cc:2272] T 1078585b45d74186be92d54ac68f787a P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.497601 11344 raft_consensus.cc:515] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:08.495477 11236 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:08.499112   431 tablet_replica.cc:333] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.500138   431 raft_consensus.cc:2243] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.501190 11347 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:08.501309 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62c671eeefb49758edb7a7273fc6a2c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
W20260709 13:06:08.502543  9904 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.502671 10008 raft_consensus.cc:3060] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:08.500480 11236 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:08.502501 11347 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:08.501600 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:08.505378 11347 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:08.501663 11344 leader_election.cc:290] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:06:08.509626  9978 leader_election.cc:336] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:06:08.500803   431 raft_consensus.cc:2272] T d3324f833b7b462d8aa8db06fb4bd74e P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.514961   431 tablet_replica.cc:333] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.515941   431 raft_consensus.cc:2243] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.516996   431 raft_consensus.cc:2272] T b5fa7348f6384183abeb9619f1fd0d75 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.519286  9975 leader_election.cc:336] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:08.519744  9975 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:08.520409   431 tablet_replica.cc:333] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.521481   431 raft_consensus.cc:2243] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.523089   431 raft_consensus.cc:2272] T eb7d115ef55d410a88514a918dee8efc P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.511339 10008 raft_consensus.cc:2468] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:06:08.524271 11347 raft_consensus.cc:2749] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:08.525735  9903 leader_election.cc:304] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:08.527019   431 tablet_replica.cc:333] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.528296   431 raft_consensus.cc:2243] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.530361   431 raft_consensus.cc:2272] T 93fbf294836d47b29e0143174bbb3df4 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.528442  9901 leader_election.cc:336] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:08.530959  9901 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:08.531710 11334 raft_consensus.cc:2749] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:08.532713  9901 leader_election.cc:336] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:08.534050   431 tablet_replica.cc:333] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
W20260709 13:06:08.534600  9904 consensus_peers.cc:597] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.535228   431 raft_consensus.cc:2243] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.530552 11236 raft_consensus.cc:2804] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:08.536365 11236 raft_consensus.cc:697] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 3 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:08.536988   431 raft_consensus.cc:2272] T e63bec87e7a64ac0bb9eea74f15908ba P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.537210 11236 consensus_queue.cc:237] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:08.539196  9978 consensus_peers.cc:597] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.541122   431 tablet_replica.cc:333] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.542377   431 raft_consensus.cc:2243] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.543159   431 raft_consensus.cc:2272] T f5dd44a42946450e936b89718864c5e3 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.544289  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8040 after lost signal to thread 434
W20260709 13:06:08.546272  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08002b32a0 after lost signal to thread 9721
W20260709 13:06:08.547851  9885 debug-util.cc:398] Leaking SignalData structure 0x7b080004ece0 after lost signal to thread 9888
W20260709 13:06:08.549846  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08000fa600 after lost signal to thread 9962
W20260709 13:06:08.551100  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6080 after lost signal to thread 10036
W20260709 13:06:08.551663  9885 debug-util.cc:398] Leaking SignalData structure 0x7b08001974c0 after lost signal to thread 10990
W20260709 13:06:08.552802  9904 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.556618  9904 consensus_peers.cc:597] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.559713   431 tablet_replica.cc:333] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.561021   431 raft_consensus.cc:2243] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.562551   431 raft_consensus.cc:2272] T 0c2d6cf525014f80a8f95687135fff6f P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.568315   431 tablet_replica.cc:333] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.567648  9686 catalog_manager.cc:5697] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 reported cstate change: term changed from 2 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:08.571264   431 raft_consensus.cc:2243] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.572476   431 raft_consensus.cc:2272] T e18d9519bce7451aaee1e04623f7fddb P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.576990   431 tablet_replica.cc:333] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.579944   431 raft_consensus.cc:2243] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.581377   431 raft_consensus.cc:2272] T f0bdd04f0ff44f2e87f76f36a7c5e540 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.585199   431 tablet_replica.cc:333] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.586156   431 raft_consensus.cc:2243] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.586814   431 raft_consensus.cc:2272] T e9ea20d35c3b4150ba41cec32b7c9e0c P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.590699   431 tablet_replica.cc:333] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.591516   431 raft_consensus.cc:2243] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.592455   431 raft_consensus.cc:2272] T 582cebb9b4674942bb40b7be9936c3f5 P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.596863   431 tablet_replica.cc:333] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.597652   431 raft_consensus.cc:2243] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.598827   431 raft_consensus.cc:2272] T 1131589c5bd9480d838a7967131195ca P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.604228   431 tablet_replica.cc:333] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.605918   431 raft_consensus.cc:2243] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.606761   431 raft_consensus.cc:2272] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.610376   431 tablet_replica.cc:333] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.611671   431 raft_consensus.cc:2243] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.612846   431 raft_consensus.cc:2272] T c48883428dea42d7b95a7cddbc07178e P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.617051   431 tablet_replica.cc:333] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.618064   431 raft_consensus.cc:2243] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.619442   431 raft_consensus.cc:2272] T 877ba104634d4068a2f26aa62d860b7b P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.623251   431 tablet_replica.cc:333] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.624322   431 raft_consensus.cc:2243] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.625730   431 raft_consensus.cc:2272] T 69af1a39e3934236883aaa506f999dee P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.631404   431 tablet_replica.cc:333] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.633190   431 raft_consensus.cc:2243] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.634148   431 raft_consensus.cc:2272] T 178dcd0da5414a9785794b2412a9cf93 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.637028  9904 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.638427   431 tablet_replica.cc:333] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
W20260709 13:06:08.639457  9978 consensus_peers.cc:597] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.639699   431 raft_consensus.cc:2243] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:08.641060   431 raft_consensus.cc:2272] T 82999b9832894f40bb6dbe02e2feb1df P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.645571   431 tablet_replica.cc:333] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.646742   431 raft_consensus.cc:2243] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.647441   431 raft_consensus.cc:2272] T 47be166313094f0fbe17bde042f05dfe P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.651103   431 tablet_replica.cc:333] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.652434   431 raft_consensus.cc:2243] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.653184   431 raft_consensus.cc:2272] T a693233cb65a40089d314773895f84a0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.657059   431 tablet_replica.cc:333] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.658027   431 raft_consensus.cc:2243] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.658697   431 raft_consensus.cc:2272] T 0f50ff8860ba431d9ee421ff76612df6 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.662569   431 tablet_replica.cc:333] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.664009   431 raft_consensus.cc:2243] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.665130   431 raft_consensus.cc:2272] T 5673ca542b1745a38fdc5b65d4357b42 P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.681481   431 tablet_replica.cc:333] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.682433   431 raft_consensus.cc:2243] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.683220   431 raft_consensus.cc:2272] T 4a08704c3c3c4030bb48886795597fe1 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.687649   431 tablet_replica.cc:333] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.700635   431 raft_consensus.cc:2243] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.701545   431 raft_consensus.cc:2272] T d775d5d3993743689257e4509a059770 P 24c6a28138294e46bf6b3d5ba324046d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.707881   431 tablet_replica.cc:333] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.709862   431 raft_consensus.cc:2243] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.711201   431 raft_consensus.cc:2272] T 1cfdee35c2f0491da78cbc51ed63ba5d P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.709364  9978 consensus_peers.cc:597] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.728572   431 tablet_replica.cc:333] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.729414   431 raft_consensus.cc:2243] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.730031   431 raft_consensus.cc:2272] T b253ddcc7d6046c6abdd94aec212612c P 24c6a28138294e46bf6b3d5ba324046d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.735046   431 tablet_replica.cc:333] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
W20260709 13:06:08.735198  9904 consensus_peers.cc:597] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.736219   431 raft_consensus.cc:2243] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.749296   431 raft_consensus.cc:2272] T 404c2d54fc4e4b6c8a892de6e4988671 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:08.752455   431 tablet_replica.cc:333] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.753943   431 raft_consensus.cc:2243] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.754673   431 raft_consensus.cc:2272] T 625608f2b39c495d9f5b471dcabde2de P 24c6a28138294e46bf6b3d5ba324046d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.754958  9975 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.764879  9901 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:08.768836   431 tablet_replica.cc:333] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d: stopping tablet replica
I20260709 13:06:08.769646   431 raft_consensus.cc:2243] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:08.770289   431 raft_consensus.cc:2272] T 4716f93906794e73b45089786b4a75f0 P 24c6a28138294e46bf6b3d5ba324046d [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:08.814020  9978 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.816234  9904 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.824788  9904 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.868214  9901 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:08.889396  9904 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.890637  9978 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.923357  9904 consensus_peers.cc:597] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.952755  9901 consensus_peers.cc:597] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:08.984719  9904 connection.cc:573] client connection to 127.0.107.194:32919 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:06:09.011363  9975 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.025802  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.026365  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.039417  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.073048  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.077374  9901 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.106046 10008 raft_consensus.cc:1275] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:09.107851 11373 consensus_queue.cc:1048] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:09.181634  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.184125  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.188242  9901 consensus_peers.cc:597] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.204502  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:09.212831  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.225263 11357 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.225904 11357 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.229035  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.230576 11357 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:06:09.237093  9975 leader_election.cc:336] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.238857  9975 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.240170 11358 raft_consensus.cc:2749] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:09.242254  9901 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.283946 11344 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.284541 11344 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.287746 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.288501 10008 raft_consensus.cc:2468] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:09.291899  9903 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.292760 11373 raft_consensus.cc:2804] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.293139 11373 raft_consensus.cc:493] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.293534 11373 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:09.295562  9901 leader_election.cc:336] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.296717 11344 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.301451 11373 raft_consensus.cc:515] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.303778 11373 leader_election.cc:290] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.305631 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "158c95c12c924f2ea6752771f4a48f0f" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:09.306289 10008 raft_consensus.cc:3060] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:09.309897  9901 leader_election.cc:336] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.314098 10008 raft_consensus.cc:2468] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:06:09.315470  9903 leader_election.cc:304] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.316362 11373 raft_consensus.cc:2804] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:09.317044 11373 raft_consensus.cc:697] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 3 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:09.318238 11373 consensus_queue.cc:237] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.330883  9685 catalog_manager.cc:5697] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 reported cstate change: term changed from 2 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.332693 11358 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.333412 11358 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.335960 11358 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:09.341602  9975 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:09.348227  9978 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.354590 11357 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.355185 11357 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.357402 11357 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.355834  9978 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.358870 11357 raft_consensus.cc:2749] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:09.359259  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.360186  9933 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:09.362039  9977 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.363606 11357 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:09.367249 11357 raft_consensus.cc:493] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.367870 11357 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:09.369485  9975 leader_election.cc:336] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:09.371325  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.385330 11358 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.386505 11236 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.387071 11236 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:06:09.388201  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.388530 11357 raft_consensus.cc:515] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.389102 11344 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.389612 11344 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.390789 11357 leader_election.cc:290] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.391284 11349 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.392901 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.392807 11349 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.391659 11344 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.386090 11358 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.398768  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3695e4783714136ad332a175348e9fc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
W20260709 13:06:09.399214  9901 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.399856  9933 raft_consensus.cc:3060] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:09.400281 11358 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.389972 11236 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.401089  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:06:09.403553  9901 leader_election.cc:336] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.404021  9901 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.404729 11344 raft_consensus.cc:2749] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.405645 11358 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.406291 11358 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.408080 11344 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.409056 11373 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.409507 11358 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.409566 11373 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.414018 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.411729 11373 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.408649 11344 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.421084 11344 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.421875 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.422947 10008 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:09.423635 11349 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.424798  9903 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.428416 11373 raft_consensus.cc:2804] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.428828 11373 raft_consensus.cc:493] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.429232 11373 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:09.421859  9978 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.434541  9904 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.438297  9901 leader_election.cc:336] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.438553  9904 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.439459 11236 raft_consensus.cc:2749] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:09.437774  9978 leader_election.cc:336] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.438017 11344 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.443481 11344 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.446661  9933 raft_consensus.cc:2468] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 4.
I20260709 13:06:09.448103  9977 leader_election.cc:304] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.438772  9901 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 5e087ae2ce3f417093313edd26b8b3ee, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.436831 11373 raft_consensus.cc:515] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.449177 11236 raft_consensus.cc:2749] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.450644 11373 leader_election.cc:290] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Requested vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.453563 11344 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:06:09.454262  9904 leader_election.cc:336] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.453074  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.454453 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a86072500f497585b83ac124a8a562" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
W20260709 13:06:09.455137  9975 leader_election.cc:336] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.455492 10006 raft_consensus.cc:3060] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:09.455807  9975 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.455535 10008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.456372  9975 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.457147 11357 raft_consensus.cc:2749] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:09.457976  9903 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:09.454357 11358 raft_consensus.cc:2804] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:09.459161 11358 raft_consensus.cc:697] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 4 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:09.459498 11344 raft_consensus.cc:2749] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.460322 11358 consensus_queue.cc:237] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.463291 10006 raft_consensus.cc:2468] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:06:09.465215 11349 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.466497 11349 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.458139  9975 leader_election.cc:336] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.469466  9975 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: d02e8a054a634be599149db641901736, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.470997 11358 raft_consensus.cc:2749] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.465214  9903 leader_election.cc:304] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.473446 11373 raft_consensus.cc:2804] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:06:09.471175  9901 leader_election.cc:336] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.474054 11373 raft_consensus.cc:697] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 3 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:09.474978 11373 consensus_queue.cc:237] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.475853  9975 leader_election.cc:336] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.476539  9975 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.477581  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.478349  9933 raft_consensus.cc:2468] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:06:09.480935  9977 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.480054 11357 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.482154 11357 raft_consensus.cc:2804] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.482570 11357 raft_consensus.cc:493] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.482065  9685 catalog_manager.cc:5697] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 3 to 4, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 4 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.483925 11349 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.482928 11357 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.487229 11349 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.487785 11349 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.484771 11344 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.490520 11349 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.490736 11344 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:09.495400 11344 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.496027 11357 raft_consensus.cc:515] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.499648  9978 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.503818  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e31a807457a433b9c28f7a67c98095c" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:09.504478  9933 raft_consensus.cc:3060] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.517127  9933 raft_consensus.cc:2468] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:09.519438  9977 leader_election.cc:304] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.520510 11349 raft_consensus.cc:2804] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:09.518158 11357 leader_election.cc:290] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.522511 11236 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.524567 11236 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.521347 11349 raft_consensus.cc:697] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:09.529933 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.530303 11349 consensus_queue.cc:237] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.532289 11236 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:06:09.535933  9975 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.536628  9975 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.537472 11357 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:09.537854  9975 leader_election.cc:336] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.521225  9686 catalog_manager.cc:5697] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 2 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:09.539681  9901 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.540282 11347 raft_consensus.cc:493] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
W20260709 13:06:09.540328  9901 leader_election.cc:336] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.541316 11347 raft_consensus.cc:515] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.543771 11347 leader_election.cc:290] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:06:09.544801  9904 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.546096 11347 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
W20260709 13:06:09.541731  9901 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.546710 11347 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.550664 11347 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:09.550876  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
W20260709 13:06:09.551508  9975 leader_election.cc:336] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.551779  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.556298 11347 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
W20260709 13:06:09.556751  9904 leader_election.cc:336] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.557850 11344 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.558151 11347 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.564456  9978 leader_election.cc:336] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.558565  9904 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:09.566869 11334 raft_consensus.cc:2749] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.565107  9978 leader_election.cc:304] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:09.558670 11344 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:09.569963 11357 raft_consensus.cc:2749] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:09.571496  9978 leader_election.cc:336] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.559726  9901 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.572045  9978 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, d02e8a054a634be599149db641901736
I20260709 13:06:09.572214  9901 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.572986 11236 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.573226  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.574499  9933 raft_consensus.cc:2468] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:06:09.575855  9977 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.576892 11236 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.577618 11236 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:06:09.578738  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.580116 11344 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.580823 11236 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:09.582341 11357 raft_consensus.cc:2749] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:09.583779 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:09.590778 10006 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 3.
I20260709 13:06:09.583767 11358 raft_consensus.cc:2804] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.592001 11358 raft_consensus.cc:493] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:09.592507 11358 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.596632  9903 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.598410 11344 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:09.598882 11344 raft_consensus.cc:493] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.599289 11344 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:06:09.605183  9901 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:09.608289  9686 catalog_manager.cc:5697] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 2 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.578158 11347 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:06:09.612075  9901 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:09.613224  9975 leader_election.cc:336] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.614650 11358 raft_consensus.cc:515] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.635131  9904 leader_election.cc:336] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.635411  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad9129f493c417eaa0d52805e5b1144" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:09.636241  9933 raft_consensus.cc:3060] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:09.635989  9904 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.639616  9904 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
W20260709 13:06:09.640286  9975 leader_election.cc:336] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.641733 11349 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.642400 11349 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.645222 11349 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:09.647599  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.648566  9936 raft_consensus.cc:2468] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:06:09.650257  9977 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.651464 11357 raft_consensus.cc:2804] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:09.652012 11357 raft_consensus.cc:493] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.652318 11357 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.659194 11334 raft_consensus.cc:2749] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.659624 11344 raft_consensus.cc:515] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.666211 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93fbf294836d47b29e0143174bbb3df4" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:09.666556 11344 leader_election.cc:290] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:09.667399 10006 raft_consensus.cc:3060] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:09.668725 11344 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.669292 11357 raft_consensus.cc:515] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.669657 11344 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:09.676882 11344 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.679395  9933 raft_consensus.cc:2468] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:09.680907  9977 leader_election.cc:304] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: e8769fa7061d433bb90a9411c71fa0df
W20260709 13:06:09.681950  9904 leader_election.cc:336] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.682621 11347 raft_consensus.cc:2804] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:09.684446 11347 raft_consensus.cc:697] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:09.685201 11358 leader_election.cc:290] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.686787 11347 consensus_queue.cc:237] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.688978  9904 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.690537  9978 leader_election.cc:336] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.682126 10006 raft_consensus.cc:2468] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 4.
W20260709 13:06:09.695995  9978 leader_election.cc:336] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.697275  9903 leader_election.cc:304] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:06:09.698948 11236 raft_consensus.cc:2804] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:09.699520 11236 raft_consensus.cc:697] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 4 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
I20260709 13:06:09.700489 11236 consensus_queue.cc:237] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:06:09.700088  9901 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.704134  9901 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.705435 11357 leader_election.cc:290] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:09.706430 11344 raft_consensus.cc:2749] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.708067  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63bec87e7a64ac0bb9eea74f15908ba" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:09.708740  9933 raft_consensus.cc:3060] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.712605  9686 catalog_manager.cc:5697] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 2 to 3, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.717154 11357 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.717803 11357 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.719959 11357 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.728331  9933 raft_consensus.cc:2468] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:09.730520  9977 leader_election.cc:304] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:06:09.732065 11357 raft_consensus.cc:2804] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:09.733057 11357 raft_consensus.cc:697] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:09.734340 11347 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.734436 11357 consensus_queue.cc:237] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.734916 11347 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
W20260709 13:06:09.736917  9975 leader_election.cc:336] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.740923  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.741894  9936 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:09.743402  9977 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.751199 11358 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:09.751883 11358 raft_consensus.cc:493] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.753338 11358 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:09.754009 11347 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
W20260709 13:06:09.760675  9978 leader_election.cc:336] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.761488  9978 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.763198 11347 raft_consensus.cc:2749] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.761195  9686 catalog_manager.cc:5697] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 reported cstate change: term changed from 3 to 4, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.779584 10006 raft_consensus.cc:1275] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:09.785032 11344 consensus_queue.cc:1048] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:09.795780  9975 leader_election.cc:336] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.797185 11358 raft_consensus.cc:515] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.801414  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a7ca5934d4fb6958de3cca90f5a36" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:09.802399  9933 raft_consensus.cc:3060] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:09.817402  9686 catalog_manager.cc:5697] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 2 to 3, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.821290  9933 raft_consensus.cc:2468] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 4.
I20260709 13:06:09.823678  9977 leader_election.cc:304] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.825416 11357 raft_consensus.cc:2804] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:09.826231 11357 raft_consensus.cc:697] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:09.826823 11358 leader_election.cc:290] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: Requested vote from peers d02e8a054a634be599149db641901736 (127.0.107.195:37607), e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801)
I20260709 13:06:09.827237 11357 consensus_queue.cc:237] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } }
I20260709 13:06:09.845572 11334 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.848627 11334 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
W20260709 13:06:09.849822  9975 leader_election.cc:336] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.864432 11334 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.871842 11357 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.872560 11357 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:06:09.874032  9904 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.880837  9978 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.889170  9901 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.890285  9901 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.893442 11357 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.894829 11373 raft_consensus.cc:2749] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:09.897516  9975 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.898065  9975 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.899230 11357 raft_consensus.cc:2749] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.901659 10006 raft_consensus.cc:1275] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:09.903782 11373 consensus_queue.cc:1048] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:09.934399 11357 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.935052 11357 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.939306  9933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:09.940183  9933 raft_consensus.cc:2468] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
I20260709 13:06:09.941957  9977 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.943111 11391 raft_consensus.cc:2804] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:09.944082  9978 leader_election.cc:336] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:09.944168 11391 raft_consensus.cc:493] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.945387 11391 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.950901 11358 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.951519 11358 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.953217 11357 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:09.954313 11358 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:09.960601  9975 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.961164 11390 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.961751 11390 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:09.964805  9933 raft_consensus.cc:1275] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:09.967265 11358 consensus_queue.cc:1048] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:09.969062  9978 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:09.969247  9975 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:09.969594  9978 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:09.970950 11357 raft_consensus.cc:2749] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:09.973616 11390 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:09.974517 11391 raft_consensus.cc:515] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:09.978125 11357 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:09.978209  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1801426ac1e84ebaadda531ec03958bd" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
I20260709 13:06:09.979001  9936 raft_consensus.cc:3060] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:09.978917 11357 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:09.986788  9936 raft_consensus.cc:2468] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:09.986207  9686 catalog_manager.cc:5697] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 2 to 4, leader changed from e8769fa7061d433bb90a9411c71fa0df (127.0.107.193) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 4 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.988749  9977 leader_election.cc:304] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:09.992183 11358 raft_consensus.cc:2804] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:09.992794 11358 raft_consensus.cc:697] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:09.993218 11391 leader_election.cc:290] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:09.994282 11358 consensus_queue.cc:237] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:06:09.996733   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9885)
W20260709 13:06:10.012198  9975 consensus_peers.cc:597] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.019495 11334 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:10.020576 11334 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:10.024091 11357 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
W20260709 13:06:10.033427  9978 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.034345  9978 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:10.039331 11236 raft_consensus.cc:493] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:10.040041 11236 raft_consensus.cc:515] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:10.042102 11236 leader_election.cc:290] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:10.052513  9904 leader_election.cc:336] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.061151  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736" is_pre_election: true
I20260709 13:06:10.061895  9936 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 2.
W20260709 13:06:10.063736  9901 consensus_peers.cc:597] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.065780  9936 raft_consensus.cc:1275] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:10.063855 11357 raft_consensus.cc:2749] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:10.064129  9978 leader_election.cc:336] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.071710 11334 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:10.072885  9978 leader_election.cc:336] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.065047 11373 raft_consensus.cc:493] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:10.075672 11373 raft_consensus.cc:515] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
W20260709 13:06:10.077594  9904 leader_election.cc:336] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.078524 11373 leader_election.cc:290] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
W20260709 13:06:10.083205  9975 consensus_peers.cc:597] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.087029  9977 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:06:10.089861 11357 raft_consensus.cc:2804] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:10.090270 11357 raft_consensus.cc:493] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:10.090642 11357 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:10.091389  9904 leader_election.cc:336] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:10.083547  9901 consensus_peers.cc:597] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.095878 11390 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:10.096554 11390 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:10.098281 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:10.098966 11391 consensus_queue.cc:1048] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:10.101588 10012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee" is_pre_election: true
I20260709 13:06:10.102432 10012 raft_consensus.cc:2468] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 1.
W20260709 13:06:10.106029  9978 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.106693  9903 leader_election.cc:304] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:06:10.104498 11390 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:10.107542 11334 raft_consensus.cc:2804] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:10.108407 11334 raft_consensus.cc:493] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:10.108917 11334 raft_consensus.cc:3060] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.108065  9686 catalog_manager.cc:5697] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 2 to 3, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.111654 11357 raft_consensus.cc:515] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:10.112916 10006 raft_consensus.cc:2393] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d02e8a054a634be599149db641901736 in current term 3: Already voted for candidate 5e087ae2ce3f417093313edd26b8b3ee in this term.
I20260709 13:06:10.117301  9903 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, 5e087ae2ce3f417093313edd26b8b3ee
I20260709 13:06:10.118278 11344 raft_consensus.cc:3060] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.127944 11344 raft_consensus.cc:2749] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:10.130708 11357 leader_election.cc:290] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), d02e8a054a634be599149db641901736 (127.0.107.195:37607)
I20260709 13:06:10.132409  9936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c48883428dea42d7b95a7cddbc07178e" candidate_uuid: "5e087ae2ce3f417093313edd26b8b3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d02e8a054a634be599149db641901736"
W20260709 13:06:10.095498  9901 leader_election.cc:336] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:10.133713  9901 leader_election.cc:304] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:10.135182 11373 raft_consensus.cc:2749] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:10.147701  9936 raft_consensus.cc:2468] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e087ae2ce3f417093313edd26b8b3ee in term 3.
I20260709 13:06:10.150919 11334 raft_consensus.cc:515] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:10.150389  9977 leader_election.cc:304] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 
I20260709 13:06:10.153813 11358 raft_consensus.cc:2804] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:06:10.154129  9978 leader_election.cc:336] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.154711 11358 raft_consensus.cc:697] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Becoming Leader. State: Replica: 5e087ae2ce3f417093313edd26b8b3ee, State: Running, Role: LEADER
I20260709 13:06:10.156023 11358 consensus_queue.cc:237] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:10.160421 10006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7d115ef55d410a88514a918dee8efc" candidate_uuid: "d02e8a054a634be599149db641901736" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e087ae2ce3f417093313edd26b8b3ee"
I20260709 13:06:10.161100 10006 raft_consensus.cc:3060] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:10.164551  9904 leader_election.cc:336] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.166326 11334 leader_election.cc:290] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Requested vote from peers 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919), 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196:38131)
I20260709 13:06:10.169545 10006 raft_consensus.cc:2468] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02e8a054a634be599149db641901736 in term 2.
I20260709 13:06:10.170858  9903 leader_election.cc:304] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee, d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d
I20260709 13:06:10.171976 11334 raft_consensus.cc:2804] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:06:10.174999  9975 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:10.175541  9975 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:10.182974 11357 raft_consensus.cc:2749] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:10.195613 11334 raft_consensus.cc:697] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Becoming Leader. State: Replica: d02e8a054a634be599149db641901736, State: Running, Role: LEADER
W20260709 13:06:10.195706  9975 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.202905 11334 consensus_queue.cc:237] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } }
I20260709 13:06:10.213115  9686 catalog_manager.cc:5697] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee reported cstate change: term changed from 2 to 3, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to 5e087ae2ce3f417093313edd26b8b3ee (127.0.107.196). New cstate: current_term: 3 leader_uuid: "5e087ae2ce3f417093313edd26b8b3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.223623  9687 catalog_manager.cc:5697] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 reported cstate change: term changed from 1 to 2, leader changed from 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194) to d02e8a054a634be599149db641901736 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "d02e8a054a634be599149db641901736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:10.230614  9975 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.262108 10006 raft_consensus.cc:1275] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:10.263756 11334 consensus_queue.cc:1048] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:10.284081  9904 consensus_peers.cc:597] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.294090  9933 raft_consensus.cc:1275] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.296048  9933 raft_consensus.cc:1275] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.296598 11391 consensus_queue.cc:1048] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:10.298424 11358 consensus_queue.cc:1048] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:10.303432  9936 raft_consensus.cc:1275] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:10.305883 11391 consensus_queue.cc:1048] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:10.348618  9978 consensus_peers.cc:597] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.513540  9978 consensus_peers.cc:597] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.530318 11357 raft_consensus.cc:493] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:10.531188 11357 raft_consensus.cc:515] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:10.534859 11357 leader_election.cc:290] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:10.538393  9975 leader_election.cc:336] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:10.540104  9978 leader_election.cc:336] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:10.540648  9978 leader_election.cc:304] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:10.541420 11357 raft_consensus.cc:2749] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:10.580911  9978 consensus_peers.cc:597] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:10.595816 10006 raft_consensus.cc:1275] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Refusing update from remote peer d02e8a054a634be599149db641901736: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.597867 11373 consensus_queue.cc:1048] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:10.626932  9933 raft_consensus.cc:1275] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.628914 11357 consensus_queue.cc:1048] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:10.636210  9904 consensus_peers.cc:597] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:10.670063  9978 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.735970  9978 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.744025  9904 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:10.750541  9936 raft_consensus.cc:1275] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Refusing update from remote peer 5e087ae2ce3f417093313edd26b8b3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:10.752399 11358 consensus_queue.cc:1048] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:10.822146  9978 consensus_peers.cc:597] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.859754  9978 consensus_peers.cc:597] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.893740  9904 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.928766  9904 consensus_peers.cc:597] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.969218  9904 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:10.997536   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 9885)
W20260709 13:06:11.039355  9978 consensus_peers.cc:597] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.061395  9904 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.071463  9978 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.071264 11391 raft_consensus.cc:493] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:11.072472 11391 raft_consensus.cc:515] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:06:11.101608  9975 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:11.107800  9904 consensus_peers.cc:597] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.105901 11391 leader_election.cc:290] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.113616  9978 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.116688  9978 consensus_peers.cc:597] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.118467  9978 leader_election.cc:336] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.118953  9978 leader_election.cc:304] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.120923 11357 raft_consensus.cc:2749] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:11.138229  9904 consensus_peers.cc:597] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.139001  9904 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.149647  9904 consensus_peers.cc:597] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.160712  9978 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.174979 11357 raft_consensus.cc:493] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:11.175678 11357 raft_consensus.cc:515] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:11.177723 11357 leader_election.cc:290] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.182258  9975 leader_election.cc:336] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:11.183413  9978 leader_election.cc:336] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.183854  9978 leader_election.cc:304] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.184633 11357 raft_consensus.cc:2749] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:11.202838  9904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111) [suppressed 489 similar messages]
W20260709 13:06:11.206331  9904 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.237241  9901 consensus_peers.cc:597] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.267387  9904 consensus_peers.cc:597] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.280445  9978 consensus_peers.cc:597] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.304006  9901 consensus_peers.cc:597] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.349690 11334 raft_consensus.cc:493] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.350593 11334 raft_consensus.cc:515] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:11.351651 11404 raft_consensus.cc:493] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.353009 11404 raft_consensus.cc:515] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:11.354156 11334 leader_election.cc:290] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:11.355989 11404 leader_election.cc:290] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.366889  9904 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:11.369143  9901 leader_election.cc:336] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.369913  9901 leader_election.cc:304] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.370944 11334 raft_consensus.cc:2749] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:11.367686  9904 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:11.372267  9901 leader_election.cc:336] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.372710  9901 leader_election.cc:304] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.374004 11404 raft_consensus.cc:2749] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:11.381762  9975 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.392309  9901 consensus_peers.cc:597] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.420029  9901 consensus_peers.cc:597] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.428328  9975 consensus_peers.cc:597] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.438655  9904 consensus_peers.cc:597] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:11.455304 11334 raft_consensus.cc:493] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.456091 11334 raft_consensus.cc:515] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:11.458513 11334 leader_election.cc:290] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:11.459103   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:11.463603  9904 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:11.464443  9901 leader_election.cc:336] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.464931  9901 leader_election.cc:304] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.465719 11334 raft_consensus.cc:2749] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:11.479453  9901 consensus_peers.cc:597] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.518332  9975 consensus_peers.cc:597] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.533253 11358 raft_consensus.cc:493] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:11.534356 11358 raft_consensus.cc:515] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:11.543066 11358 leader_election.cc:290] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.550217  9901 consensus_peers.cc:597] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.553053  9975 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:11.554345  9978 leader_election.cc:336] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.557605  9978 leader_election.cc:304] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.559440 11358 raft_consensus.cc:2749] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:11.562704  9904 consensus_peers.cc:597] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 -> Peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Couldn't send request to peer 24c6a28138294e46bf6b3d5ba324046d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:11.568437  9901 consensus_peers.cc:597] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.572152 11358 raft_consensus.cc:493] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.572969 11358 raft_consensus.cc:515] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:11.576032 11358 leader_election.cc:290] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.580905  9975 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:11.581647  9978 leader_election.cc:336] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.582185  9978 leader_election.cc:304] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.583153 11358 raft_consensus.cc:2749] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:11.595224  9975 consensus_peers.cc:597] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.606195   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:11.614406 11358 raft_consensus.cc:493] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:11.617201 11358 raft_consensus.cc:515] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:11.620163 11358 leader_election.cc:290] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.627951  9975 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:11.628925  9978 leader_election.cc:336] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.629380  9978 leader_election.cc:304] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.630216 11358 raft_consensus.cc:2749] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:11.632861 11334 raft_consensus.cc:493] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e8769fa7061d433bb90a9411c71fa0df)
I20260709 13:06:11.633510 11334 raft_consensus.cc:515] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:11.635782 11334 leader_election.cc:290] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.642122  9904 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:11.643620  9975 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:11.645833  9901 leader_election.cc:336] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.646397  9901 leader_election.cc:304] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.647343 11334 raft_consensus.cc:2749] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:11.674116  9977 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.675725  9977 consensus_peers.cc:597] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.678568 11415 raft_consensus.cc:493] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.679512 11415 raft_consensus.cc:515] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } } peers { permanent_uuid: "d02e8a054a634be599149db641901736" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37607 } }
I20260709 13:06:11.682112 11415 leader_election.cc:290] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
W20260709 13:06:11.686235  9977 consensus_peers.cc:597] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.688226   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.689632   431 tablet_replica.cc:333] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736: stopping tablet replica
W20260709 13:06:11.689077  9904 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
W20260709 13:06:11.690716  9977 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.693272  9901 leader_election.cc:336] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.693545   431 raft_consensus.cc:2243] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.693746  9901 leader_election.cc:304] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d02e8a054a634be599149db641901736; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.694393   431 raft_consensus.cc:2272] T fca2ecb398d14bd48a7c452af841da65 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.695017 11415 raft_consensus.cc:2749] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:11.698617   431 tablet_replica.cc:333] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.699622   431 raft_consensus.cc:2243] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.700804   431 raft_consensus.cc:2272] T e62c671eeefb49758edb7a7273fc6a2c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.703941   431 tablet_replica.cc:333] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.704962   431 raft_consensus.cc:2243] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.705830   431 raft_consensus.cc:2272] T 5673ca542b1745a38fdc5b65d4357b42 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.707224  9975 consensus_peers.cc:597] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.709299   431 tablet_replica.cc:333] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.710434   431 raft_consensus.cc:2243] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.711161   431 raft_consensus.cc:2272] T 9cb3e93354824df58589f250b6264d54 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.713917  9977 consensus_peers.cc:597] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.714198   431 tablet_replica.cc:333] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.715067   431 raft_consensus.cc:2243] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.715669   431 raft_consensus.cc:2272] T b17aba5ba2fe46a49bec25310d428a26 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.718626   431 tablet_replica.cc:333] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.719864   431 raft_consensus.cc:2243] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.720996   431 raft_consensus.cc:2272] T 6bab397fa40041c281ce616134672828 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.723453  9975 consensus_peers.cc:597] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.724283   431 tablet_replica.cc:333] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.725265   431 raft_consensus.cc:2243] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.725975   431 raft_consensus.cc:2272] T 12cd0294fa614c61b200754ffbbf7601 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.728572   431 tablet_replica.cc:333] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.729404   431 raft_consensus.cc:2243] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.730477   431 raft_consensus.cc:2272] T bd7ef372f79649dda9b18919bfc39298 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.734781   431 tablet_replica.cc:333] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.734604 11391 raft_consensus.cc:493] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.736016   431 raft_consensus.cc:2243] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.736435 11391 raft_consensus.cc:515] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:11.736929   431 raft_consensus.cc:2272] T 82893adc39f24309a241ee3e5ff6611e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.739308 11391 leader_election.cc:290] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:11.741022   431 tablet_replica.cc:333] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.743335   431 raft_consensus.cc:2243] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:06:11.740962  9901 consensus_peers.cc:597] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.745177   431 raft_consensus.cc:2272] T 158c95c12c924f2ea6752771f4a48f0f P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.745271 11391 raft_consensus.cc:493] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
W20260709 13:06:11.746596  9975 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.746398 11391 raft_consensus.cc:515] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:06:11.747365  9978 leader_election.cc:336] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.747921  9978 leader_election.cc:304] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.749415 11358 raft_consensus.cc:2749] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:11.749894   431 tablet_replica.cc:333] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.751178   431 raft_consensus.cc:2243] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.751919   431 raft_consensus.cc:2272] T c416362f795e46e28a60937c9ef5164f P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.749418 11391 leader_election.cc:290] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:11.756388   431 tablet_replica.cc:333] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736: stopping tablet replica
W20260709 13:06:11.756541  9978 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.758004   431 raft_consensus.cc:2243] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:11.758641  9901 consensus_peers.cc:597] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.758780   431 raft_consensus.cc:2272] T 4125c95bf84c45538728679b308f6aca P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.761934  9975 leader_election.cc:336] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.763230   431 tablet_replica.cc:333] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.763563  9975 leader_election.cc:304] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.764554   431 raft_consensus.cc:2243] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.765183 11391 raft_consensus.cc:2749] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:11.766034   431 raft_consensus.cc:2272] T 4a2a7189703643899a38fc9b896d3c78 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.766911  9977 consensus_peers.cc:597] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.769735   431 tablet_replica.cc:333] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.770677   431 raft_consensus.cc:2243] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.771363   431 raft_consensus.cc:2272] T 210c05e6675c4ba78ca28841a5315a07 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.771924  9977 consensus_peers.cc:597] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.774952   431 tablet_replica.cc:333] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.776078   431 raft_consensus.cc:2243] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.776822   431 raft_consensus.cc:2272] T 4716f93906794e73b45089786b4a75f0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.781100   431 tablet_replica.cc:333] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.782092   431 raft_consensus.cc:2243] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.782685   431 raft_consensus.cc:2272] T 7eab4bb2719246f2bf5af8bbccae641d P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.786204   431 tablet_replica.cc:333] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.787551   431 raft_consensus.cc:2243] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.788918   431 pending_rounds.cc:70] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: Trying to abort 1 pending ops.
I20260709 13:06:11.789245   431 pending_rounds.cc:77] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305635299748171776 op_type: NO_OP noop_request { }
I20260709 13:06:11.789690   431 raft_consensus.cc:2889] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:11.790136   431 raft_consensus.cc:2272] T 0f50ff8860ba431d9ee421ff76612df6 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.793865   431 tablet_replica.cc:333] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.794862   431 raft_consensus.cc:2243] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.796171   431 raft_consensus.cc:2272] T 178dcd0da5414a9785794b2412a9cf93 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.800043   431 tablet_replica.cc:333] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.800990   431 raft_consensus.cc:2243] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.802407   431 raft_consensus.cc:2272] T d3324f833b7b462d8aa8db06fb4bd74e P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.806427   431 tablet_replica.cc:333] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.807331   431 raft_consensus.cc:2243] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.808310   431 raft_consensus.cc:2272] T b5fa7348f6384183abeb9619f1fd0d75 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.811861  9975 consensus_peers.cc:597] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.812933   431 tablet_replica.cc:333] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.814070   431 raft_consensus.cc:2243] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.815196   431 raft_consensus.cc:2272] T eb7d115ef55d410a88514a918dee8efc P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.815853  9977 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.818789  9977 consensus_peers.cc:597] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.818969   431 tablet_replica.cc:333] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.819303 11358 raft_consensus.cc:493] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.819880 11358 raft_consensus.cc:515] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
I20260709 13:06:11.820109   431 raft_consensus.cc:2243] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.821434   431 raft_consensus.cc:2272] T e12836e61c6d4427a5d883c4bdf5b5b6 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.825378   431 tablet_replica.cc:333] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.824965 11358 leader_election.cc:290] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:11.826874   431 raft_consensus.cc:2243] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.827605   431 raft_consensus.cc:2272] T 1ad9129f493c417eaa0d52805e5b1144 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.831013   431 tablet_replica.cc:333] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736: stopping tablet replica
W20260709 13:06:11.831660  9978 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.832475   431 raft_consensus.cc:2243] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 4 LEADER]: Raft consensus shutting down.
W20260709 13:06:11.833293  9975 leader_election.cc:336] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
I20260709 13:06:11.833933  9975 leader_election.cc:304] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.834329   431 raft_consensus.cc:2272] T 93fbf294836d47b29e0143174bbb3df4 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.835517 11358 raft_consensus.cc:2749] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:06:11.837689  9977 consensus_peers.cc:597] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.838097   431 tablet_replica.cc:333] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.839191   431 raft_consensus.cc:2243] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.840359   431 raft_consensus.cc:2272] T e63bec87e7a64ac0bb9eea74f15908ba P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.842762  9975 consensus_peers.cc:597] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.844530   431 tablet_replica.cc:333] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.845772   431 raft_consensus.cc:2243] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.847030   431 raft_consensus.cc:2272] T 16d91cae140a4615a015d3d34ac878c4 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.850939   431 tablet_replica.cc:333] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.851785   431 raft_consensus.cc:2243] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.852587   431 raft_consensus.cc:2272] T 0c2d6cf525014f80a8f95687135fff6f P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.855870   431 tablet_replica.cc:333] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.856676   431 raft_consensus.cc:2243] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.857245   431 raft_consensus.cc:2272] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.860020   431 tablet_replica.cc:333] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.860877   431 raft_consensus.cc:2243] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.861486   431 raft_consensus.cc:2272] T f0bdd04f0ff44f2e87f76f36a7c5e540 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.864233   431 tablet_replica.cc:333] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.865198   431 raft_consensus.cc:2243] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.865970   431 raft_consensus.cc:2272] T e9ea20d35c3b4150ba41cec32b7c9e0c P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.868157   431 tablet_replica.cc:333] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.868935   431 raft_consensus.cc:2243] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.869781   431 raft_consensus.cc:2272] T 79129676db004c7bbe29ab7e9ec1fbe3 P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.872984   431 tablet_replica.cc:333] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.873876   431 raft_consensus.cc:2243] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.874481   431 raft_consensus.cc:2272] T 4e31a807457a433b9c28f7a67c98095c P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.877744   431 tablet_replica.cc:333] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.878690   431 raft_consensus.cc:2243] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.879782   431 raft_consensus.cc:2272] T b253ddcc7d6046c6abdd94aec212612c P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.880321 11358 raft_consensus.cc:493] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 24c6a28138294e46bf6b3d5ba324046d)
I20260709 13:06:11.881148 11358 raft_consensus.cc:515] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e087ae2ce3f417093313edd26b8b3ee" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38131 } } peers { permanent_uuid: "e8769fa7061d433bb90a9411c71fa0df" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 32801 } } peers { permanent_uuid: "24c6a28138294e46bf6b3d5ba324046d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 32919 } }
W20260709 13:06:11.882671  9901 consensus_peers.cc:597] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:11.883757   431 tablet_replica.cc:333] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.884394 11358 leader_election.cc:290] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801), 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919)
I20260709 13:06:11.885188   431 raft_consensus.cc:2243] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.886027   431 raft_consensus.cc:2272] T b3695e4783714136ad332a175348e9fc P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.889690   431 tablet_replica.cc:333] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736: stopping tablet replica
W20260709 13:06:11.889297  9975 leader_election.cc:336] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111)
W20260709 13:06:11.891232  9978 leader_election.cc:336] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 24c6a28138294e46bf6b3d5ba324046d (127.0.107.194:32919): Network error: Client connection negotiation failed: client connection to 127.0.107.194:32919: connect: Connection refused (error 111)
I20260709 13:06:11.891641   431 raft_consensus.cc:2243] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.891728  9978 leader_election.cc:304] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e087ae2ce3f417093313edd26b8b3ee; no voters: 24c6a28138294e46bf6b3d5ba324046d, e8769fa7061d433bb90a9411c71fa0df
I20260709 13:06:11.892891 11391 raft_consensus.cc:2749] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:11.893111   431 raft_consensus.cc:2272] T 1cfdee35c2f0491da78cbc51ed63ba5d P d02e8a054a634be599149db641901736 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.894398  9977 consensus_peers.cc:597] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.897020   431 tablet_replica.cc:333] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.897902   431 raft_consensus.cc:2243] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.899040   431 raft_consensus.cc:2272] T f8a86072500f497585b83ac124a8a562 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.902552   431 tablet_replica.cc:333] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.903366   431 raft_consensus.cc:2243] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.905166   431 raft_consensus.cc:2272] T 69af1a39e3934236883aaa506f999dee P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.905542  9977 consensus_peers.cc:597] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.909612   431 tablet_replica.cc:333] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.910967   431 raft_consensus.cc:2243] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.912549   431 raft_consensus.cc:2272] T 9625fd7e9ed046158552e5252f1ee3db P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.912825  9977 consensus_peers.cc:597] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.914211  9977 consensus_peers.cc:597] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.915313   431 tablet_replica.cc:333] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.916026   431 raft_consensus.cc:2243] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.916985   431 raft_consensus.cc:2272] T a693233cb65a40089d314773895f84a0 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.919124   431 tablet_replica.cc:333] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.919723   431 raft_consensus.cc:2243] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.920209   431 raft_consensus.cc:2272] T 1801426ac1e84ebaadda531ec03958bd P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.922341   431 tablet_replica.cc:333] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.922904   431 raft_consensus.cc:2243] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.923363   431 raft_consensus.cc:2272] T 357a7ca5934d4fb6958de3cca90f5a36 P d02e8a054a634be599149db641901736 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.925724   431 tablet_replica.cc:333] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.926435   431 raft_consensus.cc:2243] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.927476   431 raft_consensus.cc:2272] T 582cebb9b4674942bb40b7be9936c3f5 P d02e8a054a634be599149db641901736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.931355   431 tablet_replica.cc:333] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.932071   431 raft_consensus.cc:2243] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.932578   431 raft_consensus.cc:2272] T c48883428dea42d7b95a7cddbc07178e P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.935586   431 tablet_replica.cc:333] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736: stopping tablet replica
I20260709 13:06:11.936293   431 raft_consensus.cc:2243] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.937449   431 raft_consensus.cc:2272] T d775d5d3993743689257e4509a059770 P d02e8a054a634be599149db641901736 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.956441  9977 consensus_peers.cc:597] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:11.961109  9977 consensus_peers.cc:597] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:12.073201   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
W20260709 13:06:12.109781  9977 consensus_peers.cc:597] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee -> Peer d02e8a054a634be599149db641901736 (127.0.107.195:37607): Couldn't send request to peer d02e8a054a634be599149db641901736. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:37607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:12.123561  9975 consensus_peers.cc:597] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee -> Peer e8769fa7061d433bb90a9411c71fa0df (127.0.107.193:32801): Couldn't send request to peer e8769fa7061d433bb90a9411c71fa0df. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:32801: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:12.198846   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 13:06:12.279628   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:12.281019   431 tablet_replica.cc:333] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.282316   431 raft_consensus.cc:2243] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.282946   431 raft_consensus.cc:2272] T e62c671eeefb49758edb7a7273fc6a2c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.286466   431 tablet_replica.cc:333] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.287365   431 raft_consensus.cc:2243] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.287984   431 raft_consensus.cc:2272] T c68f14a359d54249b7cab91d0e63d7b0 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.290858   431 tablet_replica.cc:333] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.291756   431 raft_consensus.cc:2243] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.292941   431 raft_consensus.cc:2272] T 82893adc39f24309a241ee3e5ff6611e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.295403   431 tablet_replica.cc:333] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.296180   431 raft_consensus.cc:2243] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.296754   431 raft_consensus.cc:2272] T 158c95c12c924f2ea6752771f4a48f0f P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.299165   431 tablet_replica.cc:333] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.299955   431 raft_consensus.cc:2243] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.300913   431 raft_consensus.cc:2272] T c416362f795e46e28a60937c9ef5164f P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.303194   431 tablet_replica.cc:333] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.304325   431 raft_consensus.cc:2243] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.305774   431 raft_consensus.cc:2272] T 4125c95bf84c45538728679b308f6aca P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.310225   431 tablet_replica.cc:333] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.311996   431 raft_consensus.cc:2243] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.313536   431 raft_consensus.cc:2272] T 210c05e6675c4ba78ca28841a5315a07 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.316872   431 tablet_replica.cc:333] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.317777   431 raft_consensus.cc:2243] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.318807   431 raft_consensus.cc:2272] T 6bf82e92a7f845f4a7c658d0783ea4a5 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.321789   431 tablet_replica.cc:333] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.322659   431 raft_consensus.cc:2243] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.323197   431 raft_consensus.cc:2272] T 04298f45db464a09bd0d2b82492e61b3 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.326174   431 tablet_replica.cc:333] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.326825   431 raft_consensus.cc:2243] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.327378   431 raft_consensus.cc:2272] T 7d6f1cfb3ffd4a418cafdda5af0b219d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.331344   431 tablet_replica.cc:333] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.332942   431 raft_consensus.cc:2243] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.333622   431 raft_consensus.cc:2272] T 404c2d54fc4e4b6c8a892de6e4988671 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.337611   431 tablet_replica.cc:333] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.338430   431 raft_consensus.cc:2243] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.339406   431 raft_consensus.cc:2272] T 625608f2b39c495d9f5b471dcabde2de P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.341984   431 tablet_replica.cc:333] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.342675   431 raft_consensus.cc:2243] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.344221   431 raft_consensus.cc:2272] T 7eab4bb2719246f2bf5af8bbccae641d P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.347148   431 tablet_replica.cc:333] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.347931   431 raft_consensus.cc:2243] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.348588   431 raft_consensus.cc:2272] T e12836e61c6d4427a5d883c4bdf5b5b6 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.352064   431 tablet_replica.cc:333] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.353087   431 raft_consensus.cc:2243] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.354069   431 raft_consensus.cc:2272] T 1ad9129f493c417eaa0d52805e5b1144 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.356755   431 tablet_replica.cc:333] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.357458   431 raft_consensus.cc:2243] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.357950   431 raft_consensus.cc:2272] T 93fbf294836d47b29e0143174bbb3df4 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.360527   431 tablet_replica.cc:333] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.361207   431 raft_consensus.cc:2243] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.361733   431 raft_consensus.cc:2272] T d3324f833b7b462d8aa8db06fb4bd74e P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.365594   431 tablet_replica.cc:333] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.366357   431 raft_consensus.cc:2243] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.367087   431 raft_consensus.cc:2272] T eb7d115ef55d410a88514a918dee8efc P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.370445   431 tablet_replica.cc:333] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.371132   431 raft_consensus.cc:2243] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.372802   431 raft_consensus.cc:2272] T b17aba5ba2fe46a49bec25310d428a26 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.375778   431 tablet_replica.cc:333] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.376642   431 raft_consensus.cc:2243] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.377653   431 raft_consensus.cc:2272] T 12cd0294fa614c61b200754ffbbf7601 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.380059   431 tablet_replica.cc:333] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.380782   431 raft_consensus.cc:2243] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.381673   431 raft_consensus.cc:2272] T 1078585b45d74186be92d54ac68f787a P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.384213   431 tablet_replica.cc:333] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.384940   431 raft_consensus.cc:2243] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.386407   431 raft_consensus.cc:2272] T 36f8b3f8e9ff4e1ab10c104b1a3c2da6 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.389596   431 tablet_replica.cc:333] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.390507   431 raft_consensus.cc:2243] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.390997   431 raft_consensus.cc:2272] T e18d9519bce7451aaee1e04623f7fddb P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.393261   431 tablet_replica.cc:333] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.394111   431 raft_consensus.cc:2243] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.395299   431 raft_consensus.cc:2272] T e9ea20d35c3b4150ba41cec32b7c9e0c P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.398206   431 tablet_replica.cc:333] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.398861   431 raft_consensus.cc:2243] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.399904   431 raft_consensus.cc:2272] T 4e31a807457a433b9c28f7a67c98095c P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.402557   431 tablet_replica.cc:333] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.403199   431 raft_consensus.cc:2243] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.403692   431 raft_consensus.cc:2272] T 178dcd0da5414a9785794b2412a9cf93 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.406661   431 tablet_replica.cc:333] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.407367   431 raft_consensus.cc:2243] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.408021   431 raft_consensus.cc:2272] T 82999b9832894f40bb6dbe02e2feb1df P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.410756   431 tablet_replica.cc:333] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.411592   431 raft_consensus.cc:2243] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.412705   431 raft_consensus.cc:2272] T 47be166313094f0fbe17bde042f05dfe P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.415443   431 tablet_replica.cc:333] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.416178   431 raft_consensus.cc:2243] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.417289   431 raft_consensus.cc:2272] T b3695e4783714136ad332a175348e9fc P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.419865   431 tablet_replica.cc:333] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.420933   431 raft_consensus.cc:2243] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.421640   431 raft_consensus.cc:2272] T 1cfdee35c2f0491da78cbc51ed63ba5d P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.424958   431 tablet_replica.cc:333] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.427198   431 raft_consensus.cc:2243] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.427946   431 raft_consensus.cc:2272] T f8a86072500f497585b83ac124a8a562 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.430795   431 tablet_replica.cc:333] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.431468   431 raft_consensus.cc:2243] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.431963   431 raft_consensus.cc:2272] T 1131589c5bd9480d838a7967131195ca P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.434108   431 tablet_replica.cc:333] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.434686   431 raft_consensus.cc:2243] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.435139   431 raft_consensus.cc:2272] T 22c6e691c1264b8e8de49c1df4cd8bb5 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.437193   431 tablet_replica.cc:333] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.437980   431 raft_consensus.cc:2243] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.439293   431 raft_consensus.cc:2272] T c48883428dea42d7b95a7cddbc07178e P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.441836   431 tablet_replica.cc:333] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.442581   431 raft_consensus.cc:2243] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.443156   431 raft_consensus.cc:2272] T 877ba104634d4068a2f26aa62d860b7b P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.445951   431 tablet_replica.cc:333] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.446667   431 raft_consensus.cc:2243] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.447917   431 raft_consensus.cc:2272] T 1801426ac1e84ebaadda531ec03958bd P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.451713   431 tablet_replica.cc:333] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.452517   431 raft_consensus.cc:2243] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.453439   431 raft_consensus.cc:2272] T 357a7ca5934d4fb6958de3cca90f5a36 P 5e087ae2ce3f417093313edd26b8b3ee [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.455546   431 tablet_replica.cc:333] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.456097   431 raft_consensus.cc:2243] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.457438   431 raft_consensus.cc:2272] T 5673ca542b1745a38fdc5b65d4357b42 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.460481   431 tablet_replica.cc:333] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.461208   431 raft_consensus.cc:2243] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.462293   431 raft_consensus.cc:2272] T 4a08704c3c3c4030bb48886795597fe1 P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.465636   431 tablet_replica.cc:333] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.466455   431 raft_consensus.cc:2243] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.467370   431 raft_consensus.cc:2272] T 582cebb9b4674942bb40b7be9936c3f5 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.470829   431 tablet_replica.cc:333] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.471556   431 raft_consensus.cc:2243] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.472589   431 raft_consensus.cc:2272] T e63bec87e7a64ac0bb9eea74f15908ba P 5e087ae2ce3f417093313edd26b8b3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.475379   431 tablet_replica.cc:333] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.475982   431 raft_consensus.cc:2243] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.476593   431 raft_consensus.cc:2272] T f5dd44a42946450e936b89718864c5e3 P 5e087ae2ce3f417093313edd26b8b3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.479343   431 tablet_replica.cc:333] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.480008   431 raft_consensus.cc:2243] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.480577   431 raft_consensus.cc:2272] T 16d91cae140a4615a015d3d34ac878c4 P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.483147   431 tablet_replica.cc:333] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee: stopping tablet replica
I20260709 13:06:12.483783   431 raft_consensus.cc:2243] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:12.484314   431 raft_consensus.cc:2272] T 9625fd7e9ed046158552e5252f1ee3db P 5e087ae2ce3f417093313edd26b8b3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.626483   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:06:12.720882   431 master.cc:562] Master@127.0.107.254:37093 shutting down...
I20260709 13:06:12.768131   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.769163   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.769582   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7ac5359d00f54e37bd8a40ec7cebcc55: stopping tablet replica
I20260709 13:06:12.796672   431 master.cc:584] Master@127.0.107.254:37093 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (57551 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:06:12.925523   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:43437
I20260709 13:06:12.927099   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.935065 11419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.935422 11418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.936074 11421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.937181   431 server_base.cc:1061] running on GCE node
I20260709 13:06:12.938897   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.939129   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.939359   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602372939340 us; error 0 us; skew 500 ppm
I20260709 13:06:12.940157   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.944077   431 webserver.cc:533] Webserver started at http://127.0.107.254:34759/ using document root <none> and password file <none>
I20260709 13:06:12.944698   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.944919   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.945592   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.947302   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "265081b9ebd545d0bea4d73560310287"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-ftjc"
I20260709 13:06:12.955340   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.010s	sys 0.000s
I20260709 13:06:12.961002 11426 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.962674   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:06:12.963078   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "265081b9ebd545d0bea4d73560310287"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-ftjc"
I20260709 13:06:12.963517   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:13.000042   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:13.001839   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:13.067118   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:43437
I20260709 13:06:13.067143 11477 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:43437 every 8 connection(s)
I20260709 13:06:13.072140 11478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.091971 11478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287: Bootstrap starting.
I20260709 13:06:13.099113 11478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.104491 11478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287: No bootstrap required, opened a new log
I20260709 13:06:13.107594 11478 raft_consensus.cc:359] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265081b9ebd545d0bea4d73560310287" member_type: VOTER }
I20260709 13:06:13.108222 11478 raft_consensus.cc:385] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.108597 11478 raft_consensus.cc:740] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265081b9ebd545d0bea4d73560310287, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.109355 11478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265081b9ebd545d0bea4d73560310287" member_type: VOTER }
I20260709 13:06:13.109974 11478 raft_consensus.cc:399] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:13.110271 11478 raft_consensus.cc:493] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:13.110584 11478 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.118484 11478 raft_consensus.cc:515] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265081b9ebd545d0bea4d73560310287" member_type: VOTER }
I20260709 13:06:13.119177 11478 leader_election.cc:304] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 265081b9ebd545d0bea4d73560310287; no voters: 
I20260709 13:06:13.120837 11478 leader_election.cc:290] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:13.121623 11481 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.123546 11481 raft_consensus.cc:697] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 1 LEADER]: Becoming Leader. State: Replica: 265081b9ebd545d0bea4d73560310287, State: Running, Role: LEADER
I20260709 13:06:13.124570 11481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265081b9ebd545d0bea4d73560310287" member_type: VOTER }
I20260709 13:06:13.124974 11478 sys_catalog.cc:565] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:13.130787 11482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "265081b9ebd545d0bea4d73560310287" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265081b9ebd545d0bea4d73560310287" member_type: VOTER } }
I20260709 13:06:13.131908 11482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:13.132062 11483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 265081b9ebd545d0bea4d73560310287. Latest consensus state: current_term: 1 leader_uuid: "265081b9ebd545d0bea4d73560310287" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265081b9ebd545d0bea4d73560310287" member_type: VOTER } }
I20260709 13:06:13.132864 11483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:13.134938 11489 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:13.142958 11489 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:13.144994   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:13.155954 11489 catalog_manager.cc:1383] Generated new cluster ID: 1715e7e9f9ff439a8dc77a1aef3d4ab2
I20260709 13:06:13.156416 11489 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:13.174319 11489 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:13.176394 11489 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:13.197465 11489 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287: Generated new TSK 0
I20260709 13:06:13.198385 11489 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:13.213685   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:13.221669 11499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:13.223978 11500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:13.234995 11502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:13.236645   431 server_base.cc:1061] running on GCE node
I20260709 13:06:13.239746   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:13.240042   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:13.240279   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602373240248 us; error 0 us; skew 500 ppm
I20260709 13:06:13.241078   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:13.245076   431 webserver.cc:533] Webserver started at http://127.0.107.193:33361/ using document root <none> and password file <none>
I20260709 13:06:13.245807   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:13.246054   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:13.246434   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:13.247996   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "a6999ef0c57c494b9e4774bd01b6054d"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:13.255692   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:06:13.261492 11507 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.263259   431 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.002s
I20260709 13:06:13.263950   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "a6999ef0c57c494b9e4774bd01b6054d"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:13.264484   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:13.276001   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:13.278256   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:13.280622   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:13.284195   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:13.284543   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:06:13.284989   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:13.285820   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:06:13.350807   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:39895
I20260709 13:06:13.350935 11569 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:39895 every 8 connection(s)
I20260709 13:06:13.357949   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:13.368310 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:13.368386 11575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:13.374346 11577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:13.374552   431 server_base.cc:1061] running on GCE node
I20260709 13:06:13.378257   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:13.378608   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:13.378842   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602373378820 us; error 0 us; skew 500 ppm
I20260709 13:06:13.379824   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:13.380889 11570 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43437
I20260709 13:06:13.381366 11570 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:13.382314 11570 heartbeater.cc:507] Master 127.0.107.254:43437 requested a full tablet report, sending...
I20260709 13:06:13.383668   431 webserver.cc:533] Webserver started at http://127.0.107.194:33079/ using document root <none> and password file <none>
I20260709 13:06:13.384603   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:13.384862   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:13.385203   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:13.385495 11443 ts_manager.cc:194] Registered new tserver with Master: a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895)
I20260709 13:06:13.386858   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "40e9f867c7af439bbfca87d7e3f03afa"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:13.387882 11443 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44466
I20260709 13:06:13.393229   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:06:13.398177 11582 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.399415   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:06:13.400002   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "40e9f867c7af439bbfca87d7e3f03afa"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:13.400538   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:13.413960   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:13.415911   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:13.418220   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:13.421520   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:13.421792   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.422192   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:13.422371   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.489173   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:42175
I20260709 13:06:13.489322 11644 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:42175 every 8 connection(s)
I20260709 13:06:13.496069   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:13.506583 11648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:13.508880 11649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:13.512274 11651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:13.513648 11645 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43437
I20260709 13:06:13.514151 11645 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:13.515223 11645 heartbeater.cc:507] Master 127.0.107.254:43437 requested a full tablet report, sending...
I20260709 13:06:13.516439   431 server_base.cc:1061] running on GCE node
I20260709 13:06:13.517642   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:13.517869   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:13.518281   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602373518248 us; error 0 us; skew 500 ppm
I20260709 13:06:13.518784 11443 ts_manager.cc:194] Registered new tserver with Master: 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:13.519132   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:13.521595 11443 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44476
I20260709 13:06:13.523870   431 webserver.cc:533] Webserver started at http://127.0.107.195:39629/ using document root <none> and password file <none>
I20260709 13:06:13.525820   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:13.526387   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:13.526914   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:13.528738   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "a16fca66e4404a01911ecd98feb63dcd"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:13.536805   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:13.542536 11656 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.543867   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:06:13.544435   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "a16fca66e4404a01911ecd98feb63dcd"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:13.544848   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:13.572413   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:13.574256   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:13.576395   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:13.579273   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:13.579605   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.579901   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:13.580092   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:13.649441   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:42925
I20260709 13:06:13.649919 11718 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:42925 every 8 connection(s)
I20260709 13:06:13.674500 11719 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43437
I20260709 13:06:13.675123 11719 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:13.676451 11719 heartbeater.cc:507] Master 127.0.107.254:43437 requested a full tablet report, sending...
I20260709 13:06:13.679232 11443 ts_manager.cc:194] Registered new tserver with Master: a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:13.679462   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.02081352s
I20260709 13:06:13.681126 11443 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44486
I20260709 13:06:13.718747 11443 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44488:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:13.722791 11443 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:13.840554 11530 tablet_service.cc:1511] Processing CreateTablet for tablet 8a60b2e1c3304bafaa8826f684b0c3de (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:13.842532 11530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a60b2e1c3304bafaa8826f684b0c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.843755 11684 tablet_service.cc:1511] Processing CreateTablet for tablet 7e242cfc1aa04b228548b73077358cef (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:13.846251 11682 tablet_service.cc:1511] Processing CreateTablet for tablet 56afd91ff5be4d288bf586e5f0c222a9 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:13.847254 11680 tablet_service.cc:1511] Processing CreateTablet for tablet f1860bb4663348148576caaf0f46fab6 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:13.849705 11681 tablet_service.cc:1511] Processing CreateTablet for tablet 9f0cb9ea46d74fc9bdc2292669f97fa9 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:13.852454 11679 tablet_service.cc:1511] Processing CreateTablet for tablet 8a60b2e1c3304bafaa8826f684b0c3de (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:13.852689 11533 tablet_service.cc:1511] Processing CreateTablet for tablet 56afd91ff5be4d288bf586e5f0c222a9 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:13.854524 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56afd91ff5be4d288bf586e5f0c222a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.854874 11608 tablet_service.cc:1511] Processing CreateTablet for tablet 56afd91ff5be4d288bf586e5f0c222a9 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:13.858242 11610 tablet_service.cc:1511] Processing CreateTablet for tablet 7e242cfc1aa04b228548b73077358cef (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:13.860060 11606 tablet_service.cc:1511] Processing CreateTablet for tablet f1860bb4663348148576caaf0f46fab6 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:13.860087 11677 tablet_service.cc:1511] Processing CreateTablet for tablet 49cbc82778d042e8aaa7acde20f8b5b3 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:13.860972 11609 tablet_service.cc:1511] Processing CreateTablet for tablet e9e8b00eb8004f938def8be94fd87474 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:13.862671 11678 tablet_service.cc:1511] Processing CreateTablet for tablet cee0ad237fd241ad8c8554a0a05a50d8 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:13.863306 11532 tablet_service.cc:1511] Processing CreateTablet for tablet 9f0cb9ea46d74fc9bdc2292669f97fa9 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:13.866050 11531 tablet_service.cc:1511] Processing CreateTablet for tablet f1860bb4663348148576caaf0f46fab6 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:13.868516 11604 tablet_service.cc:1511] Processing CreateTablet for tablet cee0ad237fd241ad8c8554a0a05a50d8 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:13.869700 11535 tablet_service.cc:1511] Processing CreateTablet for tablet 7e242cfc1aa04b228548b73077358cef (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:13.872066 11603 tablet_service.cc:1511] Processing CreateTablet for tablet 49cbc82778d042e8aaa7acde20f8b5b3 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:13.872673 11534 tablet_service.cc:1511] Processing CreateTablet for tablet e9e8b00eb8004f938def8be94fd87474 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:13.849757 11683 tablet_service.cc:1511] Processing CreateTablet for tablet e9e8b00eb8004f938def8be94fd87474 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:13.876596 11529 tablet_service.cc:1511] Processing CreateTablet for tablet cee0ad237fd241ad8c8554a0a05a50d8 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:13.858242 11607 tablet_service.cc:1511] Processing CreateTablet for tablet 9f0cb9ea46d74fc9bdc2292669f97fa9 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:13.878971 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f0cb9ea46d74fc9bdc2292669f97fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.879107 11528 tablet_service.cc:1511] Processing CreateTablet for tablet 49cbc82778d042e8aaa7acde20f8b5b3 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:13.866323 11605 tablet_service.cc:1511] Processing CreateTablet for tablet 8a60b2e1c3304bafaa8826f684b0c3de (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:13.848248 11682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56afd91ff5be4d288bf586e5f0c222a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.885973 11608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56afd91ff5be4d288bf586e5f0c222a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.893910 11681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f0cb9ea46d74fc9bdc2292669f97fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.894366 11610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e242cfc1aa04b228548b73077358cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.895037 11683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e8b00eb8004f938def8be94fd87474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.900029 11604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee0ad237fd241ad8c8554a0a05a50d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.901103 11531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1860bb4663348148576caaf0f46fab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.901320 11609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e8b00eb8004f938def8be94fd87474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.902110 11603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cbc82778d042e8aaa7acde20f8b5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.905120 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e8b00eb8004f938def8be94fd87474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.905542 11605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a60b2e1c3304bafaa8826f684b0c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.911442 11678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee0ad237fd241ad8c8554a0a05a50d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.912439 11527 tablet_service.cc:1511] Processing CreateTablet for tablet e1937785aba94020a04aa593cb885143 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:13.914780 11527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1937785aba94020a04aa593cb885143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.915261 11680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1860bb4663348148576caaf0f46fab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.914690 11607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f0cb9ea46d74fc9bdc2292669f97fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.918529 11606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1860bb4663348148576caaf0f46fab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.922366 11676 tablet_service.cc:1511] Processing CreateTablet for tablet e1937785aba94020a04aa593cb885143 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:13.924584 11676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1937785aba94020a04aa593cb885143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.928905 11528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cbc82778d042e8aaa7acde20f8b5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.933199 11529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cee0ad237fd241ad8c8554a0a05a50d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.925663 11602 tablet_service.cc:1511] Processing CreateTablet for tablet e1937785aba94020a04aa593cb885143 (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:13.935220 11677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cbc82778d042e8aaa7acde20f8b5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.935925 11602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1937785aba94020a04aa593cb885143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.937527 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e242cfc1aa04b228548b73077358cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.939970 11684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e242cfc1aa04b228548b73077358cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.973826 11679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a60b2e1c3304bafaa8826f684b0c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.980820 11526 tablet_service.cc:1511] Processing CreateTablet for tablet 64f3495644d5424798c65e2a9730d8bf (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:13.983263 11526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64f3495644d5424798c65e2a9730d8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.997413 11601 tablet_service.cc:1511] Processing CreateTablet for tablet 64f3495644d5424798c65e2a9730d8bf (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:13.999292 11601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64f3495644d5424798c65e2a9730d8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:14.000366 11675 tablet_service.cc:1511] Processing CreateTablet for tablet 64f3495644d5424798c65e2a9730d8bf (DEFAULT_TABLE table=test-workload [id=ffb2ccf642234641a1db320c1bb87541]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:14.002135 11675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64f3495644d5424798c65e2a9730d8bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:14.146956 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:14.181464 11737 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.189859 11738 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.197990 11737 tablet_bootstrap.cc:654] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.209380 11736 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.229378 11738 tablet_bootstrap.cc:654] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.255357 11736 tablet_bootstrap.cc:654] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.258615 11737 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.259410 11737 ts_tablet_manager.cc:1403] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.027s
I20260709 13:06:14.263587 11737 raft_consensus.cc:359] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.264613 11737 raft_consensus.cc:385] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.264968 11737 raft_consensus.cc:740] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.274127 11737 consensus_queue.cc:260] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.277495 11738 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.278292 11738 ts_tablet_manager.cc:1403] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.089s	user 0.028s	sys 0.029s
I20260709 13:06:14.283592 11738 raft_consensus.cc:359] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.285816 11738 raft_consensus.cc:385] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.286378 11738 raft_consensus.cc:740] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.287609 11719 heartbeater.cc:499] Master 127.0.107.254:43437 was elected leader, sending a full tablet report...
I20260709 13:06:14.287829 11738 consensus_queue.cc:260] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.289420 11736 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.290755 11736 ts_tablet_manager.cc:1403] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.084s	user 0.026s	sys 0.000s
I20260709 13:06:14.293972 11737 ts_tablet_manager.cc:1434] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.007s
I20260709 13:06:14.295435 11737 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.295393 11736 raft_consensus.cc:359] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.296543 11736 raft_consensus.cc:385] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.296913 11736 raft_consensus.cc:740] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.298053 11736 consensus_queue.cc:260] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.300143 11645 heartbeater.cc:499] Master 127.0.107.254:43437 was elected leader, sending a full tablet report...
I20260709 13:06:14.301751 11738 ts_tablet_manager.cc:1434] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.023s	user 0.013s	sys 0.006s
I20260709 13:06:14.303332 11738 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.309549 11736 ts_tablet_manager.cc:1434] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:06:14.310909 11570 heartbeater.cc:499] Master 127.0.107.254:43437 was elected leader, sending a full tablet report...
I20260709 13:06:14.312992 11738 tablet_bootstrap.cc:654] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.314599 11737 tablet_bootstrap.cc:654] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.318210 11736 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.332405 11736 tablet_bootstrap.cc:654] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.342949 11738 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.343067 11737 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.343816 11738 ts_tablet_manager.cc:1403] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.041s	user 0.027s	sys 0.009s
I20260709 13:06:14.344549 11737 ts_tablet_manager.cc:1403] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.023s
I20260709 13:06:14.349215 11738 raft_consensus.cc:359] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.350595 11738 raft_consensus.cc:385] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.349627 11737 raft_consensus.cc:359] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.351011 11738 raft_consensus.cc:740] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.351354 11737 raft_consensus.cc:385] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.352104 11737 raft_consensus.cc:740] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.352288 11738 consensus_queue.cc:260] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.353906 11737 consensus_queue.cc:260] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.356000 11738 ts_tablet_manager.cc:1434] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:06:14.357349 11738 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.358381 11737 ts_tablet_manager.cc:1434] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.005s
I20260709 13:06:14.360538 11737 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.366262 11738 tablet_bootstrap.cc:654] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.372563 11737 tablet_bootstrap.cc:654] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:14.377763 11571 tablet.cc:2406] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:14.398092 11737 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.401067 11737 ts_tablet_manager.cc:1403] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.014s
I20260709 13:06:14.407749 11737 raft_consensus.cc:359] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.409039 11737 raft_consensus.cc:385] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.409555 11737 raft_consensus.cc:740] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.413385 11737 consensus_queue.cc:260] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.420279 11737 ts_tablet_manager.cc:1434] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:06:14.422465 11737 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.427668 11741 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.428781 11741 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.432446 11736 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.433207 11736 ts_tablet_manager.cc:1403] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.115s	user 0.049s	sys 0.055s
I20260709 13:06:14.433699 11738 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.434499 11738 ts_tablet_manager.cc:1403] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.077s	user 0.035s	sys 0.039s
I20260709 13:06:14.437068 11741 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:14.436473 11736 raft_consensus.cc:359] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.440416 11736 raft_consensus.cc:385] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.437920 11738 raft_consensus.cc:359] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.440945 11736 raft_consensus.cc:740] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.441246 11738 raft_consensus.cc:385] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.441704 11738 raft_consensus.cc:740] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.442515 11736 consensus_queue.cc:260] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.442814 11738 consensus_queue.cc:260] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.445943 11738 ts_tablet_manager.cc:1434] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:06:14.447487 11738 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.452054 11737 tablet_bootstrap.cc:654] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.456182 11738 tablet_bootstrap.cc:654] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.457952 11736 ts_tablet_manager.cc:1434] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:06:14.462096 11741 raft_consensus.cc:493] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.462832 11741 raft_consensus.cc:515] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.465858 11741 leader_election.cc:290] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:14.472473 11736 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.479835 11737 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.480823 11737 ts_tablet_manager.cc:1403] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.016s
I20260709 13:06:14.483301 11738 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.483733 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:14.484163 11738 ts_tablet_manager.cc:1403] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.037s	user 0.003s	sys 0.010s
I20260709 13:06:14.484483 11736 tablet_bootstrap.cc:654] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.484905 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.486161 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:14.487527 11743 raft_consensus.cc:493] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:14.487954 11660 leader_election.cc:343] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.488180 11743 raft_consensus.cc:515] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.488552 11738 raft_consensus.cc:359] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.489358 11738 raft_consensus.cc:385] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.483875 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.489710 11738 raft_consensus.cc:740] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
W20260709 13:06:14.491886 11660 leader_election.cc:343] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:14.492843 11660 leader_election.cc:343] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.492424 11738 consensus_queue.cc:260] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.493369 11660 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d
W20260709 13:06:14.496745 11660 leader_election.cc:343] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.497608 11660 leader_election.cc:304] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:14.495132 11741 raft_consensus.cc:2749] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.499836 11741 raft_consensus.cc:2749] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.504966 11737 raft_consensus.cc:359] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.505306 11736 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.505995 11737 raft_consensus.cc:385] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.506620 11737 raft_consensus.cc:740] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.506870 11736 ts_tablet_manager.cc:1403] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.007s
I20260709 13:06:14.508550 11743 leader_election.cc:290] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.510030 11737 consensus_queue.cc:260] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.513535 11736 raft_consensus.cc:359] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.514791 11736 raft_consensus.cc:385] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.515180 11736 raft_consensus.cc:740] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.520123 11738 ts_tablet_manager.cc:1434] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.012s
I20260709 13:06:14.519654 11736 consensus_queue.cc:260] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.521483 11738 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.531642 11737 ts_tablet_manager.cc:1434] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.005s
I20260709 13:06:14.533561 11737 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.538971 11736 ts_tablet_manager.cc:1434] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 13:06:14.540308 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.540617 11736 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.541766 11738 tablet_bootstrap.cc:654] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:14.542448 11586 leader_election.cc:343] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.550443 11736 tablet_bootstrap.cc:654] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.556519 11738 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.557224 11738 ts_tablet_manager.cc:1403] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.006s
I20260709 13:06:14.559063 11744 raft_consensus.cc:493] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.559907 11744 raft_consensus.cc:515] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.560963 11738 raft_consensus.cc:359] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:14.561848 11738 raft_consensus.cc:385] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.562189 11738 raft_consensus.cc:740] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.562721 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.563097 11738 consensus_queue.cc:260] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:14.563598 11694 raft_consensus.cc:2468] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.566666 11738 ts_tablet_manager.cc:1434] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:06:14.568559 11738 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.568971 11743 raft_consensus.cc:493] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.569051 11744 leader_election.cc:290] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.569598 11743 raft_consensus.cc:515] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.572386 11743 leader_election.cc:290] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.568146 11586 leader_election.cc:304] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:14.574492 11743 raft_consensus.cc:2804] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.575027 11743 raft_consensus.cc:493] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.575409 11743 raft_consensus.cc:3060] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.578392 11738 tablet_bootstrap.cc:654] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.578994 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1937785aba94020a04aa593cb885143" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.579887 11694 raft_consensus.cc:2468] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.581756 11586 leader_election.cc:304] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:14.556542 11737 tablet_bootstrap.cc:654] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.586426 11759 raft_consensus.cc:2804] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.587234 11759 raft_consensus.cc:493] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.587747 11759 raft_consensus.cc:3060] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.610608 11736 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.611702 11736 ts_tablet_manager.cc:1403] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.071s	user 0.022s	sys 0.042s
I20260709 13:06:14.623530 11737 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.625434 11737 ts_tablet_manager.cc:1403] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.092s	user 0.026s	sys 0.000s
I20260709 13:06:14.626230 11759 raft_consensus.cc:515] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.621376 11766 raft_consensus.cc:493] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.628930 11766 raft_consensus.cc:515] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.630152 11743 raft_consensus.cc:515] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.636301 11743 leader_election.cc:290] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.636034 11737 raft_consensus.cc:359] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.638127 11737 raft_consensus.cc:385] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.638523 11736 raft_consensus.cc:359] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.638721 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:14.638705 11737 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.641571 11738 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.640283 11736 raft_consensus.cc:385] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.642357 11738 ts_tablet_manager.cc:1403] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.074s	user 0.025s	sys 0.028s
I20260709 13:06:14.640606 11694 raft_consensus.cc:3060] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.644622 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1937785aba94020a04aa593cb885143" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:14.645903 11693 raft_consensus.cc:3060] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.648130 11738 raft_consensus.cc:359] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.648854 11744 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.649241 11738 raft_consensus.cc:385] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.649628 11738 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.649477 11744 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.650625 11738 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.652148 11744 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.654184 11738 ts_tablet_manager.cc:1434] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 13:06:14.655627 11738 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.642380 11736 raft_consensus.cc:740] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.660928 11737 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.660946 11743 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.663043 11743 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:14.664213 11737 ts_tablet_manager.cc:1434] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.004s
I20260709 13:06:14.666117 11743 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.671993 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.666738 11737 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.676472 11759 leader_election.cc:290] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.679473 11693 raft_consensus.cc:2468] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
W20260709 13:06:14.675886 11586 leader_election.cc:343] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.681493 11586 leader_election.cc:304] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:14.682606 11759 raft_consensus.cc:2804] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:14.683318 11759 raft_consensus.cc:697] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:14.675967 11766 leader_election.cc:290] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.684798 11759 consensus_queue.cc:237] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.685714 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1937785aba94020a04aa593cb885143" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.688081 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:14.682106 11736 consensus_queue.cc:260] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.691543 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.694923 11738 tablet_bootstrap.cc:654] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.693346 11545 raft_consensus.cc:3060] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.695227 11694 raft_consensus.cc:2468] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:14.696648 11736 ts_tablet_manager.cc:1434] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.007s
I20260709 13:06:14.695104 11544 raft_consensus.cc:2468] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.696964 11586 leader_election.cc:304] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:14.697983 11766 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.698148 11743 raft_consensus.cc:2804] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:14.698359 11736 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.698567 11766 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.693848 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.711292 11766 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.712126 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.712473 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1937785aba94020a04aa593cb885143" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
W20260709 13:06:14.713338 11586 leader_election.cc:343] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:14.714553 11586 leader_election.cc:343] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.715126 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.715899 11545 raft_consensus.cc:2468] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
W20260709 13:06:14.717965 11586 leader_election.cc:343] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.718350 11736 tablet_bootstrap.cc:654] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.723594 11586 leader_election.cc:304] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.724663 11737 tablet_bootstrap.cc:654] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.725696 11766 raft_consensus.cc:2804] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.726439 11766 raft_consensus.cc:493] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.728667 11766 raft_consensus.cc:3060] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.728587 11744 raft_consensus.cc:493] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.719090 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
W20260709 13:06:14.734304 11586 leader_election.cc:343] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.733842 11744 raft_consensus.cc:515] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.733870 11544 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.727035 11743 raft_consensus.cc:697] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
W20260709 13:06:14.738266 11586 leader_election.cc:343] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.743088 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:14.745651 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.746227 11618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60b2e1c3304bafaa8826f684b0c3de" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:14.738893 11586 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa; no voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:14.745653 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:14.748637 11620 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 0.
I20260709 13:06:14.739225 11743 consensus_queue.cc:237] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.747200 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60b2e1c3304bafaa8826f684b0c3de" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.738541 11744 leader_election.cc:290] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.746789 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.753614 11738 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.754927 11738 ts_tablet_manager.cc:1403] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.007s
I20260709 13:06:14.758037 11769 raft_consensus.cc:493] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.758839 11769 raft_consensus.cc:515] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.761780 11769 leader_election.cc:290] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.762398 11769 raft_consensus.cc:2749] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.757040 11766 raft_consensus.cc:515] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.763991 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.765653 11619 raft_consensus.cc:2393] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6999ef0c57c494b9e4774bd01b6054d in current term 1: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:14.755865 11443 catalog_manager.cc:5697] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.765658 11693 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.769843 11586 leader_election.cc:304] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:14.770332 11766 leader_election.cc:290] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.771322 11766 raft_consensus.cc:2804] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.772171 11766 raft_consensus.cc:493] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.772899 11766 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.773304 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
W20260709 13:06:14.756913 11511 leader_election.cc:343] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:14.775632 11511 leader_election.cc:343] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:14.776142 11586 leader_election.cc:343] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:14.776474 11511 leader_election.cc:343] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:14.777361 11511 leader_election.cc:343] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.777885 11511 leader_election.cc:304] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6999ef0c57c494b9e4774bd01b6054d; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.778837 11744 raft_consensus.cc:2749] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.779192 11736 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.779341 11511 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.779934 11736 ts_tablet_manager.cc:1403] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.006s
I20260709 13:06:14.780499 11744 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.781010 11744 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.781368 11744 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.783088 11738 raft_consensus.cc:359] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.783875 11766 raft_consensus.cc:515] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.783941 11736 raft_consensus.cc:359] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.785611 11736 raft_consensus.cc:385] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.786152 11736 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.787379 11766 leader_election.cc:290] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.787576 11736 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.790302 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:14.790921 11511 leader_election.cc:304] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6999ef0c57c494b9e4774bd01b6054d; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.791229 11693 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.799659 11737 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.801774 11693 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:14.802850 11737 ts_tablet_manager.cc:1403] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.137s	user 0.006s	sys 0.018s
I20260709 13:06:14.804032 11586 leader_election.cc:304] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:14.805357 11743 raft_consensus.cc:2804] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:14.801936 11744 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.806033 11743 raft_consensus.cc:697] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:14.807274 11743 consensus_queue.cc:237] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.808106 11744 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Requested vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.808892 11737 raft_consensus.cc:359] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.809814 11776 raft_consensus.cc:3060] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.809855 11737 raft_consensus.cc:385] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.811589 11737 raft_consensus.cc:740] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.811813 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.813112 11737 consensus_queue.cc:260] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.813467 11766 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.814087 11766 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.814599 11766 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.816815 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:14.816154 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:14.819934 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.820880 11544 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.784112 11738 raft_consensus.cc:385] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.821492 11738 raft_consensus.cc:740] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.822692 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:14.822526 11738 consensus_queue.cc:260] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
W20260709 13:06:14.825733 11511 leader_election.cc:343] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.824288 11766 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.832051 11619 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a6999ef0c57c494b9e4774bd01b6054d in current term 1: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:14.824540 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:14.826999 11737 ts_tablet_manager.cc:1434] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:06:14.807274 11736 ts_tablet_manager.cc:1434] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.001s
I20260709 13:06:14.834118 11545 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.838424 11758 raft_consensus.cc:493] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.839125 11736 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.839006 11758 raft_consensus.cc:515] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.834910 11766 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.835287 11776 raft_consensus.cc:2749] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.836444 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:14.843273 11543 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 40e9f867c7af439bbfca87d7e3f03afa in current term 1: Already voted for candidate a6999ef0c57c494b9e4774bd01b6054d in this term.
I20260709 13:06:14.844116 11758 leader_election.cc:290] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:14.844779 11737 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.836442 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
W20260709 13:06:14.847815 11586 leader_election.cc:343] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.851119 11738 ts_tablet_manager.cc:1434] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.096s	user 0.010s	sys 0.000s
I20260709 13:06:14.853404 11738 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.853266 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:14.855966 11619 raft_consensus.cc:2393] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a16fca66e4404a01911ecd98feb63dcd in current term 1: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:14.858515 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.859508 11544 raft_consensus.cc:2468] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:14.859397 11545 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:14.837184 11511 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6999ef0c57c494b9e4774bd01b6054d; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.860798 11660 leader_election.cc:304] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:14.862102 11758 raft_consensus.cc:3060] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.861924 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa; no voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:14.863857 11738 tablet_bootstrap.cc:654] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.866783 11776 raft_consensus.cc:2749] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.869699 11586 leader_election.cc:304] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:14.871134 11737 tablet_bootstrap.cc:654] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.871253 11743 raft_consensus.cc:2804] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:14.871215 11766 raft_consensus.cc:2749] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.872264 11743 raft_consensus.cc:697] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:14.874190 11743 consensus_queue.cc:237] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.878090 11769 raft_consensus.cc:493] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.878619 11736 tablet_bootstrap.cc:654] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.879103 11769 raft_consensus.cc:515] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.886750 11737 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.887588 11737 ts_tablet_manager.cc:1403] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.015s
I20260709 13:06:14.890445 11738 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:14.891022 11738 ts_tablet_manager.cc:1403] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.009s
I20260709 13:06:14.896423 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:14.896276 11737 raft_consensus.cc:359] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.897549 11693 raft_consensus.cc:2468] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:14.897675 11737 raft_consensus.cc:385] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.898291 11737 raft_consensus.cc:740] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.899397 11586 leader_election.cc:304] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:14.899363 11737 consensus_queue.cc:260] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.900682 11743 raft_consensus.cc:2804] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:14.900928 11758 raft_consensus.cc:2749] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:14.901237 11743 raft_consensus.cc:493] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.901746 11743 raft_consensus.cc:3060] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.902580 11737 ts_tablet_manager.cc:1434] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.005s
I20260709 13:06:14.895000 11769 leader_election.cc:290] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.906430 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:14.909107 11737 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
W20260709 13:06:14.909389 11586 leader_election.cc:343] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.919852 11743 raft_consensus.cc:515] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.926304 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:14.928570 11737 tablet_bootstrap.cc:654] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:14.928763 11586 leader_election.cc:343] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:14.932368 11743 leader_election.cc:290] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:14.935038 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:14.936012 11693 raft_consensus.cc:3060] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:14.937392 11736 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.938267 11736 ts_tablet_manager.cc:1403] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.022s
I20260709 13:06:14.941263 11738 raft_consensus.cc:359] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.942308 11738 raft_consensus.cc:385] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.942906 11738 raft_consensus.cc:740] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.943725 11736 raft_consensus.cc:359] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.945494 11736 raft_consensus.cc:385] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.946002 11736 raft_consensus.cc:740] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.947252 11736 consensus_queue.cc:260] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.949054 11693 raft_consensus.cc:2468] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:14.947916 11443 catalog_manager.cc:5697] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.950826 11586 leader_election.cc:304] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:14.951910 11736 ts_tablet_manager.cc:1434] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.004s
I20260709 13:06:14.952383 11743 raft_consensus.cc:2804] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:14.951550 11443 catalog_manager.cc:5697] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.948719 11738 consensus_queue.cc:260] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.953447 11743 raft_consensus.cc:697] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:14.953985 11736 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:14.955462 11743 consensus_queue.cc:237] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.957804 11443 catalog_manager.cc:5697] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.956961 11738 ts_tablet_manager.cc:1434] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.008s
I20260709 13:06:14.964423 11738 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:14.965217 11737 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:14.966920 11737 ts_tablet_manager.cc:1403] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.015s
I20260709 13:06:14.969188 11736 tablet_bootstrap.cc:654] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.970604 11737 raft_consensus.cc:359] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:14.971410 11737 raft_consensus.cc:385] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.971673 11738 tablet_bootstrap.cc:654] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.971818 11737 raft_consensus.cc:740] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.973001 11737 consensus_queue.cc:260] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:14.976143 11737 ts_tablet_manager.cc:1434] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:14.979256 11737 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:14.987805 11737 tablet_bootstrap.cc:654] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:14.987987 11736 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:14.989457 11736 ts_tablet_manager.cc:1403] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.036s	user 0.028s	sys 0.003s
I20260709 13:06:14.994576 11736 raft_consensus.cc:359] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:14.995402 11736 raft_consensus.cc:385] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:14.995728 11736 raft_consensus.cc:740] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:14.996752 11736 consensus_queue.cc:260] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.000891 11736 ts_tablet_manager.cc:1434] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.001s
I20260709 13:06:15.002661 11736 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:15.011837 11736 tablet_bootstrap.cc:654] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:15.021427 11737 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:15.022367 11737 ts_tablet_manager.cc:1403] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.043s	user 0.023s	sys 0.009s
I20260709 13:06:15.024588 11776 raft_consensus.cc:493] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.025568 11776 raft_consensus.cc:515] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.026918 11737 raft_consensus.cc:359] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.028311 11737 raft_consensus.cc:385] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.028764 11737 raft_consensus.cc:740] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.030071 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:15.030285 11737 consensus_queue.cc:260] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.033078 11736 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:15.033896 11736 ts_tablet_manager.cc:1403] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.000s
I20260709 13:06:15.033830 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:15.035214 11693 raft_consensus.cc:2393] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6999ef0c57c494b9e4774bd01b6054d in current term 1: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:15.037674 11511 leader_election.cc:304] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6999ef0c57c494b9e4774bd01b6054d; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:15.038170 11736 raft_consensus.cc:359] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:15.038825 11744 raft_consensus.cc:3060] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.041492 11737 ts_tablet_manager.cc:1434] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.003s
I20260709 13:06:15.043305 11737 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: Bootstrap starting.
I20260709 13:06:15.045161 11776 leader_election.cc:290] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:15.054189 11736 raft_consensus.cc:385] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.054667 11736 raft_consensus.cc:740] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.055630 11736 consensus_queue.cc:260] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:15.061203 11737 tablet_bootstrap.cc:654] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:15.065944 11738 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:15.066592 11738 ts_tablet_manager.cc:1403] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.000s
I20260709 13:06:15.057695 11744 raft_consensus.cc:2749] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:15.069731 11736 ts_tablet_manager.cc:1434] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:06:15.071704 11736 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:15.072556 11443 catalog_manager.cc:5697] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.081919 11736 tablet_bootstrap.cc:654] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:15.097203 11738 raft_consensus.cc:359] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.099067 11738 raft_consensus.cc:385] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.099464 11738 raft_consensus.cc:740] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.099823 11736 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:15.100292 11738 consensus_queue.cc:260] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.102442 11744 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.103963 11738 ts_tablet_manager.cc:1434] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:06:15.103173 11744 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:15.104602 11736 ts_tablet_manager.cc:1403] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.004s
I20260709 13:06:15.107050 11744 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:15.108202 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:15.108991 11738 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa: Bootstrap starting.
I20260709 13:06:15.109308 11693 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 0.
I20260709 13:06:15.111567 11511 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.111932 11737 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: No bootstrap required, opened a new log
I20260709 13:06:15.112910 11744 raft_consensus.cc:2804] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:15.113663 11744 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.114223 11744 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.115090 11545 raft_consensus.cc:3060] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.116734 11738 tablet_bootstrap.cc:654] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:15.113549 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:15.118861 11619 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 0.
I20260709 13:06:15.126063 11744 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:15.112874 11737 ts_tablet_manager.cc:1403] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: Time spent bootstrapping tablet: real 0.070s	user 0.028s	sys 0.024s
I20260709 13:06:15.111567 11736 raft_consensus.cc:359] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.129094 11736 raft_consensus.cc:385] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.129482 11736 raft_consensus.cc:740] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.130295 11736 consensus_queue.cc:260] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.130686 11743 consensus_queue.cc:1048] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.131747 11737 raft_consensus.cc:359] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.132575 11737 raft_consensus.cc:385] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.132730 11736 ts_tablet_manager.cc:1434] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.001s
I20260709 13:06:15.132876 11737 raft_consensus.cc:740] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.134099 11737 consensus_queue.cc:260] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.135833 11744 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Requested vote from peers a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:15.134099 11736 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:15.141525 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:15.142803 11619 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.144124 11736 tablet_bootstrap.cc:654] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:15.144822 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:15.146236 11693 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:15.150903 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:15.157804 11737 ts_tablet_manager.cc:1434] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:06:15.159255 11736 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: No bootstrap required, opened a new log
I20260709 13:06:15.160100 11736 ts_tablet_manager.cc:1403] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.008s
I20260709 13:06:15.164072 11736 raft_consensus.cc:359] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.168532 11736 raft_consensus.cc:385] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.165205 11693 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 1.
I20260709 13:06:15.169700 11736 raft_consensus.cc:740] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.172606 11511 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.173722 11744 raft_consensus.cc:2804] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:15.174420 11744 raft_consensus.cc:697] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 LEADER]: Becoming Leader. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Running, Role: LEADER
I20260709 13:06:15.175623 11744 consensus_queue.cc:237] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:15.177945 11743 consensus_queue.cc:1048] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.178778 11741 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.179373 11741 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.181835 11741 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:15.185973 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:15.187992 11620 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a16fca66e4404a01911ecd98feb63dcd in current term 1: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:15.188813 11619 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 1.
I20260709 13:06:15.191779 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:15.173734 11736 consensus_queue.cc:260] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.194550 11544 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a16fca66e4404a01911ecd98feb63dcd in current term 1: Already voted for candidate a6999ef0c57c494b9e4774bd01b6054d in this term.
I20260709 13:06:15.195856 11736 ts_tablet_manager.cc:1434] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:06:15.196719 11660 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:15.197995 11758 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.198504 11738 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa: No bootstrap required, opened a new log
I20260709 13:06:15.199074 11738 ts_tablet_manager.cc:1403] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent bootstrapping tablet: real 0.090s	user 0.032s	sys 0.019s
I20260709 13:06:15.204648 11738 raft_consensus.cc:359] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.205535 11738 raft_consensus.cc:385] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:15.205883 11738 raft_consensus.cc:740] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Initialized, Role: FOLLOWER
I20260709 13:06:15.206782 11738 consensus_queue.cc:260] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.213320 11758 raft_consensus.cc:2749] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:15.220057 11738 ts_tablet_manager.cc:1434] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:06:15.227658 11443 catalog_manager.cc:5697] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d reported cstate change: term changed from 0 to 1, leader changed from <none> to a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6999ef0c57c494b9e4774bd01b6054d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.238252 11766 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.253903 11766 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.285983 11758 raft_consensus.cc:493] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.286610 11758 raft_consensus.cc:515] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.288805 11758 leader_election.cc:290] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:15.290210 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:15.290697 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:15.291064 11545 raft_consensus.cc:2468] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 0.
I20260709 13:06:15.291810 11619 raft_consensus.cc:2468] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 0.
I20260709 13:06:15.292740 11660 leader_election.cc:304] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.293917 11758 raft_consensus.cc:2804] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:15.294238 11766 raft_consensus.cc:493] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.294502 11758 raft_consensus.cc:493] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.294932 11758 raft_consensus.cc:3060] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.294837 11766 raft_consensus.cc:515] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.297891 11766 leader_election.cc:290] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:15.301090 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60b2e1c3304bafaa8826f684b0c3de" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:15.302649 11545 raft_consensus.cc:2468] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:15.302788 11766 consensus_queue.cc:1048] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.304488 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60b2e1c3304bafaa8826f684b0c3de" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:15.304481 11743 consensus_queue.cc:1048] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.305467 11694 raft_consensus.cc:2468] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 0.
I20260709 13:06:15.306509 11758 raft_consensus.cc:515] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.309312 11586 leader_election.cc:304] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.310535 11743 raft_consensus.cc:2804] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:15.311211 11743 raft_consensus.cc:493] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.311447 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:15.311822 11743 raft_consensus.cc:3060] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.312400 11544 raft_consensus.cc:3060] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.313320 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:15.314126 11619 raft_consensus.cc:3060] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.323625 11619 raft_consensus.cc:2468] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 1.
I20260709 13:06:15.323959 11766 consensus_queue.cc:1048] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.325654 11660 leader_election.cc:304] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:15.326923 11741 raft_consensus.cc:2804] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:15.326988 11544 raft_consensus.cc:2468] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 1.
I20260709 13:06:15.327674 11743 raft_consensus.cc:515] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.330744 11743 leader_election.cc:290] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:15.333429 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60b2e1c3304bafaa8826f684b0c3de" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:15.333815 11758 leader_election.cc:290] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 1 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:15.334475 11544 raft_consensus.cc:3060] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.334784 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60b2e1c3304bafaa8826f684b0c3de" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:15.335656 11693 raft_consensus.cc:3060] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.339910 11744 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.340672 11744 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.352818 11744 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:15.354743 11741 raft_consensus.cc:697] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:15.357559 11544 raft_consensus.cc:2468] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:15.359753 11586 leader_election.cc:304] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.361311 11743 raft_consensus.cc:2804] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:15.361918 11743 raft_consensus.cc:697] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:15.362804 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:15.362993 11743 consensus_queue.cc:237] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.363677 11694 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 0.
I20260709 13:06:15.365741 11693 raft_consensus.cc:2468] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:15.366310 11511 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.367221 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:15.368127 11744 raft_consensus.cc:2804] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:15.368666 11744 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.369046 11744 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.368145 11619 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 0.
I20260709 13:06:15.380599 11807 consensus_queue.cc:1048] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.383816 11744 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.357817 11741 consensus_queue.cc:237] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.389981 11744 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Requested vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:15.393527 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:15.393796 11619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:15.394917 11693 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.395301 11619 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:15.412828 11693 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 1.
I20260709 13:06:15.419687 11619 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 1.
I20260709 13:06:15.424685 11511 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:15.426826 11744 raft_consensus.cc:2804] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:15.427531 11744 raft_consensus.cc:697] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 LEADER]: Becoming Leader. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Running, Role: LEADER
I20260709 13:06:15.428644 11744 consensus_queue.cc:237] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:15.435070 11766 consensus_queue.cc:1048] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.438266 11442 catalog_manager.cc:5697] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 0 to 1, leader changed from <none> to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 1 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.446471 11440 catalog_manager.cc:5697] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 0 to 1, leader changed from <none> to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:15.449832 11443 catalog_manager.cc:5697] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d reported cstate change: term changed from 0 to 1, leader changed from <none> to a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6999ef0c57c494b9e4774bd01b6054d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:15.461308 11807 consensus_queue.cc:1048] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:15.563325 11744 consensus_queue.cc:1048] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:15.581164 11816 consensus_queue.cc:1048] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.771679 11741 consensus_queue.cc:1048] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:15.873622 11805 consensus_queue.cc:1048] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:06:15.897774 11816 consensus_queue.cc:1048] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:15.929500 11743 consensus_queue.cc:1048] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.934000 11766 consensus_queue.cc:1048] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:15.995481 11816 consensus_queue.cc:1048] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:06:16.155766 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:16.507006 11766 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.509516 11766 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:16.514806 11766 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:16.517335 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:16.518764 11694 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:16.529474 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:16.533619 11766 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:16.534437 11766 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.535166 11766 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:16.742110 11827 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.743486 11827 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:16.749503 11827 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:16.822918 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:16.822983 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:16.824783 11693 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 1.
I20260709 13:06:16.826260 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d" is_pre_election: true
I20260709 13:06:16.827219 11543 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
I20260709 13:06:16.831408 11766 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:16.836625 11620 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6999ef0c57c494b9e4774bd01b6054d in current term 2: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:16.837212 11766 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:16.839970 11511 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:16.842670 11827 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:16.843261 11827 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.843701 11827 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:16.862933 11827 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:16.869481 11827 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 2 election: Requested vote from peers 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:16.873776 11693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:16.897578 11693 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:16.921018 11693 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6999ef0c57c494b9e4774bd01b6054d in term 2.
I20260709 13:06:17.018428 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:17.020879 11543 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 40e9f867c7af439bbfca87d7e3f03afa in current term 2: Already voted for candidate a6999ef0c57c494b9e4774bd01b6054d in this term.
I20260709 13:06:17.057744 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a6999ef0c57c494b9e4774bd01b6054d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:17.127437 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:17.131477 11692 raft_consensus.cc:2393] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 40e9f867c7af439bbfca87d7e3f03afa in current term 2: Already voted for candidate a6999ef0c57c494b9e4774bd01b6054d in this term.
I20260709 13:06:17.142827 11511 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 40e9f867c7af439bbfca87d7e3f03afa
I20260709 13:06:17.143850 11834 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:17.143841 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa; no voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:17.144508 11834 raft_consensus.cc:697] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 LEADER]: Becoming Leader. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Running, Role: LEADER
I20260709 13:06:17.145409 11833 raft_consensus.cc:2749] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:17.146380 11834 consensus_queue.cc:237] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
W20260709 13:06:17.162062 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:17.162072 11442 catalog_manager.cc:5697] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d reported cstate change: term changed from 0 to 2, leader changed from <none> to a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a6999ef0c57c494b9e4774bd01b6054d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.745689 11692 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Refusing update from remote peer a6999ef0c57c494b9e4774bd01b6054d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:17.747615 11834 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:17.772774 11620 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a6999ef0c57c494b9e4774bd01b6054d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:17.775260 11827 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:17.782135   431 ts_manager.cc:295] Set tserver state for a6999ef0c57c494b9e4774bd01b6054d to MAINTENANCE_MODE
I20260709 13:06:17.784031   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:17.859748   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:17.861354   431 tablet_replica.cc:333] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.862617   431 raft_consensus.cc:2243] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.863240   431 raft_consensus.cc:2272] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.865934   431 tablet_replica.cc:333] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.866696   431 raft_consensus.cc:2243] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.867806   431 raft_consensus.cc:2272] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.872519   431 tablet_replica.cc:333] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.873484   431 raft_consensus.cc:2243] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.874146   431 raft_consensus.cc:2272] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.877933   431 tablet_replica.cc:333] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.878851   431 raft_consensus.cc:2243] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:17.880050   431 raft_consensus.cc:2272] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.883219   431 tablet_replica.cc:333] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.884047   431 raft_consensus.cc:2243] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.885242   431 raft_consensus.cc:2272] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:17.890381 11586 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 152 similar messages]
I20260709 13:06:17.890926   431 tablet_replica.cc:333] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.891920   431 raft_consensus.cc:2243] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:17.893405   431 raft_consensus.cc:2272] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:17.903366 11586 consensus_peers.cc:597] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:17.904240   431 tablet_replica.cc:333] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.905352   431 raft_consensus.cc:2243] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.906241   431 raft_consensus.cc:2272] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.910833   431 tablet_replica.cc:333] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.911641   431 raft_consensus.cc:2243] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:17.912832   431 raft_consensus.cc:2272] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.916199   431 tablet_replica.cc:333] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.916986   431 raft_consensus.cc:2243] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.917747   431 raft_consensus.cc:2272] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.921631   431 tablet_replica.cc:333] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: stopping tablet replica
I20260709 13:06:17.922483   431 raft_consensus.cc:2243] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.923205   431 raft_consensus.cc:2272] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:17.934034 11566 debug-util.cc:398] Leaking SignalData structure 0x7b0800275620 after lost signal to thread 434
W20260709 13:06:17.935719 11566 debug-util.cc:398] Leaking SignalData structure 0x7b0800096880 after lost signal to thread 11477
W20260709 13:06:17.937613 11566 debug-util.cc:398] Leaking SignalData structure 0x7b0800239360 after lost signal to thread 11569
W20260709 13:06:17.939139 11566 debug-util.cc:398] Leaking SignalData structure 0x7b08000c41e0 after lost signal to thread 11644
W20260709 13:06:17.940595 11566 debug-util.cc:398] Leaking SignalData structure 0x7b08000ac060 after lost signal to thread 11718
W20260709 13:06:18.021826 11586 consensus_peers.cc:597] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.028038 11586 consensus_peers.cc:597] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.041002 11586 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.151132 11660 consensus_peers.cc:597] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.163573 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:18.233172 11586 consensus_peers.cc:597] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.234019 11586 consensus_peers.cc:597] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:18.970825   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11566)
W20260709 13:06:19.165591 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:19.207640 11856 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.208714 11856 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:19.210443 11857 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.211280 11857 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:19.212261 11856 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:19.215669 11857 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:19.217077 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:19.218175 11620 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 1.
I20260709 13:06:19.218559 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:19.219444 11692 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 1.
W20260709 13:06:19.220497 11660 leader_election.cc:336] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.221220 11586 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:19.221588 11660 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.222499 11857 raft_consensus.cc:2804] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.222991 11857 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting leader election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.223439 11857 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.223469 11856 raft_consensus.cc:2804] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:19.224061 11856 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting leader election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.224503 11856 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:19.233394 11586 leader_election.cc:336] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.239502 11856 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:19.239893 11857 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:19.242529 11857 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:19.242573 11856 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:19.245071 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:19.246115 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:19.246925 11692 raft_consensus.cc:2393] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 40e9f867c7af439bbfca87d7e3f03afa in current term 2: Already voted for candidate a16fca66e4404a01911ecd98feb63dcd in this term.
I20260709 13:06:19.247296 11620 raft_consensus.cc:2393] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a16fca66e4404a01911ecd98feb63dcd in current term 2: Already voted for candidate 40e9f867c7af439bbfca87d7e3f03afa in this term.
I20260709 13:06:19.252072 11853 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.253011 11853 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
W20260709 13:06:19.254002 11660 leader_election.cc:336] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
W20260709 13:06:19.255607 11586 leader_election.cc:336] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.256685 11586 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa; no voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.257155 11853 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:19.254777 11660 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.258150 11857 raft_consensus.cc:2749] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.259479 11856 raft_consensus.cc:2749] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.261265 11856 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.261430 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:19.261956 11856 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.265595 11856 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
W20260709 13:06:19.266115 11586 leader_election.cc:336] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.267608 11586 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa; no voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.268618 11853 raft_consensus.cc:2749] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:19.270581 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
W20260709 13:06:19.274051 11660 leader_election.cc:336] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.274726 11660 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd; no voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.275810 11856 raft_consensus.cc:2749] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:06:19.280853 11853 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.282073 11853 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.284659 11853 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:19.286725 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd" is_pre_election: true
I20260709 13:06:19.287963 11692 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 2.
I20260709 13:06:19.289920 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
W20260709 13:06:19.291092 11586 leader_election.cc:336] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.291841 11853 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:06:19.292681 11853 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting leader election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.293169 11853 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.307848 11853 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.310727 11853 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:19.314577 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:19.315685 11692 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:19.317138 11586 leader_election.cc:336] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.323457 11692 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 3.
I20260709 13:06:19.324793 11856 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.325517 11856 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.325806 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.326918 11853 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:19.328305 11856 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:19.329130 11853 raft_consensus.cc:697] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:19.330549 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:19.330556 11853 consensus_queue.cc:237] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.331665 11620 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 1.
I20260709 13:06:19.334427 11660 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:19.335831 11856 raft_consensus.cc:2804] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:06:19.335889 11660 leader_election.cc:336] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.336705 11856 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting leader election (detected failure of leader a6999ef0c57c494b9e4774bd01b6054d)
I20260709 13:06:19.337291 11856 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.344838 11440 catalog_manager.cc:5697] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 2 to 3, leader changed from a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193) to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 3 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:19.352676 11856 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.358774 11856 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:19.362224 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:19.363126 11620 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:19.365234 11660 leader_election.cc:336] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:19.378297 11620 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:19.380751 11660 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:19.381853 11856 raft_consensus.cc:2804] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:19.383855 11856 raft_consensus.cc:697] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:19.385406 11856 consensus_queue.cc:237] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:19.400929 11440 catalog_manager.cc:5697] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 2, leader changed from a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.767323 11620 raft_consensus.cc:1275] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.769850 11856 consensus_queue.cc:1048] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:19.801826 11660 consensus_peers.cc:597] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:19.827349 11692 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:19.829363 11853 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:19.850917 11586 consensus_peers.cc:597] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:19.971594   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11566)
W20260709 13:06:20.167260 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:20.332291 11586 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:20.495846 11660 consensus_peers.cc:597] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:20.519941 11586 consensus_peers.cc:597] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:20.591758 11586 consensus_peers.cc:597] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:20.699077 11586 consensus_peers.cc:597] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:20.754317 11586 consensus_peers.cc:597] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:20.766937 11586 consensus_peers.cc:597] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:20.780683 11863 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.781558 11863 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:20.783774 11863 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:20.786924 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa" is_pre_election: true
I20260709 13:06:20.788285 11620 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:20.789966 11660 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: 
I20260709 13:06:20.790863 11863 raft_consensus.cc:2804] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:06:20.791239 11660 leader_election.cc:336] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:20.791353 11863 raft_consensus.cc:493] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.792259 11863 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:20.804652 11863 raft_consensus.cc:515] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:20.807881 11863 leader_election.cc:290] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:20.811211 11620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e242cfc1aa04b228548b73077358cef" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:20.811995 11620 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:06:20.814441 11660 leader_election.cc:336] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111)
I20260709 13:06:20.819905 11620 raft_consensus.cc:2468] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 3.
I20260709 13:06:20.821707 11660 leader_election.cc:304] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a16fca66e4404a01911ecd98feb63dcd; no voters: a6999ef0c57c494b9e4774bd01b6054d
I20260709 13:06:20.822855 11863 raft_consensus.cc:2804] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:20.823855 11863 raft_consensus.cc:697] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 3 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:20.825232 11863 consensus_queue.cc:237] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:20.839761 11440 catalog_manager.cc:5697] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 3, leader changed from a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:20.972695   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 11566)
W20260709 13:06:21.169065 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:21.217732 11620 raft_consensus.cc:1275] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:21.224759 11872 consensus_queue.cc:1048] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:06:21.250237 11660 consensus_peers.cc:597] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:21.973592   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 11566)
W20260709 13:06:22.162794 11660 consensus_peers.cc:597] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:22.171228 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:22.185851 11586 consensus_peers.cc:597] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:22.365540   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:22.696111 11586 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:22.839272 11586 consensus_peers.cc:597] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:22.946347 11586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111) [suppressed 100 similar messages]
W20260709 13:06:23.159736 11586 consensus_peers.cc:597] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:23.173058 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:23.314366 11586 consensus_peers.cc:597] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:23.323731 11586 consensus_peers.cc:597] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:23.396126 11586 consensus_peers.cc:597] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:23.457852 11660 consensus_peers.cc:597] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:23.718003 11660 consensus_peers.cc:597] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:24.174444 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:24.400736 11586 consensus_peers.cc:597] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:24.563259 11660 consensus_peers.cc:597] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:25.176177 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:25.199640 11586 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:25.220196 11586 consensus_peers.cc:597] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:25.661463 11586 consensus_peers.cc:597] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:25.761175 11660 consensus_peers.cc:597] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:25.921624 11586 consensus_peers.cc:597] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:26.050664 11586 consensus_peers.cc:597] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:26.178179 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:06:26.235062 11586 consensus_peers.cc:597] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:26.374089 11660 consensus_peers.cc:597] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:27.025511 11660 consensus_peers.cc:597] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:27.138195 11586 consensus_peers.cc:597] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:27.179566 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:27.413877   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:27.424201 11888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:27.424222 11889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:27.429890   431 server_base.cc:1061] running on GCE node
W20260709 13:06:27.429759 11891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:27.431854   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:27.432184   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:27.432452   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602387432429 us; error 0 us; skew 500 ppm
I20260709 13:06:27.433264   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:27.437263   431 webserver.cc:533] Webserver started at http://127.0.107.193:33361/ using document root <none> and password file <none>
I20260709 13:06:27.437954   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:27.438186   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:06:27.443315   431 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:27.449419 11896 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:27.450486   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:27.450922   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "a6999ef0c57c494b9e4774bd01b6054d"
format_stamp: "Formatted at 2026-07-09 13:06:13 on dist-test-slave-ftjc"
I20260709 13:06:27.451620   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:27.479156   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:27.480877   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:27.482849   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:27.488070 11903 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:06:27.570924   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:06:27.571350   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.085s	user 0.004s	sys 0.000s
I20260709 13:06:27.571823   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:06:27.580876 11903 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
W20260709 13:06:27.599246 11586 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:27.600798 11903 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.601795 11903 tablet_bootstrap.cc:492] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.602600 11903 ts_tablet_manager.cc:1403] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.005s
I20260709 13:06:27.605705 11903 raft_consensus.cc:359] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.606981 11903 raft_consensus.cc:740] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.607802 11903 consensus_queue.cc:260] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.611481 11903 ts_tablet_manager.cc:1434] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.007s
I20260709 13:06:27.612907 11903 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.631489 11903 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.632767 11903 tablet_bootstrap.cc:492] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.633564 11903 ts_tablet_manager.cc:1403] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 13:06:27.636345 11903 raft_consensus.cc:359] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.637221 11903 raft_consensus.cc:740] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.639322 11903 consensus_queue.cc:260] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.640407   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:06:27.642076   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.070s	user 0.061s	sys 0.006s
I20260709 13:06:27.643081 11903 ts_tablet_manager.cc:1434] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:27.644515 11903 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.667917 11903 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.669467 11903 tablet_bootstrap.cc:492] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.670342 11903 ts_tablet_manager.cc:1403] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.008s
I20260709 13:06:27.672997 11903 raft_consensus.cc:359] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.673686 11903 raft_consensus.cc:740] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.674491 11903 consensus_queue.cc:260] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.676332 11903 ts_tablet_manager.cc:1434] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:27.677148 11903 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.697158 11903 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.698148 11903 tablet_bootstrap.cc:492] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.698800 11903 ts_tablet_manager.cc:1403] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:06:27.701052 11903 raft_consensus.cc:359] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.701650 11903 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.702380 11903 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.704859 11903 ts_tablet_manager.cc:1434] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:27.706380 11903 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.726294 11903 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.727447 11903 tablet_bootstrap.cc:492] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.728219 11903 ts_tablet_manager.cc:1403] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:06:27.730950 11903 raft_consensus.cc:359] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.731853 11903 raft_consensus.cc:740] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.732192   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:39895
I20260709 13:06:27.732393 11965 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:39895 every 8 connection(s)
I20260709 13:06:27.732802 11903 consensus_queue.cc:260] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.735013 11903 ts_tablet_manager.cc:1434] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:27.736012 11903 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.741151   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:27.790069 11966 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43437
I20260709 13:06:27.789929 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:27.790719 11966 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:27.790947 11617 raft_consensus.cc:606] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:27.792309 11966 heartbeater.cc:507] Master 127.0.107.254:43437 requested a full tablet report, sending...
I20260709 13:06:27.799086 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1937785aba94020a04aa593cb885143"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:27.799815 11617 raft_consensus.cc:606] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:27.800369 11440 ts_manager.cc:194] Re-registered known tserver with Master: a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895)
I20260709 13:06:27.805943 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:27.807221 11617 raft_consensus.cc:606] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:27.809648   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:27.810678   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:27.812001   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:27.820416 11440 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59214
I20260709 13:06:27.821906 11903 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.823379 11903 tablet_bootstrap.cc:492] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.824272 11903 ts_tablet_manager.cc:1403] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.089s	user 0.030s	sys 0.042s
I20260709 13:06:27.828090 11903 raft_consensus.cc:359] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.829016 11903 raft_consensus.cc:740] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.829892 11903 consensus_queue.cc:260] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.831449 11966 heartbeater.cc:499] Master 127.0.107.254:43437 was elected leader, sending a full tablet report...
I20260709 13:06:27.832677 11903 ts_tablet_manager.cc:1434] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:27.833837 11903 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.853941 11903 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.855105 11903 tablet_bootstrap.cc:492] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.855798 11903 ts_tablet_manager.cc:1403] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.007s
I20260709 13:06:27.859042 11903 raft_consensus.cc:359] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.859853 11903 raft_consensus.cc:740] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.860612 11903 consensus_queue.cc:260] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.862607 11903 ts_tablet_manager.cc:1434] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:06:27.863646 11903 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.880752 11903 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.881613 11903 tablet_bootstrap.cc:492] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.882520 11903 ts_tablet_manager.cc:1403] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 13:06:27.885239 11903 raft_consensus.cc:359] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.886163 11903 raft_consensus.cc:740] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.887773 11903 consensus_queue.cc:260] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.889518 11903 ts_tablet_manager.cc:1434] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:27.890643 11903 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.906993 11903 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.908015 11903 tablet_bootstrap.cc:492] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.908777 11903 ts_tablet_manager.cc:1403] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.007s
I20260709 13:06:27.911691 11903 raft_consensus.cc:359] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.912366 11903 raft_consensus.cc:740] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.913115 11903 consensus_queue.cc:260] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:27.915535 11903 ts_tablet_manager.cc:1434] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:27.916690 11903 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap starting.
I20260709 13:06:27.932300 11903 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:27.933316 11903 tablet_bootstrap.cc:492] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Bootstrap complete.
I20260709 13:06:27.933983 11903 ts_tablet_manager.cc:1403] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.009s
I20260709 13:06:27.936265 11903 raft_consensus.cc:359] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:27.936970 11903 raft_consensus.cc:740] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6999ef0c57c494b9e4774bd01b6054d, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.937680 11903 consensus_queue.cc:260] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } }
I20260709 13:06:27.940779 11903 ts_tablet_manager.cc:1434] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:28.057658 11979 raft_consensus.cc:993] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:28.058256 11979 raft_consensus.cc:1081] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:28.060873 11692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:28.061674 11692 raft_consensus.cc:493] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:28.062001 11692 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.071924 11692 raft_consensus.cc:515] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.078085 11692 leader_election.cc:290] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:28.077747 11940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:28.079337 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:28.079854 11940 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.080205 11617 raft_consensus.cc:3055] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:28.081090 11617 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:28.082717 11617 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.086427 11617 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.092772 11940 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:28.094461 11660 leader_election.cc:304] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:28.095571 11863 raft_consensus.cc:2804] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.097631 11863 raft_consensus.cc:697] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:28.098989 11617 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:28.098929 11863 consensus_queue.cc:237] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.114121 11886 raft_consensus.cc:993] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:28.113458 11440 catalog_manager.cc:5697] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 2, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.114938 11979 raft_consensus.cc:1081] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:28.118407 11692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:28.119374 11692 raft_consensus.cc:493] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:28.119879 11692 raft_consensus.cc:3060] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.132939 11692 raft_consensus.cc:515] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.135411 11692 leader_election.cc:290] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:28.136646 11940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:28.137563 11940 raft_consensus.cc:3060] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.138337 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e8b00eb8004f938def8be94fd87474" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:28.139214 11617 raft_consensus.cc:3055] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:28.139801 11617 raft_consensus.cc:740] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:28.141351 11617 consensus_queue.cc:260] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.142783 11939 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.143029 11617 raft_consensus.cc:3060] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.145416 11940 raft_consensus.cc:2468] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:28.147230 11660 leader_election.cc:304] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:28.148182 11863 raft_consensus.cc:2804] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.149037 11863 raft_consensus.cc:697] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:28.150084 11863 consensus_queue.cc:237] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.154664 11617 raft_consensus.cc:2468] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:28.160377 11440 catalog_manager.cc:5697] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 2, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.180999 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:28.210166 11939 raft_consensus.cc:3060] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.257505 11939 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:28.259804 11991 raft_consensus.cc:993] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:28.260763 11991 raft_consensus.cc:1081] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:28.263348 11694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1937785aba94020a04aa593cb885143"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:28.263957 11694 raft_consensus.cc:493] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:28.264334 11694 raft_consensus.cc:3060] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.274797 11694 raft_consensus.cc:515] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.277209 11694 leader_election.cc:290] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:28.278378 11939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1937785aba94020a04aa593cb885143" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:28.278379 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1937785aba94020a04aa593cb885143" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:28.279197 11939 raft_consensus.cc:3060] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.279682 11617 raft_consensus.cc:3055] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:28.280471 11617 raft_consensus.cc:740] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:28.281621 11617 consensus_queue.cc:260] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.284590 11617 raft_consensus.cc:3060] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:28.293079 11939 raft_consensus.cc:2468] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:28.294486 11660 leader_election.cc:304] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:28.295435 11617 raft_consensus.cc:2468] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:28.295425 11863 raft_consensus.cc:2804] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:28.296808 11863 raft_consensus.cc:697] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:28.297947 11863 consensus_queue.cc:237] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:28.307853 11440 catalog_manager.cc:5697] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 2, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.526583 11617 raft_consensus.cc:1275] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.528577 11863 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:28.544677 11939 raft_consensus.cc:1275] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.546562 11863 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:28.600183 11617 raft_consensus.cc:1275] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.602257 11863 consensus_queue.cc:1048] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:28.617008 11939 raft_consensus.cc:1275] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.618993 12000 consensus_queue.cc:1048] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:28.713192 11617 raft_consensus.cc:1275] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.715283 11981 consensus_queue.cc:1048] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:28.717031 11939 raft_consensus.cc:1275] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:28.718753 11981 consensus_queue.cc:1048] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:28.832903   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:28.855997 11694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
mode: GRACEFUL
new_leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:53512
I20260709 13:06:28.856694 11694 raft_consensus.cc:606] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Received request to transfer leadership to TS 40e9f867c7af439bbfca87d7e3f03afa
I20260709 13:06:28.860877 11694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
mode: GRACEFUL
new_leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:53512
I20260709 13:06:28.861557 11694 raft_consensus.cc:606] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 LEADER]: Received request to transfer leadership to TS 40e9f867c7af439bbfca87d7e3f03afa
I20260709 13:06:28.863088   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:28.863768   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:28.864670   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:29.015080 11996 raft_consensus.cc:993] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: : Instructing follower 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:29.015870 11981 raft_consensus.cc:1081] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Signalling peer 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:29.018388 11620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:40150
I20260709 13:06:29.019132 11996 raft_consensus.cc:993] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: : Instructing follower 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:29.019086 11620 raft_consensus.cc:493] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.019848 11981 raft_consensus.cc:1081] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 LEADER]: Signalling peer 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:29.019840 11620 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.021854 11617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:40150
I20260709 13:06:29.022552 11617 raft_consensus.cc:493] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.022948 11617 raft_consensus.cc:3060] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.030941 11617 raft_consensus.cc:515] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.031013 11620 raft_consensus.cc:515] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.033900 11617 leader_election.cc:290] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:29.033928 11620 leader_election.cc:290] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:29.035219 11939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:29.035444 11940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:29.035991 11939 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.036196 11940 raft_consensus.cc:3060] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.036607 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:29.037510 11692 raft_consensus.cc:3055] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:29.037868 11692 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:29.039270 11692 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.041170 11692 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:29.044905 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cee0ad237fd241ad8c8554a0a05a50d8" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:29.045825 11694 raft_consensus.cc:3055] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.046257 11694 raft_consensus.cc:740] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:29.046537 11940 raft_consensus.cc:2468] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 2.
I20260709 13:06:29.048185 11586 leader_election.cc:304] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:29.048494 11939 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 3.
I20260709 13:06:29.048245 11694 consensus_queue.cc:260] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.049440 11991 raft_consensus.cc:2804] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:29.050160 11694 raft_consensus.cc:3060] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.050318 11586 leader_election.cc:304] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:29.051354 11886 raft_consensus.cc:2804] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:29.051808 11692 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 3.
I20260709 13:06:29.051940 11886 raft_consensus.cc:697] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:29.050861 11991 raft_consensus.cc:697] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:29.053074 11886 consensus_queue.cc:237] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.054111 11991 consensus_queue.cc:237] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.058276 11694 raft_consensus.cc:2468] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 2.
I20260709 13:06:29.064184 11443 catalog_manager.cc:5697] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 1 to 2, leader changed from a16fca66e4404a01911ecd98feb63dcd (127.0.107.195) to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 2 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.091277 11443 catalog_manager.cc:5697] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 2 to 3, leader changed from a16fca66e4404a01911ecd98feb63dcd (127.0.107.195) to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 3 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:29.182367 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:29.442111 11939 raft_consensus.cc:1275] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:29.443986 11991 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:29.456686 11694 raft_consensus.cc:1275] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:29.460629 11886 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:29.480883 11694 raft_consensus.cc:1275] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:29.482672 11991 consensus_queue.cc:1048] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:29.495499 11939 raft_consensus.cc:1275] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:29.498286 11991 consensus_queue.cc:1048] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:29.874879   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:29.888144 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:29.888949 11617 raft_consensus.cc:606] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:29.892948 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:29.893733 11617 raft_consensus.cc:606] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:29.895251   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:29.896183   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:29.897231   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:29.920046 11979 raft_consensus.cc:993] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:29.920935 11979 raft_consensus.cc:1081] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:29.923589 11694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:29.924156 11694 raft_consensus.cc:493] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:29.925042 11694 raft_consensus.cc:3060] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.932114 11694 raft_consensus.cc:515] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.934527 11694 leader_election.cc:290] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:29.935957 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:29.936014 11939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1860bb4663348148576caaf0f46fab6" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:29.937757 11939 raft_consensus.cc:3060] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.938189 11617 raft_consensus.cc:3055] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:29.938838 11617 raft_consensus.cc:740] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:29.940238 11617 consensus_queue.cc:260] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.942046 11617 raft_consensus.cc:3060] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:29.946297 11939 raft_consensus.cc:2468] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:29.948652 11660 leader_election.cc:304] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:29.951244 11981 raft_consensus.cc:2804] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:29.952778 11981 raft_consensus.cc:697] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:29.954149 11617 raft_consensus.cc:2468] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:29.954176 11981 consensus_queue.cc:237] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:29.967018 11442 catalog_manager.cc:5697] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 2, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:30.183555 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:30.382376 11617 raft_consensus.cc:1275] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:30.384049 11981 consensus_queue.cc:1048] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:30.396572 11939 raft_consensus.cc:1275] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:30.400038 11981 consensus_queue.cc:1048] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:30.412622 11979 raft_consensus.cc:993] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:30.413286 12021 raft_consensus.cc:1081] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:30.415642 11694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:30.416764 11694 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:30.417227 11694 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.425338 11694 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:30.427709 11694 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 4 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:30.428711 11940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:30.429039 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:30.429549 11940 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.429733 11617 raft_consensus.cc:3055] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:30.430202 11617 raft_consensus.cc:740] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:30.431047 11617 consensus_queue.cc:260] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:30.432278 11617 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:30.437624 11940 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 4.
I20260709 13:06:30.438683 11617 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 4.
I20260709 13:06:30.439298 11660 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:30.440268 12016 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:30.440838 12016 raft_consensus.cc:697] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:30.442332 12016 consensus_queue.cc:237] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:30.452850 11443 catalog_manager.cc:5697] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 3 to 4, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:30.877223 11940 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:30.879325 12016 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:30.895363 11617 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:30.904188 11996 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:30.906423   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:30.920432 11694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
mode: GRACEFUL
new_leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:53512
I20260709 13:06:30.921182 11694 raft_consensus.cc:606] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Received request to transfer leadership to TS 40e9f867c7af439bbfca87d7e3f03afa
I20260709 13:06:30.924721 11694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
mode: GRACEFUL
new_leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:53512
I20260709 13:06:30.925413 11694 raft_consensus.cc:606] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Received request to transfer leadership to TS 40e9f867c7af439bbfca87d7e3f03afa
I20260709 13:06:30.927299   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:30.928107   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:30.928910   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:31.156881 11996 raft_consensus.cc:993] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: : Instructing follower 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:31.157598 11996 raft_consensus.cc:1081] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Signalling peer 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:31.159567 11617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:40150
I20260709 13:06:31.160377 11617 raft_consensus.cc:493] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:31.160714 11617 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.166826 11617 raft_consensus.cc:515] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:31.169518 11617 leader_election.cc:290] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:31.170660 11940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:31.170912 11694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49cbc82778d042e8aaa7acde20f8b5b3" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:31.171507 11940 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.171646 11694 raft_consensus.cc:3055] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:31.172127 11694 raft_consensus.cc:740] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:31.173516 11694 consensus_queue.cc:260] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:31.174918 11694 raft_consensus.cc:3060] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:31.179037 11940 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 3.
I20260709 13:06:31.180753 11586 leader_election.cc:304] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:31.181474 12021 raft_consensus.cc:2804] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:31.181803 11694 raft_consensus.cc:2468] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 3.
I20260709 13:06:31.182570 12021 raft_consensus.cc:697] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:31.183491 12021 consensus_queue.cc:237] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
W20260709 13:06:31.185145 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:31.192147 11443 catalog_manager.cc:5697] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 2 to 3, leader changed from a16fca66e4404a01911ecd98feb63dcd (127.0.107.195) to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 3 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.509403 11996 raft_consensus.cc:993] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: : Instructing follower 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:31.510109 11981 raft_consensus.cc:1081] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Signalling peer 40e9f867c7af439bbfca87d7e3f03afa to start an election
I20260709 13:06:31.512163 11617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
 from {username='slave'} at 127.0.0.1:40150
I20260709 13:06:31.512877 11617 raft_consensus.cc:493] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:31.513484 11617 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:31.521363 11617 raft_consensus.cc:515] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:31.523850 11617 leader_election.cc:290] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 5 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), a16fca66e4404a01911ecd98feb63dcd (127.0.107.195:42925)
I20260709 13:06:31.525027 11939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:31.526147 11939 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:31.526499 11692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56afd91ff5be4d288bf586e5f0c222a9" candidate_uuid: "40e9f867c7af439bbfca87d7e3f03afa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
I20260709 13:06:31.527196 11692 raft_consensus.cc:3055] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:31.527534 11692 raft_consensus.cc:740] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:31.528431 11692 consensus_queue.cc:260] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:31.530167 11692 raft_consensus.cc:3060] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:31.535329 11939 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 5.
I20260709 13:06:31.538268 11586 leader_election.cc:304] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40e9f867c7af439bbfca87d7e3f03afa, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:31.538951 11692 raft_consensus.cc:2468] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40e9f867c7af439bbfca87d7e3f03afa in term 5.
I20260709 13:06:31.539562 12021 raft_consensus.cc:2804] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:31.540210 12021 raft_consensus.cc:697] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 5 LEADER]: Becoming Leader. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:31.540977 12021 consensus_queue.cc:237] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:31.550844 11443 catalog_manager.cc:5697] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa reported cstate change: term changed from 4 to 5, leader changed from a16fca66e4404a01911ecd98feb63dcd (127.0.107.195) to 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194). New cstate: current_term: 5 leader_uuid: "40e9f867c7af439bbfca87d7e3f03afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.565300 11692 raft_consensus.cc:1275] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:31.567148 12021 consensus_queue.cc:1048] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:31.580633 11939 raft_consensus.cc:1275] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:31.586234 12032 consensus_queue.cc:1048] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:31.938802   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:31.949905 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:31.950577 11617 raft_consensus.cc:606] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:31.954236 11617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9"
dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
mode: GRACEFUL
new_leader_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:06:31.955031 11617 raft_consensus.cc:606] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Received request to transfer leadership to TS a16fca66e4404a01911ecd98feb63dcd
I20260709 13:06:31.956740   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:31.957552   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:31.958223   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:32.012779 11692 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 5 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:32.014531 12043 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:32.021019 11939 raft_consensus.cc:1275] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 5 FOLLOWER]: Refusing update from remote peer 40e9f867c7af439bbfca87d7e3f03afa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:06:32.022945 12032 consensus_queue.cc:1048] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:06:32.186576 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:32.205020 12021 raft_consensus.cc:993] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:32.205708 12045 raft_consensus.cc:1081] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:32.207602 11692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:32.208249 11692 raft_consensus.cc:493] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:32.208910 11692 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.215921 11692 raft_consensus.cc:515] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:32.217780 11692 leader_election.cc:290] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 4 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:32.219110 11939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:32.219414 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64f3495644d5424798c65e2a9730d8bf" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:32.219754 11939 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.220242 11617 raft_consensus.cc:3055] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:32.220615 11617 raft_consensus.cc:740] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:32.221416 11617 consensus_queue.cc:260] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:32.222512 11617 raft_consensus.cc:3060] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:32.226720 11939 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 4.
I20260709 13:06:32.228307 11617 raft_consensus.cc:2468] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 4.
I20260709 13:06:32.228324 11660 leader_election.cc:304] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:32.229995 11981 raft_consensus.cc:2804] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:32.231112 11981 raft_consensus.cc:697] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:32.232125 11981 consensus_queue.cc:237] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:32.241902 11443 catalog_manager.cc:5697] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 3 to 4, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 4 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.278074 12021 raft_consensus.cc:993] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: : Instructing follower a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:32.278802 12021 raft_consensus.cc:1081] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Signalling peer a16fca66e4404a01911ecd98feb63dcd to start an election
I20260709 13:06:32.280797 11692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9"
dest_uuid: "a16fca66e4404a01911ecd98feb63dcd"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:06:32.281329 11692 raft_consensus.cc:493] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:32.281733 11692 raft_consensus.cc:3060] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.287352 11692 raft_consensus.cc:515] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:32.289278 11692 leader_election.cc:290] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Requested vote from peers a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895), 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175)
I20260709 13:06:32.290357 11939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6999ef0c57c494b9e4774bd01b6054d"
I20260709 13:06:32.290810 11617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f0cb9ea46d74fc9bdc2292669f97fa9" candidate_uuid: "a16fca66e4404a01911ecd98feb63dcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "40e9f867c7af439bbfca87d7e3f03afa"
I20260709 13:06:32.291416 11617 raft_consensus.cc:3055] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:32.291296 11939 raft_consensus.cc:3060] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.291879 11617 raft_consensus.cc:740] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 40e9f867c7af439bbfca87d7e3f03afa, State: Running, Role: LEADER
I20260709 13:06:32.292989 11617 consensus_queue.cc:260] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:32.294690 11617 raft_consensus.cc:3060] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.299839 11939 raft_consensus.cc:2468] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:32.301172 11617 raft_consensus.cc:2468] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a16fca66e4404a01911ecd98feb63dcd in term 2.
I20260709 13:06:32.301476 11660 leader_election.cc:304] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a16fca66e4404a01911ecd98feb63dcd, a6999ef0c57c494b9e4774bd01b6054d; no voters: 
I20260709 13:06:32.302359 11981 raft_consensus.cc:2804] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:32.302834 11981 raft_consensus.cc:697] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Becoming Leader. State: Replica: a16fca66e4404a01911ecd98feb63dcd, State: Running, Role: LEADER
I20260709 13:06:32.303601 11981 consensus_queue.cc:237] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } }
I20260709 13:06:32.312930 11443 catalog_manager.cc:5697] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd reported cstate change: term changed from 1 to 2, leader changed from 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194) to a16fca66e4404a01911ecd98feb63dcd (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a16fca66e4404a01911ecd98feb63dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16fca66e4404a01911ecd98feb63dcd" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 42925 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.709043 11939 raft_consensus.cc:1275] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:32.711028 12047 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:32.725698 11617 raft_consensus.cc:1275] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:32.727795 11981 consensus_queue.cc:1048] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:32.757594 11939 raft_consensus.cc:1275] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.759137 12047 consensus_queue.cc:1048] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6999ef0c57c494b9e4774bd01b6054d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:32.771235 11617 raft_consensus.cc:1275] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Refusing update from remote peer a16fca66e4404a01911ecd98feb63dcd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:32.773936 12056 consensus_queue.cc:1048] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e9f867c7af439bbfca87d7e3f03afa" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 42175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:32.966064   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:33.008227   431 tablet_server.cc:179] TabletServer@127.0.107.193:39895 shutting down...
I20260709 13:06:33.046200   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:33.047336   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.048283   431 raft_consensus.cc:2243] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.048916   431 raft_consensus.cc:2272] T 49cbc82778d042e8aaa7acde20f8b5b3 P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.051075   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.051932   431 raft_consensus.cc:2243] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.053128   431 raft_consensus.cc:2272] T 8a60b2e1c3304bafaa8826f684b0c3de P a6999ef0c57c494b9e4774bd01b6054d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.055933   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.056658   431 raft_consensus.cc:2243] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.057130   431 raft_consensus.cc:2272] T f1860bb4663348148576caaf0f46fab6 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.059042   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.059705   431 raft_consensus.cc:2243] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.060511   431 raft_consensus.cc:2272] T cee0ad237fd241ad8c8554a0a05a50d8 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.062851   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.063500   431 raft_consensus.cc:2243] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.063990   431 raft_consensus.cc:2272] T e1937785aba94020a04aa593cb885143 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.066557   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.067319   431 raft_consensus.cc:2243] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.068058   431 raft_consensus.cc:2272] T 64f3495644d5424798c65e2a9730d8bf P a6999ef0c57c494b9e4774bd01b6054d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.073302   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.074520   431 raft_consensus.cc:2243] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.075362   431 raft_consensus.cc:2272] T 7e242cfc1aa04b228548b73077358cef P a6999ef0c57c494b9e4774bd01b6054d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.078794   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.079648   431 raft_consensus.cc:2243] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.080412   431 raft_consensus.cc:2272] T 56afd91ff5be4d288bf586e5f0c222a9 P a6999ef0c57c494b9e4774bd01b6054d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.083793   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.084676   431 raft_consensus.cc:2243] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.085337   431 raft_consensus.cc:2272] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.087901   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:33.088827   431 raft_consensus.cc:2243] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.089346   431 raft_consensus.cc:2272] T e9e8b00eb8004f938def8be94fd87474 P a6999ef0c57c494b9e4774bd01b6054d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:33.092689 11660 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 93 similar messages]
W20260709 13:06:33.096273 11660 consensus_peers.cc:597] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:33.127144   431 tablet_server.cc:196] TabletServer@127.0.107.193:39895 shutdown complete.
I20260709 13:06:33.154816   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:33.187994 11491 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:33.191892   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:33.193650   431 tablet_replica.cc:333] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.195050   431 raft_consensus.cc:2243] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.196182   431 raft_consensus.cc:2272] T 49cbc82778d042e8aaa7acde20f8b5b3 P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:33.197012 11660 consensus_peers.cc:597] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:33.197876 11660 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:33.198441 11660 consensus_peers.cc:597] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:33.200064   431 tablet_replica.cc:333] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.201020   431 raft_consensus.cc:2243] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.201908   431 raft_consensus.cc:2272] T 8a60b2e1c3304bafaa8826f684b0c3de P 40e9f867c7af439bbfca87d7e3f03afa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.205448   431 tablet_replica.cc:333] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.206266   431 raft_consensus.cc:2243] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.207178   431 raft_consensus.cc:2272] T cee0ad237fd241ad8c8554a0a05a50d8 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.211668   431 tablet_replica.cc:333] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.212463   431 raft_consensus.cc:2243] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.212898   431 raft_consensus.cc:2272] T f1860bb4663348148576caaf0f46fab6 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.215538   431 tablet_replica.cc:333] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.216317   431 raft_consensus.cc:2243] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.217165   431 raft_consensus.cc:2272] T 64f3495644d5424798c65e2a9730d8bf P 40e9f867c7af439bbfca87d7e3f03afa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.220993   431 tablet_replica.cc:333] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.222162   431 raft_consensus.cc:2243] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.222810   431 raft_consensus.cc:2272] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.226907   431 tablet_replica.cc:333] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.227660   431 raft_consensus.cc:2243] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.228613   431 raft_consensus.cc:2272] T 56afd91ff5be4d288bf586e5f0c222a9 P 40e9f867c7af439bbfca87d7e3f03afa [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.231122   431 tablet_replica.cc:333] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.231755   431 raft_consensus.cc:2243] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.232455   431 raft_consensus.cc:2272] T 7e242cfc1aa04b228548b73077358cef P 40e9f867c7af439bbfca87d7e3f03afa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.235347   431 tablet_replica.cc:333] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.235996   431 raft_consensus.cc:2243] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.236955   431 raft_consensus.cc:2272] T e9e8b00eb8004f938def8be94fd87474 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.239780   431 tablet_replica.cc:333] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa: stopping tablet replica
I20260709 13:06:33.240461   431 raft_consensus.cc:2243] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.240974   431 raft_consensus.cc:2272] T e1937785aba94020a04aa593cb885143 P 40e9f867c7af439bbfca87d7e3f03afa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:33.252457 11660 consensus_peers.cc:597] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd -> Peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Couldn't send request to peer 40e9f867c7af439bbfca87d7e3f03afa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:33.255915 11660 consensus_peers.cc:597] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:33.261665 11660 consensus_peers.cc:597] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd -> Peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Couldn't send request to peer 40e9f867c7af439bbfca87d7e3f03afa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:33.288272   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:06:33.322551   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:33.332921 11660 consensus_peers.cc:597] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd -> Peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Couldn't send request to peer 40e9f867c7af439bbfca87d7e3f03afa. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:42175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:33.358791 11660 consensus_peers.cc:597] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd -> Peer a6999ef0c57c494b9e4774bd01b6054d (127.0.107.193:39895): Couldn't send request to peer a6999ef0c57c494b9e4774bd01b6054d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:39895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:33.374150   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:33.375168   431 tablet_replica.cc:333] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.376004   431 raft_consensus.cc:2243] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.376634   431 raft_consensus.cc:2272] T cee0ad237fd241ad8c8554a0a05a50d8 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.379848   431 tablet_replica.cc:333] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.380725   431 raft_consensus.cc:2243] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.381243   431 raft_consensus.cc:2272] T 56afd91ff5be4d288bf586e5f0c222a9 P a16fca66e4404a01911ecd98feb63dcd [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.384604   431 tablet_replica.cc:333] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.385636   431 raft_consensus.cc:2243] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.387069   431 raft_consensus.cc:2272] T 7e242cfc1aa04b228548b73077358cef P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:33.388069 11660 consensus_peers.cc:597] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd -> Peer 40e9f867c7af439bbfca87d7e3f03afa (127.0.107.194:42175): Couldn't send request to peer 40e9f867c7af439bbfca87d7e3f03afa. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:42175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:33.390419   431 tablet_replica.cc:333] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.391258   431 raft_consensus.cc:2243] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.392429   431 raft_consensus.cc:2272] T 9f0cb9ea46d74fc9bdc2292669f97fa9 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.395974   431 tablet_replica.cc:333] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.396735   431 raft_consensus.cc:2243] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.397706   431 raft_consensus.cc:2272] T 64f3495644d5424798c65e2a9730d8bf P a16fca66e4404a01911ecd98feb63dcd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.401347   431 tablet_replica.cc:333] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.402056   431 raft_consensus.cc:2243] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.402994   431 raft_consensus.cc:2272] T e1937785aba94020a04aa593cb885143 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.405771   431 tablet_replica.cc:333] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.406667   431 raft_consensus.cc:2243] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.407637   431 raft_consensus.cc:2272] T e9e8b00eb8004f938def8be94fd87474 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.410588   431 tablet_replica.cc:333] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.411361   431 raft_consensus.cc:2243] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.411991   431 raft_consensus.cc:2272] T 8a60b2e1c3304bafaa8826f684b0c3de P a16fca66e4404a01911ecd98feb63dcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.414826   431 tablet_replica.cc:333] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.415540   431 raft_consensus.cc:2243] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:33.416172   431 raft_consensus.cc:2272] T 49cbc82778d042e8aaa7acde20f8b5b3 P a16fca66e4404a01911ecd98feb63dcd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.418763   431 tablet_replica.cc:333] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd: stopping tablet replica
I20260709 13:06:33.419416   431 raft_consensus.cc:2243] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.420477   431 raft_consensus.cc:2272] T f1860bb4663348148576caaf0f46fab6 P a16fca66e4404a01911ecd98feb63dcd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.462422   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:33.498121   431 master.cc:562] Master@127.0.107.254:43437 shutting down...
I20260709 13:06:33.536998   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:33.537770   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:33.538182   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 265081b9ebd545d0bea4d73560310287: stopping tablet replica
I20260709 13:06:33.563510   431 master.cc:584] Master@127.0.107.254:43437 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (20700 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:06:33.626632   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:45195
I20260709 13:06:33.628129   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:33.639119 12069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:33.639367 12070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:33.639894 12072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:33.640516   431 server_base.cc:1061] running on GCE node
I20260709 13:06:33.641706   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:33.641906   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:33.642030   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602393642018 us; error 0 us; skew 500 ppm
I20260709 13:06:33.642578   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:33.645864   431 webserver.cc:533] Webserver started at http://127.0.107.254:43283/ using document root <none> and password file <none>
I20260709 13:06:33.646667   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:33.646937   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:33.647205   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:33.648711   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "18e0b5f4251c4819986f05346e1334c0"
format_stamp: "Formatted at 2026-07-09 13:06:33 on dist-test-slave-ftjc"
I20260709 13:06:33.653553   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:33.658383 12077 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:33.659762   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:33.660290   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "18e0b5f4251c4819986f05346e1334c0"
format_stamp: "Formatted at 2026-07-09 13:06:33 on dist-test-slave-ftjc"
I20260709 13:06:33.661257   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:33.680004   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:33.681793   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:33.737807   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:45195
I20260709 13:06:33.737974 12128 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:45195 every 8 connection(s)
I20260709 13:06:33.742523 12129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:33.760609 12129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0: Bootstrap starting.
I20260709 13:06:33.769241 12129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:33.776108 12129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0: No bootstrap required, opened a new log
I20260709 13:06:33.779296 12129 raft_consensus.cc:359] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e0b5f4251c4819986f05346e1334c0" member_type: VOTER }
I20260709 13:06:33.779978 12129 raft_consensus.cc:385] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:33.780267 12129 raft_consensus.cc:740] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e0b5f4251c4819986f05346e1334c0, State: Initialized, Role: FOLLOWER
I20260709 13:06:33.781092 12129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e0b5f4251c4819986f05346e1334c0" member_type: VOTER }
I20260709 13:06:33.781581 12129 raft_consensus.cc:399] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:33.781873 12129 raft_consensus.cc:493] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:33.782187 12129 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.789036 12129 raft_consensus.cc:515] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e0b5f4251c4819986f05346e1334c0" member_type: VOTER }
I20260709 13:06:33.789803 12129 leader_election.cc:304] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18e0b5f4251c4819986f05346e1334c0; no voters: 
I20260709 13:06:33.791365 12129 leader_election.cc:290] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:33.792045 12132 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:33.793928 12132 raft_consensus.cc:697] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 1 LEADER]: Becoming Leader. State: Replica: 18e0b5f4251c4819986f05346e1334c0, State: Running, Role: LEADER
I20260709 13:06:33.794826 12132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e0b5f4251c4819986f05346e1334c0" member_type: VOTER }
I20260709 13:06:33.795989 12129 sys_catalog.cc:565] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:33.803210 12133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18e0b5f4251c4819986f05346e1334c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e0b5f4251c4819986f05346e1334c0" member_type: VOTER } }
I20260709 13:06:33.804092 12134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18e0b5f4251c4819986f05346e1334c0. Latest consensus state: current_term: 1 leader_uuid: "18e0b5f4251c4819986f05346e1334c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e0b5f4251c4819986f05346e1334c0" member_type: VOTER } }
I20260709 13:06:33.805501 12133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:33.806245 12134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:33.810386 12141 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:33.818825 12141 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:33.820070   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:33.832855 12141 catalog_manager.cc:1383] Generated new cluster ID: c4c3e23b911b437b9bd224f3e115fddc
I20260709 13:06:33.833313 12141 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:33.866410 12141 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:33.868090 12141 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:33.882587 12141 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0: Generated new TSK 0
I20260709 13:06:33.883391 12141 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:33.894287   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:33.903862 12150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:33.905306 12151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:33.907886 12153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:33.908109   431 server_base.cc:1061] running on GCE node
I20260709 13:06:33.910058   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:33.910416   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:33.910885   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602393910851 us; error 0 us; skew 500 ppm
I20260709 13:06:33.911767   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:33.915284   431 webserver.cc:533] Webserver started at http://127.0.107.193:42623/ using document root <none> and password file <none>
I20260709 13:06:33.916095   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:33.916694   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:33.917114   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:33.918540   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
format_stamp: "Formatted at 2026-07-09 13:06:33 on dist-test-slave-ftjc"
I20260709 13:06:33.925886   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:06:33.931269 12158 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:33.932404   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:33.932696   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
format_stamp: "Formatted at 2026-07-09 13:06:33 on dist-test-slave-ftjc"
I20260709 13:06:33.932955   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:33.947858   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:33.949905   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:33.951979   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:33.955674   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:33.955961   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:33.956177   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:33.956327   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:34.016273   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42405
I20260709 13:06:34.016438 12220 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42405 every 8 connection(s)
I20260709 13:06:34.023303   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:34.033635 12225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:34.034871 12226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:34.040261 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:34.041921   431 server_base.cc:1061] running on GCE node
I20260709 13:06:34.043604   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:34.043910   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:34.044134   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602394044110 us; error 0 us; skew 500 ppm
I20260709 13:06:34.044312 12221 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45195
I20260709 13:06:34.044888 12221 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:34.045059   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:34.046025 12221 heartbeater.cc:507] Master 127.0.107.254:45195 requested a full tablet report, sending...
I20260709 13:06:34.049309   431 webserver.cc:533] Webserver started at http://127.0.107.194:44235/ using document root <none> and password file <none>
I20260709 13:06:34.049780 12094 ts_manager.cc:194] Registered new tserver with Master: ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405)
I20260709 13:06:34.050354   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:34.050660   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:34.050983   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:34.052119   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "348081022fbf46babc8493994d4ff6b7"
format_stamp: "Formatted at 2026-07-09 13:06:34 on dist-test-slave-ftjc"
I20260709 13:06:34.052227 12094 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58118
I20260709 13:06:34.059204   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:34.064316 12233 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:34.065443   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:34.065958   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "348081022fbf46babc8493994d4ff6b7"
format_stamp: "Formatted at 2026-07-09 13:06:34 on dist-test-slave-ftjc"
I20260709 13:06:34.066303   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:34.083393   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:34.085148   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:34.087316   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:34.091364   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:34.091938   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:06:34.092269   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:34.092581   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.002s	sys 0.000s
I20260709 13:06:34.152853   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40171
I20260709 13:06:34.153697 12295 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40171 every 8 connection(s)
I20260709 13:06:34.159821   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:34.170728 12299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:34.173328 12300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:34.177826   431 server_base.cc:1061] running on GCE node
W20260709 13:06:34.178020 12302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:34.178543 12296 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45195
I20260709 13:06:34.179091 12296 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:34.179683   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:06:34.180034 12296 heartbeater.cc:507] Master 127.0.107.254:45195 requested a full tablet report, sending...
W20260709 13:06:34.180174   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:34.180663   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602394180642 us; error 0 us; skew 500 ppm
I20260709 13:06:34.181411   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:34.183275 12094 ts_manager.cc:194] Registered new tserver with Master: 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:34.185067   431 webserver.cc:533] Webserver started at http://127.0.107.195:35211/ using document root <none> and password file <none>
I20260709 13:06:34.185422 12094 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58120
I20260709 13:06:34.185993   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:34.186316   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:34.186666   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:34.189333   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "44d28a6a8f0f49de902578a825e503ad"
format_stamp: "Formatted at 2026-07-09 13:06:34 on dist-test-slave-ftjc"
I20260709 13:06:34.196861   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:06:34.201928 12307 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:34.203394   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:34.204069   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "44d28a6a8f0f49de902578a825e503ad"
format_stamp: "Formatted at 2026-07-09 13:06:34 on dist-test-slave-ftjc"
I20260709 13:06:34.204463   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:34.226795   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:34.228313   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:34.230561   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:34.233870   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:34.234119   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:34.234467   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:34.234660   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:34.302479   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:37391
I20260709 13:06:34.302599 12369 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:37391 every 8 connection(s)
I20260709 13:06:34.322806 12370 heartbeater.cc:344] Connected to a master server at 127.0.107.254:45195
I20260709 13:06:34.323342 12370 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:34.324923 12370 heartbeater.cc:507] Master 127.0.107.254:45195 requested a full tablet report, sending...
I20260709 13:06:34.329792 12094 ts_manager.cc:194] Registered new tserver with Master: 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391)
I20260709 13:06:34.329949   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023896679s
I20260709 13:06:34.331571 12094 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58126
I20260709 13:06:34.372623 12094 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58142:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:34.374927 12094 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:34.451018 12335 tablet_service.cc:1511] Processing CreateTablet for tablet e22a7bf9822240eca5e4f60103be9d8a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=813e65e04f3340a6b16a113946906258]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:34.452903 12335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22a7bf9822240eca5e4f60103be9d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.452708 12334 tablet_service.cc:1511] Processing CreateTablet for tablet fe367654cac84faeb4006bfb4af27afd (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=813e65e04f3340a6b16a113946906258]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:34.454406 12334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe367654cac84faeb4006bfb4af27afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.460165 12261 tablet_service.cc:1511] Processing CreateTablet for tablet e22a7bf9822240eca5e4f60103be9d8a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=813e65e04f3340a6b16a113946906258]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:34.461114 12186 tablet_service.cc:1511] Processing CreateTablet for tablet e22a7bf9822240eca5e4f60103be9d8a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=813e65e04f3340a6b16a113946906258]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:34.462019 12261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22a7bf9822240eca5e4f60103be9d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.461890 12185 tablet_service.cc:1511] Processing CreateTablet for tablet fe367654cac84faeb4006bfb4af27afd (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=813e65e04f3340a6b16a113946906258]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:34.462707 12186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22a7bf9822240eca5e4f60103be9d8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.460165 12260 tablet_service.cc:1511] Processing CreateTablet for tablet fe367654cac84faeb4006bfb4af27afd (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=813e65e04f3340a6b16a113946906258]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:34.463606 12185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe367654cac84faeb4006bfb4af27afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.464727 12260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe367654cac84faeb4006bfb4af27afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.498216 12388 tablet_bootstrap.cc:492] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e: Bootstrap starting.
I20260709 13:06:34.509654 12389 tablet_bootstrap.cc:492] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: Bootstrap starting.
I20260709 13:06:34.513345 12388 tablet_bootstrap.cc:654] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.513779 12390 tablet_bootstrap.cc:492] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7: Bootstrap starting.
I20260709 13:06:34.521714 12389 tablet_bootstrap.cc:654] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.525540 12390 tablet_bootstrap.cc:654] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.530408 12388 tablet_bootstrap.cc:492] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e: No bootstrap required, opened a new log
I20260709 13:06:34.530589 12389 tablet_bootstrap.cc:492] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: No bootstrap required, opened a new log
I20260709 13:06:34.531723 12389 ts_tablet_manager.cc:1403] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 13:06:34.531735 12388 ts_tablet_manager.cc:1403] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.005s
I20260709 13:06:34.533717 12390 tablet_bootstrap.cc:492] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7: No bootstrap required, opened a new log
I20260709 13:06:34.534350 12390 ts_tablet_manager.cc:1403] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:06:34.536134 12388 raft_consensus.cc:359] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.536415 12389 raft_consensus.cc:359] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.537479 12388 raft_consensus.cc:385] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.537606 12389 raft_consensus.cc:385] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.537873 12388 raft_consensus.cc:740] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.538416 12389 raft_consensus.cc:740] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.537707 12390 raft_consensus.cc:359] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.539019 12390 raft_consensus.cc:385] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.539045 12388 consensus_queue.cc:260] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.539445 12390 raft_consensus.cc:740] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348081022fbf46babc8493994d4ff6b7, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.539449 12389 consensus_queue.cc:260] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.540592 12390 consensus_queue.cc:260] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.542925 12221 heartbeater.cc:499] Master 127.0.107.254:45195 was elected leader, sending a full tablet report...
I20260709 13:06:34.551556 12388 ts_tablet_manager.cc:1434] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.008s
I20260709 13:06:34.552935 12370 heartbeater.cc:499] Master 127.0.107.254:45195 was elected leader, sending a full tablet report...
I20260709 13:06:34.553179 12388 tablet_bootstrap.cc:492] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e: Bootstrap starting.
I20260709 13:06:34.554541 12389 ts_tablet_manager.cc:1434] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.012s
I20260709 13:06:34.556047 12389 tablet_bootstrap.cc:492] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad: Bootstrap starting.
I20260709 13:06:34.569867 12390 ts_tablet_manager.cc:1434] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7: Time spent starting tablet: real 0.035s	user 0.013s	sys 0.019s
I20260709 13:06:34.571776 12296 heartbeater.cc:499] Master 127.0.107.254:45195 was elected leader, sending a full tablet report...
I20260709 13:06:34.572080 12390 tablet_bootstrap.cc:492] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7: Bootstrap starting.
I20260709 13:06:34.574945 12389 tablet_bootstrap.cc:654] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.577555 12388 tablet_bootstrap.cc:654] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.580992 12390 tablet_bootstrap.cc:654] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.587981 12388 tablet_bootstrap.cc:492] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e: No bootstrap required, opened a new log
I20260709 13:06:34.588065 12389 tablet_bootstrap.cc:492] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad: No bootstrap required, opened a new log
I20260709 13:06:34.588900 12388 ts_tablet_manager.cc:1403] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.003s
I20260709 13:06:34.588932 12389 ts_tablet_manager.cc:1403] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.010s
I20260709 13:06:34.590365 12390 tablet_bootstrap.cc:492] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7: No bootstrap required, opened a new log
I20260709 13:06:34.591336 12390 ts_tablet_manager.cc:1403] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.007s
I20260709 13:06:34.593609 12389 raft_consensus.cc:359] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.595028 12389 raft_consensus.cc:385] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.595352 12389 raft_consensus.cc:740] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.595209 12390 raft_consensus.cc:359] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.596639 12390 raft_consensus.cc:385] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.596194 12388 raft_consensus.cc:359] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.597313 12390 raft_consensus.cc:740] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348081022fbf46babc8493994d4ff6b7, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.597486 12388 raft_consensus.cc:385] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.596873 12389 consensus_queue.cc:260] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.597944 12388 raft_consensus.cc:740] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.598544 12390 consensus_queue.cc:260] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.599062 12388 consensus_queue.cc:260] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.600422 12389 ts_tablet_manager.cc:1434] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:06:34.601210 12390 ts_tablet_manager.cc:1434] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 13:06:34.602959 12388 ts_tablet_manager.cc:1434] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:06:34.605921 12394 raft_consensus.cc:493] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.606771 12394 raft_consensus.cc:515] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.611181 12394 leader_election.cc:290] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:34.635205 12345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22a7bf9822240eca5e4f60103be9d8a" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d28a6a8f0f49de902578a825e503ad" is_pre_election: true
I20260709 13:06:34.636998 12345 raft_consensus.cc:2468] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 0.
I20260709 13:06:34.637877 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22a7bf9822240eca5e4f60103be9d8a" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7" is_pre_election: true
I20260709 13:06:34.638648 12162 leader_election.cc:304] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:34.638880 12271 raft_consensus.cc:2468] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 0.
I20260709 13:06:34.639815 12394 raft_consensus.cc:2804] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:34.640327 12394 raft_consensus.cc:493] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.640739 12394 raft_consensus.cc:3060] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:34.650765 12394 raft_consensus.cc:515] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.652747 12394 leader_election.cc:290] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 election: Requested vote from peers 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:34.653642 12345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22a7bf9822240eca5e4f60103be9d8a" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
I20260709 13:06:34.654208 12345 raft_consensus.cc:3060] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:34.654273 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22a7bf9822240eca5e4f60103be9d8a" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7"
I20260709 13:06:34.655182 12271 raft_consensus.cc:3060] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:34.662840 12271 raft_consensus.cc:2468] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 1.
I20260709 13:06:34.664077 12345 raft_consensus.cc:2468] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 1.
I20260709 13:06:34.664538 12161 leader_election.cc:304] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348081022fbf46babc8493994d4ff6b7, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:34.665606 12394 raft_consensus.cc:2804] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:34.667471 12394 raft_consensus.cc:697] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 LEADER]: Becoming Leader. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Running, Role: LEADER
I20260709 13:06:34.668731 12394 consensus_queue.cc:237] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } }
I20260709 13:06:34.679958 12093 catalog_manager.cc:5697] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e reported cstate change: term changed from 0 to 1, leader changed from <none> to ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.689409 12395 raft_consensus.cc:493] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.690061 12395 raft_consensus.cc:515] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.694224 12395 leader_election.cc:290] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:34.708122 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" is_pre_election: true
I20260709 13:06:34.709007 12196 raft_consensus.cc:2468] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 0.
I20260709 13:06:34.710353 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7" is_pre_election: true
I20260709 13:06:34.710546 12308 leader_election.cc:304] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:34.711956 12271 raft_consensus.cc:2468] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 0.
I20260709 13:06:34.712601 12395 raft_consensus.cc:2804] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:34.713096 12395 raft_consensus.cc:493] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.713857 12395 raft_consensus.cc:3060] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:34.721803 12395 raft_consensus.cc:515] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.724880 12395 leader_election.cc:290] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 election: Requested vote from peers ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:34.726008 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
I20260709 13:06:34.726204 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7"
I20260709 13:06:34.726890 12196 raft_consensus.cc:3060] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:34.727420 12271 raft_consensus.cc:3060] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:34.735008 12196 raft_consensus.cc:2468] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 1.
I20260709 13:06:34.736145 12271 raft_consensus.cc:2468] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 1.
I20260709 13:06:34.736965 12308 leader_election.cc:304] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:34.738118 12395 raft_consensus.cc:2804] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:34.739313 12395 raft_consensus.cc:697] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Becoming Leader. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Running, Role: LEADER
I20260709 13:06:34.740285 12395 consensus_queue.cc:237] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.750205 12093 catalog_manager.cc:5697] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 44d28a6a8f0f49de902578a825e503ad (127.0.107.195). New cstate: current_term: 1 leader_uuid: "44d28a6a8f0f49de902578a825e503ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.871155 12093 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58156:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:34.874269 12093 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:34.910682 12185 tablet_service.cc:1511] Processing CreateTablet for tablet 99706f5a1f9b48c194f948644197014e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a47e11af0d84907bfbeaeb041ba811c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:34.912501 12185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99706f5a1f9b48c194f948644197014e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.912642 12260 tablet_service.cc:1511] Processing CreateTablet for tablet 99706f5a1f9b48c194f948644197014e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a47e11af0d84907bfbeaeb041ba811c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:34.914769 12260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99706f5a1f9b48c194f948644197014e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.916209 12334 tablet_service.cc:1511] Processing CreateTablet for tablet 99706f5a1f9b48c194f948644197014e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a47e11af0d84907bfbeaeb041ba811c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:34.918155 12334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99706f5a1f9b48c194f948644197014e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.918232 12186 tablet_service.cc:1511] Processing CreateTablet for tablet 29eea3e418864f638583682f76b83843 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a47e11af0d84907bfbeaeb041ba811c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:34.919258 12261 tablet_service.cc:1511] Processing CreateTablet for tablet 29eea3e418864f638583682f76b83843 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a47e11af0d84907bfbeaeb041ba811c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:34.919839 12186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29eea3e418864f638583682f76b83843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.920413 12261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29eea3e418864f638583682f76b83843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.935402 12335 tablet_service.cc:1511] Processing CreateTablet for tablet 29eea3e418864f638583682f76b83843 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a47e11af0d84907bfbeaeb041ba811c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:34.937295 12335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29eea3e418864f638583682f76b83843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.944006 12390 tablet_bootstrap.cc:492] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7: Bootstrap starting.
I20260709 13:06:34.945268 12388 tablet_bootstrap.cc:492] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e: Bootstrap starting.
I20260709 13:06:34.954257 12390 tablet_bootstrap.cc:654] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.954478 12388 tablet_bootstrap.cc:654] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.963917 12389 tablet_bootstrap.cc:492] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Bootstrap starting.
I20260709 13:06:34.968209 12390 tablet_bootstrap.cc:492] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7: No bootstrap required, opened a new log
I20260709 13:06:34.969112 12390 ts_tablet_manager.cc:1403] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.011s
I20260709 13:06:34.973115 12390 raft_consensus.cc:359] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.974100 12390 raft_consensus.cc:385] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.974360 12388 tablet_bootstrap.cc:492] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e: No bootstrap required, opened a new log
I20260709 13:06:34.974417 12390 raft_consensus.cc:740] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348081022fbf46babc8493994d4ff6b7, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.975623 12388 ts_tablet_manager.cc:1403] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.004s
I20260709 13:06:34.976545 12389 tablet_bootstrap.cc:654] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.976722 12390 consensus_queue.cc:260] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.979911 12388 raft_consensus.cc:359] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.980926 12388 raft_consensus.cc:385] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:34.981259 12388 raft_consensus.cc:740] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Initialized, Role: FOLLOWER
I20260709 13:06:34.982844 12390 ts_tablet_manager.cc:1434] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:06:34.982545 12388 consensus_queue.cc:260] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:34.984541 12390 tablet_bootstrap.cc:492] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7: Bootstrap starting.
I20260709 13:06:34.985309 12388 ts_tablet_manager.cc:1434] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.003s
I20260709 13:06:34.986989 12388 tablet_bootstrap.cc:492] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: Bootstrap starting.
I20260709 13:06:34.994881 12388 tablet_bootstrap.cc:654] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:34.994913 12389 tablet_bootstrap.cc:492] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: No bootstrap required, opened a new log
I20260709 13:06:34.995846 12389 ts_tablet_manager.cc:1403] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.016s
I20260709 13:06:35.000447 12390 tablet_bootstrap.cc:654] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.002174 12389 raft_consensus.cc:359] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.003559 12389 raft_consensus.cc:385] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.004061 12389 raft_consensus.cc:740] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.005537 12389 consensus_queue.cc:260] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.010242 12388 tablet_bootstrap.cc:492] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: No bootstrap required, opened a new log
I20260709 13:06:35.010905 12388 ts_tablet_manager.cc:1403] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.008s
I20260709 13:06:35.015314 12388 raft_consensus.cc:359] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.016290 12388 raft_consensus.cc:385] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.016685 12388 raft_consensus.cc:740] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.017681 12388 consensus_queue.cc:260] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.019281 12389 ts_tablet_manager.cc:1434] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 13:06:35.021167 12388 ts_tablet_manager.cc:1434] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.010s
I20260709 13:06:35.023660 12389 tablet_bootstrap.cc:492] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad: Bootstrap starting.
I20260709 13:06:35.029340 12395 raft_consensus.cc:493] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.030331 12395 raft_consensus.cc:515] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.034700 12390 tablet_bootstrap.cc:492] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7: No bootstrap required, opened a new log
I20260709 13:06:35.035665 12390 ts_tablet_manager.cc:1403] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.020s
I20260709 13:06:35.036208 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" is_pre_election: true
I20260709 13:06:35.037129 12196 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 0.
I20260709 13:06:35.037227 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7" is_pre_election: true
I20260709 13:06:35.038911 12389 tablet_bootstrap.cc:654] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:35.039091 12308 leader_election.cc:304] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:35.039736 12390 raft_consensus.cc:359] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.040745 12390 raft_consensus.cc:385] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.040892 12405 raft_consensus.cc:2804] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.041129 12390 raft_consensus.cc:740] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348081022fbf46babc8493994d4ff6b7, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.041899 12405 raft_consensus.cc:493] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.042333 12405 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.042801 12390 consensus_queue.cc:260] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
W20260709 13:06:35.043720 12310 leader_election.cc:343] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:35.048918 12395 leader_election.cc:290] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:35.053622 12390 ts_tablet_manager.cc:1434] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.003s
I20260709 13:06:35.062413 12405 raft_consensus.cc:515] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.063997 12389 tablet_bootstrap.cc:492] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad: No bootstrap required, opened a new log
I20260709 13:06:35.064692 12389 ts_tablet_manager.cc:1403] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad: Time spent bootstrapping tablet: real 0.041s	user 0.031s	sys 0.000s
I20260709 13:06:35.066123 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
I20260709 13:06:35.066977 12196 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.068290 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7"
I20260709 13:06:35.068810 12405 leader_election.cc:290] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 election: Requested vote from peers ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:35.069237 12271 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.070472 12389 raft_consensus.cc:359] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.071369 12389 raft_consensus.cc:385] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:35.071439 12394 consensus_queue.cc:1048] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Connected to new peer: Peer: permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.073256 12402 raft_consensus.cc:493] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.073896 12402 raft_consensus.cc:515] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.071815 12389 raft_consensus.cc:740] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Initialized, Role: FOLLOWER
I20260709 13:06:35.076030 12389 consensus_queue.cc:260] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.077737 12196 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 1.
I20260709 13:06:35.079367 12389 ts_tablet_manager.cc:1434] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:35.079396 12308 leader_election.cc:304] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:35.080940 12405 raft_consensus.cc:2804] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.081185 12345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29eea3e418864f638583682f76b83843" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d28a6a8f0f49de902578a825e503ad" is_pre_election: true
I20260709 13:06:35.081470 12405 raft_consensus.cc:697] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Becoming Leader. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Running, Role: LEADER
I20260709 13:06:35.082085 12345 raft_consensus.cc:2468] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 0.
I20260709 13:06:35.082593 12405 consensus_queue.cc:237] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.084942 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29eea3e418864f638583682f76b83843" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7" is_pre_election: true
I20260709 13:06:35.085868 12269 raft_consensus.cc:2468] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 0.
I20260709 13:06:35.084206 12162 leader_election.cc:304] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:35.087307 12394 raft_consensus.cc:2804] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:35.087975 12394 raft_consensus.cc:493] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.088660 12394 raft_consensus.cc:3060] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.088771 12402 leader_election.cc:290] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171), 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391)
I20260709 13:06:35.102027 12394 raft_consensus.cc:515] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.106364 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29eea3e418864f638583682f76b83843" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348081022fbf46babc8493994d4ff6b7"
I20260709 13:06:35.106576 12394 leader_election.cc:290] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 election: Requested vote from peers 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171), 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391)
I20260709 13:06:35.107177 12269 raft_consensus.cc:3060] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.106381 12345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29eea3e418864f638583682f76b83843" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
I20260709 13:06:35.109330 12345 raft_consensus.cc:3060] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:35.113319 12271 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 1.
I20260709 13:06:35.121711 12345 raft_consensus.cc:2468] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 1.
I20260709 13:06:35.123453 12162 leader_election.cc:304] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d28a6a8f0f49de902578a825e503ad, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:35.124665 12394 raft_consensus.cc:2804] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:35.125394 12394 raft_consensus.cc:697] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 LEADER]: Becoming Leader. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Running, Role: LEADER
I20260709 13:06:35.126696 12394 consensus_queue.cc:237] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.164158 12395 consensus_queue.cc:1048] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.167078 12269 raft_consensus.cc:2468] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 1.
W20260709 13:06:35.191128 12162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37391, user_credentials={real_user=slave}} blocked reactor thread for 67802us
I20260709 13:06:35.193944 12091 catalog_manager.cc:5697] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 44d28a6a8f0f49de902578a825e503ad (127.0.107.195). New cstate: current_term: 1 leader_uuid: "44d28a6a8f0f49de902578a825e503ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:35.196873 12161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40171, user_credentials={real_user=slave}} blocked reactor thread for 44078us
I20260709 13:06:35.197719 12423 consensus_queue.cc:1048] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.222113 12093 catalog_manager.cc:5697] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e reported cstate change: term changed from 0 to 1, leader changed from <none> to ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.226255 12429 consensus_queue.cc:1048] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.289413 12345 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd"
dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
mode: GRACEFUL
new_leader_uuid: "348081022fbf46babc8493994d4ff6b7"
 from {username='slave'} at 127.0.0.1:57408
I20260709 13:06:35.290407 12345 raft_consensus.cc:606] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Received request to transfer leadership to TS 348081022fbf46babc8493994d4ff6b7
I20260709 13:06:35.299918 12345 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99706f5a1f9b48c194f948644197014e"
dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
mode: GRACEFUL
new_leader_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
 from {username='slave'} at 127.0.0.1:57408
I20260709 13:06:35.301184 12345 raft_consensus.cc:606] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Received request to transfer leadership to TS ac90bb31a0db4dcf8f15b5a12cb1728e
I20260709 13:06:35.497442 12430 consensus_queue.cc:1048] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.519598 12430 consensus_queue.cc:1048] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.534323 12430 raft_consensus.cc:993] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: : Instructing follower ac90bb31a0db4dcf8f15b5a12cb1728e to start an election
I20260709 13:06:35.535012 12405 raft_consensus.cc:1081] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Signalling peer ac90bb31a0db4dcf8f15b5a12cb1728e to start an election
I20260709 13:06:35.538040 12196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99706f5a1f9b48c194f948644197014e"
dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
 from {username='slave'} at 127.0.0.1:52626
I20260709 13:06:35.538899 12196 raft_consensus.cc:493] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:35.539453 12196 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:35.547609 12196 raft_consensus.cc:515] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.552035 12196 leader_election.cc:290] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 2 election: Requested vote from peers 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171), 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391)
I20260709 13:06:35.553373 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "348081022fbf46babc8493994d4ff6b7"
I20260709 13:06:35.553400 12345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
I20260709 13:06:35.554867 12269 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:35.555022 12345 raft_consensus.cc:3055] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:35.555764 12345 raft_consensus.cc:740] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Running, Role: LEADER
I20260709 13:06:35.557552 12345 consensus_queue.cc:260] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.560146 12345 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:35.563628 12269 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 2.
I20260709 13:06:35.565147 12161 leader_election.cc:304] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348081022fbf46babc8493994d4ff6b7, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:35.566111 12423 raft_consensus.cc:2804] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:35.566668 12423 raft_consensus.cc:697] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 LEADER]: Becoming Leader. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Running, Role: LEADER
I20260709 13:06:35.567718 12423 consensus_queue.cc:237] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.570865 12345 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90bb31a0db4dcf8f15b5a12cb1728e in term 2.
I20260709 13:06:35.576617 12402 consensus_queue.cc:1048] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Connected to new peer: Peer: permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:35.583503 12091 catalog_manager.cc:5697] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e reported cstate change: term changed from 1 to 2, leader changed from 44d28a6a8f0f49de902578a825e503ad (127.0.107.195) to ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.592278 12425 consensus_queue.cc:1048] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:35.623350 12430 raft_consensus.cc:993] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: : Instructing follower 348081022fbf46babc8493994d4ff6b7 to start an election
I20260709 13:06:35.623909 12405 raft_consensus.cc:1081] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Signalling peer 348081022fbf46babc8493994d4ff6b7 to start an election
I20260709 13:06:35.625816 12269 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd"
dest_uuid: "348081022fbf46babc8493994d4ff6b7"
 from {username='slave'} at 127.0.0.1:47634
I20260709 13:06:35.626458 12269 raft_consensus.cc:493] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:35.626818 12269 raft_consensus.cc:3060] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:35.634387 12269 raft_consensus.cc:515] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.637245 12269 leader_election.cc:290] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [CANDIDATE]: Term 2 election: Requested vote from peers ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405), 44d28a6a8f0f49de902578a825e503ad (127.0.107.195:37391)
I20260709 13:06:35.655148 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd" candidate_uuid: "348081022fbf46babc8493994d4ff6b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
I20260709 13:06:35.655895 12196 raft_consensus.cc:3060] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:35.656659 12345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe367654cac84faeb4006bfb4af27afd" candidate_uuid: "348081022fbf46babc8493994d4ff6b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
I20260709 13:06:35.657558 12345 raft_consensus.cc:3055] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:35.657938 12345 raft_consensus.cc:740] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Running, Role: LEADER
I20260709 13:06:35.658998 12345 consensus_queue.cc:260] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.660641 12345 raft_consensus.cc:3060] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:35.665958 12196 raft_consensus.cc:2468] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348081022fbf46babc8493994d4ff6b7 in term 2.
I20260709 13:06:35.667759 12234 leader_election.cc:304] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348081022fbf46babc8493994d4ff6b7, ac90bb31a0db4dcf8f15b5a12cb1728e; no voters: 
I20260709 13:06:35.669334 12396 raft_consensus.cc:2804] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:35.670354 12345 raft_consensus.cc:2468] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348081022fbf46babc8493994d4ff6b7 in term 2.
I20260709 13:06:35.671468 12396 raft_consensus.cc:697] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 2 LEADER]: Becoming Leader. State: Replica: 348081022fbf46babc8493994d4ff6b7, State: Running, Role: LEADER
I20260709 13:06:35.672668 12396 consensus_queue.cc:237] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:35.681452 12091 catalog_manager.cc:5697] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 reported cstate change: term changed from 1 to 2, leader changed from 44d28a6a8f0f49de902578a825e503ad (127.0.107.195) to 348081022fbf46babc8493994d4ff6b7 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "348081022fbf46babc8493994d4ff6b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:35.717139   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:06:35.719424   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:06:35.720031   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:06:35.737236 12196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99706f5a1f9b48c194f948644197014e"
dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
mode: GRACEFUL
new_leader_uuid: "44d28a6a8f0f49de902578a825e503ad"
 from {username='slave'} at 127.0.0.1:52634
I20260709 13:06:35.737900 12196 raft_consensus.cc:606] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 LEADER]: Received request to transfer leadership to TS 44d28a6a8f0f49de902578a825e503ad
I20260709 13:06:35.739526   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:06:36.048943 12345 raft_consensus.cc:1275] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Refusing update from remote peer ac90bb31a0db4dcf8f15b5a12cb1728e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.051882 12394 consensus_queue.cc:1048] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:36.062700 12402 raft_consensus.cc:993] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: : Instructing follower 44d28a6a8f0f49de902578a825e503ad to start an election
I20260709 13:06:36.063663 12402 raft_consensus.cc:1081] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 LEADER]: Signalling peer 44d28a6a8f0f49de902578a825e503ad to start an election
I20260709 13:06:36.067293 12344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99706f5a1f9b48c194f948644197014e"
dest_uuid: "44d28a6a8f0f49de902578a825e503ad"
 from {username='slave'} at 127.0.0.1:57398
I20260709 13:06:36.068646 12344 raft_consensus.cc:493] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:36.069203 12344 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:36.075737 12269 raft_consensus.cc:1275] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 2 FOLLOWER]: Refusing update from remote peer ac90bb31a0db4dcf8f15b5a12cb1728e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.079345 12402 consensus_queue.cc:1048] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [LEADER]: Connected to new peer: Peer: permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:36.089838 12344 raft_consensus.cc:515] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:36.091764 12345 raft_consensus.cc:1240] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 FOLLOWER]: Rejecting Update request from peer ac90bb31a0db4dcf8f15b5a12cb1728e for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:36.094259 12344 leader_election.cc:290] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 3 election: Requested vote from peers ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193:42405), 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171)
I20260709 13:06:36.097091 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "348081022fbf46babc8493994d4ff6b7"
I20260709 13:06:36.097589 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99706f5a1f9b48c194f948644197014e" candidate_uuid: "44d28a6a8f0f49de902578a825e503ad" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e"
I20260709 13:06:36.098434 12269 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:36.098673 12196 raft_consensus.cc:3055] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:36.100867 12196 raft_consensus.cc:740] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac90bb31a0db4dcf8f15b5a12cb1728e, State: Running, Role: LEADER
I20260709 13:06:36.102211 12196 consensus_queue.cc:260] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:36.103894 12196 raft_consensus.cc:3060] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:36.111987 12269 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 3.
I20260709 13:06:36.116263 12310 leader_election.cc:304] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348081022fbf46babc8493994d4ff6b7, 44d28a6a8f0f49de902578a825e503ad; no voters: 
I20260709 13:06:36.117908 12405 raft_consensus.cc:2804] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:36.118607 12405 raft_consensus.cc:697] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 LEADER]: Becoming Leader. State: Replica: 44d28a6a8f0f49de902578a825e503ad, State: Running, Role: LEADER
I20260709 13:06:36.120962 12196 raft_consensus.cc:2468] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d28a6a8f0f49de902578a825e503ad in term 3.
I20260709 13:06:36.123067 12405 consensus_queue.cc:237] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } }
I20260709 13:06:36.139403 12091 catalog_manager.cc:5697] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad reported cstate change: term changed from 2 to 3, leader changed from ac90bb31a0db4dcf8f15b5a12cb1728e (127.0.107.193) to 44d28a6a8f0f49de902578a825e503ad (127.0.107.195). New cstate: current_term: 3 leader_uuid: "44d28a6a8f0f49de902578a825e503ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348081022fbf46babc8493994d4ff6b7" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:36.150545 12345 raft_consensus.cc:1275] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Refusing update from remote peer 348081022fbf46babc8493994d4ff6b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.154632 12396 consensus_queue.cc:1048] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d28a6a8f0f49de902578a825e503ad" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:36.170269 12196 raft_consensus.cc:1275] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Refusing update from remote peer 348081022fbf46babc8493994d4ff6b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:36.172739 12442 consensus_queue.cc:1048] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac90bb31a0db4dcf8f15b5a12cb1728e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:36.445600   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:36.482175   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:36.483057   431 tablet_replica.cc:333] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e: stopping tablet replica
I20260709 13:06:36.483968   431 raft_consensus.cc:2243] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.484616   431 raft_consensus.cc:2272] T 99706f5a1f9b48c194f948644197014e P ac90bb31a0db4dcf8f15b5a12cb1728e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.488610   431 tablet_replica.cc:333] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e: stopping tablet replica
I20260709 13:06:36.489427   431 raft_consensus.cc:2243] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:36.490708   431 raft_consensus.cc:2272] T 29eea3e418864f638583682f76b83843 P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.495656   431 tablet_replica.cc:333] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e: stopping tablet replica
I20260709 13:06:36.496671   431 raft_consensus.cc:2243] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.497305   431 raft_consensus.cc:2272] T fe367654cac84faeb4006bfb4af27afd P ac90bb31a0db4dcf8f15b5a12cb1728e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.501075   431 tablet_replica.cc:333] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e: stopping tablet replica
I20260709 13:06:36.501838   431 raft_consensus.cc:2243] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:36.503571   431 raft_consensus.cc:2272] T e22a7bf9822240eca5e4f60103be9d8a P ac90bb31a0db4dcf8f15b5a12cb1728e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.529377   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:06:36.555065   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:06:36.588810   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:36.589874   431 tablet_replica.cc:333] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7: stopping tablet replica
I20260709 13:06:36.591352   431 raft_consensus.cc:2243] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.592046   431 raft_consensus.cc:2272] T 99706f5a1f9b48c194f948644197014e P 348081022fbf46babc8493994d4ff6b7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.597580   431 tablet_replica.cc:333] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7: stopping tablet replica
I20260709 13:06:36.598688   431 raft_consensus.cc:2243] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.599663   431 raft_consensus.cc:2272] T 29eea3e418864f638583682f76b83843 P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.603751   431 tablet_replica.cc:333] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7: stopping tablet replica
I20260709 13:06:36.604614   431 raft_consensus.cc:2243] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:36.606137   431 raft_consensus.cc:2272] T fe367654cac84faeb4006bfb4af27afd P 348081022fbf46babc8493994d4ff6b7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:36.607028 12310 consensus_peers.cc:597] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad -> Peer 348081022fbf46babc8493994d4ff6b7 (127.0.107.194:40171): Couldn't send request to peer 348081022fbf46babc8493994d4ff6b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:36.611704   431 tablet_replica.cc:333] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7: stopping tablet replica
I20260709 13:06:36.612519   431 raft_consensus.cc:2243] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.613332   431 raft_consensus.cc:2272] T e22a7bf9822240eca5e4f60103be9d8a P 348081022fbf46babc8493994d4ff6b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.644917   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:06:36.668274   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:36.702751   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:36.703855   431 tablet_replica.cc:333] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad: stopping tablet replica
I20260709 13:06:36.704788   431 raft_consensus.cc:2243] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.705360   431 raft_consensus.cc:2272] T 29eea3e418864f638583682f76b83843 P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.708870   431 tablet_replica.cc:333] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: stopping tablet replica
I20260709 13:06:36.709537   431 raft_consensus.cc:2243] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:36.710469   431 pending_rounds.cc:70] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Trying to abort 2 pending ops.
I20260709 13:06:36.710664   431 pending_rounds.cc:77] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635414537269248 op_type: NO_OP noop_request { }
I20260709 13:06:36.711105   431 raft_consensus.cc:2889] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:36.711465   431 pending_rounds.cc:77] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635412272771072 op_type: NO_OP noop_request { }
I20260709 13:06:36.711796   431 raft_consensus.cc:2889] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:36.712417   431 raft_consensus.cc:2272] T 99706f5a1f9b48c194f948644197014e P 44d28a6a8f0f49de902578a825e503ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.715885   431 tablet_replica.cc:333] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad: stopping tablet replica
I20260709 13:06:36.716698   431 raft_consensus.cc:2243] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.717270   431 raft_consensus.cc:2272] T e22a7bf9822240eca5e4f60103be9d8a P 44d28a6a8f0f49de902578a825e503ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.720095   431 tablet_replica.cc:333] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad: stopping tablet replica
I20260709 13:06:36.720733   431 raft_consensus.cc:2243] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.721212   431 raft_consensus.cc:2272] T fe367654cac84faeb4006bfb4af27afd P 44d28a6a8f0f49de902578a825e503ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.752630   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:36.778311   431 master.cc:562] Master@127.0.107.254:45195 shutting down...
I20260709 13:06:36.810662   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:36.811728   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.812222   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 18e0b5f4251c4819986f05346e1334c0: stopping tablet replica
I20260709 13:06:36.835386   431 master.cc:584] Master@127.0.107.254:45195 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3265 ms)
[----------] 6 tests from LeaderRebalancerTest (394107 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:06:36.892413   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:36045
I20260709 13:06:36.893792   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:36.900979 12447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:36.901051 12448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:36.902872 12450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:36.904107   431 server_base.cc:1061] running on GCE node
I20260709 13:06:36.905438   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:36.905654   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:36.905783   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602396905770 us; error 0 us; skew 500 ppm
I20260709 13:06:36.906383   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:36.910847   431 webserver.cc:533] Webserver started at http://127.0.107.254:42607/ using document root <none> and password file <none>
I20260709 13:06:36.912292   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:36.912882   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:36.913331   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:36.915488   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "b4d2da7b000443c5b6a2a1e688d1a339"
format_stamp: "Formatted at 2026-07-09 13:06:36 on dist-test-slave-ftjc"
I20260709 13:06:36.921881   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.009s	sys 0.000s
I20260709 13:06:36.926688 12455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:36.928110   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:36.928534   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "b4d2da7b000443c5b6a2a1e688d1a339"
format_stamp: "Formatted at 2026-07-09 13:06:36 on dist-test-slave-ftjc"
I20260709 13:06:36.928887   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:36.959937   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:36.961664   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:37.016458   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:36045
I20260709 13:06:37.016649 12506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:36045 every 8 connection(s)
I20260709 13:06:37.021508 12507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.041746 12507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339: Bootstrap starting.
I20260709 13:06:37.048130 12507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:37.054598 12507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339: No bootstrap required, opened a new log
I20260709 13:06:37.057031 12507 raft_consensus.cc:359] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" member_type: VOTER }
I20260709 13:06:37.057502 12507 raft_consensus.cc:385] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:37.057710 12507 raft_consensus.cc:740] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d2da7b000443c5b6a2a1e688d1a339, State: Initialized, Role: FOLLOWER
I20260709 13:06:37.058495 12507 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" member_type: VOTER }
I20260709 13:06:37.059123 12507 raft_consensus.cc:399] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:37.059341 12507 raft_consensus.cc:493] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:37.059589 12507 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:37.066648 12507 raft_consensus.cc:515] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" member_type: VOTER }
I20260709 13:06:37.067286 12507 leader_election.cc:304] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4d2da7b000443c5b6a2a1e688d1a339; no voters: 
I20260709 13:06:37.068930 12507 leader_election.cc:290] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:37.069468 12510 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:37.071790 12510 raft_consensus.cc:697] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 1 LEADER]: Becoming Leader. State: Replica: b4d2da7b000443c5b6a2a1e688d1a339, State: Running, Role: LEADER
I20260709 13:06:37.072674 12510 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" member_type: VOTER }
I20260709 13:06:37.073050 12507 sys_catalog.cc:565] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:37.089196 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4d2da7b000443c5b6a2a1e688d1a339. Latest consensus state: current_term: 1 leader_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" member_type: VOTER } }
I20260709 13:06:37.090153 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:37.089995 12511 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d2da7b000443c5b6a2a1e688d1a339" member_type: VOTER } }
I20260709 13:06:37.091362 12511 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:37.095412 12522 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:37.102418 12522 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:37.103796   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:37.116819 12522 catalog_manager.cc:1383] Generated new cluster ID: cca47569c4f5428bb7fc6eb1dee45c1c
I20260709 13:06:37.117239 12522 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:37.136646 12522 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:37.138803 12522 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:37.155656 12522 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339: Generated new TSK 0
I20260709 13:06:37.156723 12522 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:37.175643   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:37.183419 12528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:37.184783 12529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:37.187192 12531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:37.187398   431 server_base.cc:1061] running on GCE node
I20260709 13:06:37.189059   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:37.189327   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:37.189476   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602397189464 us; error 0 us; skew 500 ppm
I20260709 13:06:37.190212   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:37.193617   431 webserver.cc:533] Webserver started at http://127.0.107.193:35319/ using document root <none> and password file <none>
I20260709 13:06:37.194181   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:37.194463   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:37.194773   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:37.196084   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "b8c676f2c2264c40b7833e26cf972b7e"
format_stamp: "Formatted at 2026-07-09 13:06:37 on dist-test-slave-ftjc"
I20260709 13:06:37.202135   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:37.206665 12536 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.207774   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:37.208119   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "b8c676f2c2264c40b7833e26cf972b7e"
format_stamp: "Formatted at 2026-07-09 13:06:37 on dist-test-slave-ftjc"
I20260709 13:06:37.208546   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:37.229002   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:37.230556   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:37.232743   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:37.235630   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:37.235944   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.236256   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:37.236582   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:06:37.296548   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:45497
I20260709 13:06:37.296764 12598 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:45497 every 8 connection(s)
I20260709 13:06:37.303227   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:37.312438 12603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:37.316991 12604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:37.317952   431 server_base.cc:1061] running on GCE node
W20260709 13:06:37.319962 12606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:37.321398   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:37.321707   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:37.321962   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602397321938 us; error 0 us; skew 500 ppm
I20260709 13:06:37.322062 12599 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36045
I20260709 13:06:37.322588 12599 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:37.322962   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:37.323735 12599 heartbeater.cc:507] Master 127.0.107.254:36045 requested a full tablet report, sending...
I20260709 13:06:37.326603   431 webserver.cc:533] Webserver started at http://127.0.107.194:39007/ using document root <none> and password file <none>
I20260709 13:06:37.326830 12472 ts_manager.cc:194] Registered new tserver with Master: b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497)
I20260709 13:06:37.327342   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:37.327674   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:37.327996   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:37.329322 12472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56836
I20260709 13:06:37.329592   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "4d47e229db4042c1b76fd3976dd01e8c"
format_stamp: "Formatted at 2026-07-09 13:06:37 on dist-test-slave-ftjc"
I20260709 13:06:37.336820   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:06:37.342145 12611 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.343472   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.004s
I20260709 13:06:37.343940   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "4d47e229db4042c1b76fd3976dd01e8c"
format_stamp: "Formatted at 2026-07-09 13:06:37 on dist-test-slave-ftjc"
I20260709 13:06:37.344249   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:37.363057   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:37.364837   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:37.367538   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:37.370649   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:37.370963   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.371273   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:37.371477   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.440384   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:37275
I20260709 13:06:37.440755 12673 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:37275 every 8 connection(s)
I20260709 13:06:37.447050   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:37.460848 12677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:37.461198 12678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:37.467260 12674 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36045
W20260709 13:06:37.467437 12680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:37.467962 12674 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:37.469277 12674 heartbeater.cc:507] Master 127.0.107.254:36045 requested a full tablet report, sending...
I20260709 13:06:37.470266   431 server_base.cc:1061] running on GCE node
I20260709 13:06:37.471873   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:37.472200   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:37.472522   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602397472495 us; error 0 us; skew 500 ppm
I20260709 13:06:37.472666 12472 ts_manager.cc:194] Registered new tserver with Master: 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:37.473461   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:37.475278 12472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56848
I20260709 13:06:37.477682   431 webserver.cc:533] Webserver started at http://127.0.107.195:42325/ using document root <none> and password file <none>
I20260709 13:06:37.478471   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:37.478688   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:37.479002   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:37.480470   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
format_stamp: "Formatted at 2026-07-09 13:06:37 on dist-test-slave-ftjc"
I20260709 13:06:37.486303   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 13:06:37.491374 12685 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.492689   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 13:06:37.493110   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
format_stamp: "Formatted at 2026-07-09 13:06:37 on dist-test-slave-ftjc"
I20260709 13:06:37.493454   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:37.539007   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:37.540472   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:37.542599   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:37.545897   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:37.546175   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.546537   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:37.546705   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:37.604360   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:34547
I20260709 13:06:37.604576 12747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:34547 every 8 connection(s)
I20260709 13:06:37.623251 12748 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36045
I20260709 13:06:37.623837 12748 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:37.624871 12748 heartbeater.cc:507] Master 127.0.107.254:36045 requested a full tablet report, sending...
I20260709 13:06:37.628054 12472 ts_manager.cc:194] Registered new tserver with Master: 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:37.629155   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020512463s
I20260709 13:06:37.630223 12472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56862
I20260709 13:06:37.667309 12472 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56866:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:37.670831 12472 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:37.775656 12564 tablet_service.cc:1511] Processing CreateTablet for tablet c8a84ad985aa4f64a366695c31ee67e3 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:37.777549 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a84ad985aa4f64a366695c31ee67e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.777818 12639 tablet_service.cc:1511] Processing CreateTablet for tablet c8a84ad985aa4f64a366695c31ee67e3 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:37.779443 12639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a84ad985aa4f64a366695c31ee67e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.781684 12563 tablet_service.cc:1511] Processing CreateTablet for tablet 5322ad4126d44f58989dbc5a195acd33 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:37.783052 12637 tablet_service.cc:1511] Processing CreateTablet for tablet d507a98e09764fcbb8e0c5b00e90d829 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:37.783357 12635 tablet_service.cc:1511] Processing CreateTablet for tablet 1197098bc264479cba55fe28105b7432 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:37.785151 12635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1197098bc264479cba55fe28105b7432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.785663 12636 tablet_service.cc:1511] Processing CreateTablet for tablet 3989aa91bcd34ca48467526cc431d712 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:37.787583 12636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3989aa91bcd34ca48467526cc431d712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.787873 12562 tablet_service.cc:1511] Processing CreateTablet for tablet d507a98e09764fcbb8e0c5b00e90d829 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:37.788838 12637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d507a98e09764fcbb8e0c5b00e90d829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.781684 12638 tablet_service.cc:1511] Processing CreateTablet for tablet 5322ad4126d44f58989dbc5a195acd33 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:37.785811 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 3989aa91bcd34ca48467526cc431d712 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:37.783553 12563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5322ad4126d44f58989dbc5a195acd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.793135 12634 tablet_service.cc:1511] Processing CreateTablet for tablet ebf63419e0c04cb98a73d13ee04340a9 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:37.793988 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 1197098bc264479cba55fe28105b7432 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:37.795759 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1197098bc264479cba55fe28105b7432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.797717 12638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5322ad4126d44f58989dbc5a195acd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.799754 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d507a98e09764fcbb8e0c5b00e90d829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.801739 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3989aa91bcd34ca48467526cc431d712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.805989 12559 tablet_service.cc:1511] Processing CreateTablet for tablet ebf63419e0c04cb98a73d13ee04340a9 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:37.810797 12558 tablet_service.cc:1511] Processing CreateTablet for tablet 77a9074ed8cc4d38b77ec07e44134c6e (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:37.812071 12633 tablet_service.cc:1511] Processing CreateTablet for tablet 77a9074ed8cc4d38b77ec07e44134c6e (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:37.814776 12633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a9074ed8cc4d38b77ec07e44134c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.817633 12634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf63419e0c04cb98a73d13ee04340a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.817376 12632 tablet_service.cc:1511] Processing CreateTablet for tablet bbb54e72ac8946d0a99c1d2409745095 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:37.808104 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf63419e0c04cb98a73d13ee04340a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.819295 12632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb54e72ac8946d0a99c1d2409745095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.824836 12631 tablet_service.cc:1511] Processing CreateTablet for tablet 3af3a07da5e8476f9005e2ded8f90fdb (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:37.826946 12631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af3a07da5e8476f9005e2ded8f90fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.830593 12712 tablet_service.cc:1511] Processing CreateTablet for tablet 5322ad4126d44f58989dbc5a195acd33 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:37.831769 12710 tablet_service.cc:1511] Processing CreateTablet for tablet 3989aa91bcd34ca48467526cc431d712 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:37.832765 12712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5322ad4126d44f58989dbc5a195acd33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.833935 12713 tablet_service.cc:1511] Processing CreateTablet for tablet c8a84ad985aa4f64a366695c31ee67e3 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:37.834400 12557 tablet_service.cc:1511] Processing CreateTablet for tablet bbb54e72ac8946d0a99c1d2409745095 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:37.835374 12713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8a84ad985aa4f64a366695c31ee67e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.834856 12709 tablet_service.cc:1511] Processing CreateTablet for tablet 1197098bc264479cba55fe28105b7432 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:37.838008 12711 tablet_service.cc:1511] Processing CreateTablet for tablet d507a98e09764fcbb8e0c5b00e90d829 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:37.838428 12556 tablet_service.cc:1511] Processing CreateTablet for tablet 3af3a07da5e8476f9005e2ded8f90fdb (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:37.839953 12711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d507a98e09764fcbb8e0c5b00e90d829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.840059 12707 tablet_service.cc:1511] Processing CreateTablet for tablet 77a9074ed8cc4d38b77ec07e44134c6e (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:37.841795 12707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a9074ed8cc4d38b77ec07e44134c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.846263 12706 tablet_service.cc:1511] Processing CreateTablet for tablet bbb54e72ac8946d0a99c1d2409745095 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:37.848209 12705 tablet_service.cc:1511] Processing CreateTablet for tablet 3af3a07da5e8476f9005e2ded8f90fdb (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:37.838281 12708 tablet_service.cc:1511] Processing CreateTablet for tablet ebf63419e0c04cb98a73d13ee04340a9 (DEFAULT_TABLE table=test-workload [id=0761ca3abad24c4694de896ec72d40b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:37.853252 12706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb54e72ac8946d0a99c1d2409745095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.855752 12705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af3a07da5e8476f9005e2ded8f90fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.856851 12710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3989aa91bcd34ca48467526cc431d712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.824827 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a9074ed8cc4d38b77ec07e44134c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.858208 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af3a07da5e8476f9005e2ded8f90fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.859119 12709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1197098bc264479cba55fe28105b7432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.860476 12557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb54e72ac8946d0a99c1d2409745095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:37.860599 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf63419e0c04cb98a73d13ee04340a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:38.008383 12766 tablet_bootstrap.cc:492] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.025056 12766 tablet_bootstrap.cc:654] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.065347 12768 tablet_bootstrap.cc:492] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.067687 12767 tablet_bootstrap.cc:492] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.068542 12766 tablet_bootstrap.cc:492] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.069190 12766 ts_tablet_manager.cc:1403] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.038s
I20260709 13:06:38.073532 12766 raft_consensus.cc:359] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.074532 12766 raft_consensus.cc:385] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.074900 12766 raft_consensus.cc:740] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.075124 12768 tablet_bootstrap.cc:654] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.076139 12766 consensus_queue.cc:260] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.078327 12767 tablet_bootstrap.cc:654] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.080256 12748 heartbeater.cc:499] Master 127.0.107.254:36045 was elected leader, sending a full tablet report...
I20260709 13:06:38.080502 12766 ts_tablet_manager.cc:1434] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.001s
I20260709 13:06:38.082408 12766 tablet_bootstrap.cc:492] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
W20260709 13:06:38.086280 12518 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:38.092424 12768 tablet_bootstrap.cc:492] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.093364 12766 tablet_bootstrap.cc:654] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.093510 12768 ts_tablet_manager.cc:1403] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.000s
I20260709 13:06:38.094414 12767 tablet_bootstrap.cc:492] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.095109 12767 ts_tablet_manager.cc:1403] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.006s
I20260709 13:06:38.098219 12768 raft_consensus.cc:359] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.099076 12767 raft_consensus.cc:359] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.100049 12768 raft_consensus.cc:385] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.101282 12768 raft_consensus.cc:740] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.102833 12768 consensus_queue.cc:260] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.100087 12767 raft_consensus.cc:385] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.104154 12767 raft_consensus.cc:740] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.105301 12767 consensus_queue.cc:260] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.107303 12766 tablet_bootstrap.cc:492] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.108183 12766 ts_tablet_manager.cc:1403] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.005s
I20260709 13:06:38.115497 12766 raft_consensus.cc:359] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.116586 12766 raft_consensus.cc:385] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.116973 12766 raft_consensus.cc:740] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.118467 12766 consensus_queue.cc:260] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.121800 12766 ts_tablet_manager.cc:1434] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:06:38.120643 12674 heartbeater.cc:499] Master 127.0.107.254:36045 was elected leader, sending a full tablet report...
I20260709 13:06:38.125844 12599 heartbeater.cc:499] Master 127.0.107.254:36045 was elected leader, sending a full tablet report...
I20260709 13:06:38.127516 12766 tablet_bootstrap.cc:492] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.121599 12768 ts_tablet_manager.cc:1434] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.028s	user 0.015s	sys 0.009s
I20260709 13:06:38.129451 12768 tablet_bootstrap.cc:492] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.129992 12767 ts_tablet_manager.cc:1434] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.034s	user 0.018s	sys 0.004s
I20260709 13:06:38.138397 12766 tablet_bootstrap.cc:654] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.139104 12767 tablet_bootstrap.cc:492] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.145454 12768 tablet_bootstrap.cc:654] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.147977 12767 tablet_bootstrap.cc:654] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.151052 12770 raft_consensus.cc:493] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.152045 12770 raft_consensus.cc:515] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.158825 12766 tablet_bootstrap.cc:492] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.159631 12766 ts_tablet_manager.cc:1403] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.008s
I20260709 13:06:38.163419 12766 raft_consensus.cc:359] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.164378 12766 raft_consensus.cc:385] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.164738 12766 raft_consensus.cc:740] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.165704 12766 consensus_queue.cc:260] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.172137 12768 tablet_bootstrap.cc:492] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.173360 12768 ts_tablet_manager.cc:1403] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.010s
I20260709 13:06:38.173653 12770 leader_election.cc:290] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.178867 12768 raft_consensus.cc:359] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.180380 12768 raft_consensus.cc:385] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.181077 12768 raft_consensus.cc:740] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.182137 12768 consensus_queue.cc:260] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.187801 12767 tablet_bootstrap.cc:492] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.189036 12767 ts_tablet_manager.cc:1403] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.050s	user 0.028s	sys 0.000s
I20260709 13:06:38.190902 12770 raft_consensus.cc:493] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.191823 12770 raft_consensus.cc:515] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.192544 12768 ts_tablet_manager.cc:1434] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.002s
I20260709 13:06:38.198072 12768 tablet_bootstrap.cc:492] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.200134 12770 leader_election.cc:290] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.195585 12767 raft_consensus.cc:359] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.201521 12767 raft_consensus.cc:385] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.202993 12767 raft_consensus.cc:740] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.206128 12767 consensus_queue.cc:260] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.204531 12766 ts_tablet_manager.cc:1434] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.044s	user 0.031s	sys 0.017s
I20260709 13:06:38.208930 12767 ts_tablet_manager.cc:1434] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 13:06:38.213088 12768 tablet_bootstrap.cc:654] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.213937 12766 tablet_bootstrap.cc:492] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.225623 12781 raft_consensus.cc:493] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.226438 12781 raft_consensus.cc:515] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.229099 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5322ad4126d44f58989dbc5a195acd33" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.229089 12781 leader_election.cc:290] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.231159 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.232475 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af3a07da5e8476f9005e2ded8f90fdb" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.233332 12573 raft_consensus.cc:2468] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:38.233839 12766 tablet_bootstrap.cc:654] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:38.235016 12689 leader_election.cc:343] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.236133 12767 tablet_bootstrap.cc:492] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.236918 12689 leader_election.cc:304] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
W20260709 13:06:38.238070 12689 leader_election.cc:343] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.238808 12781 raft_consensus.cc:2804] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.239249 12781 raft_consensus.cc:493] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.239614 12781 raft_consensus.cc:3060] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.245203 12768 tablet_bootstrap.cc:492] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.245045 12649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.246290 12768 ts_tablet_manager.cc:1403] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.008s
I20260709 13:06:38.247413 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5322ad4126d44f58989dbc5a195acd33" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.250501 12767 tablet_bootstrap.cc:654] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.248078 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af3a07da5e8476f9005e2ded8f90fdb" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.251073 12768 raft_consensus.cc:359] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.252122 12768 raft_consensus.cc:385] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.252583 12768 raft_consensus.cc:740] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
W20260709 13:06:38.249244 12687 leader_election.cc:343] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.253880 12687 leader_election.cc:304] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.254979 12770 raft_consensus.cc:2749] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.254125 12768 consensus_queue.cc:260] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
W20260709 13:06:38.256640 12687 leader_election.cc:343] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.257323 12687 leader_election.cc:304] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.258244 12770 raft_consensus.cc:2749] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.258776 12767 tablet_bootstrap.cc:492] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.258776 12766 tablet_bootstrap.cc:492] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.259749 12766 ts_tablet_manager.cc:1403] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.012s
I20260709 13:06:38.259749 12767 ts_tablet_manager.cc:1403] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:06:38.261370 12768 ts_tablet_manager.cc:1434] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:06:38.262755 12768 tablet_bootstrap.cc:492] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.263949 12766 raft_consensus.cc:359] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.264828 12766 raft_consensus.cc:385] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.265141 12766 raft_consensus.cc:740] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.265676 12767 raft_consensus.cc:359] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.266125 12766 consensus_queue.cc:260] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.267007 12767 raft_consensus.cc:385] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.267419 12767 raft_consensus.cc:740] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.269033 12766 ts_tablet_manager.cc:1434] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:06:38.268631 12767 consensus_queue.cc:260] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.270691 12766 tablet_bootstrap.cc:492] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.275003 12768 tablet_bootstrap.cc:654] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:38.278254 12687 leader_election.cc:343] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.279949 12766 tablet_bootstrap.cc:654] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.288030 12768 tablet_bootstrap.cc:492] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.288770 12768 ts_tablet_manager.cc:1403] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.007s
I20260709 13:06:38.288770 12781 raft_consensus.cc:515] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.292936 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af3a07da5e8476f9005e2ded8f90fdb" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:38.293828 12574 raft_consensus.cc:3060] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.294274 12768 raft_consensus.cc:359] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.295204 12768 raft_consensus.cc:385] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.295555 12768 raft_consensus.cc:740] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.298132 12768 consensus_queue.cc:260] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.298631 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af3a07da5e8476f9005e2ded8f90fdb" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:38.301206 12768 ts_tablet_manager.cc:1434] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.006s
W20260709 13:06:38.301926 12687 leader_election.cc:343] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.302500 12768 tablet_bootstrap.cc:492] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.302726 12766 tablet_bootstrap.cc:492] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.303426 12766 ts_tablet_manager.cc:1403] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.016s
I20260709 13:06:38.303512 12767 ts_tablet_manager.cc:1434] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:06:38.308661 12766 raft_consensus.cc:359] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.309646 12766 raft_consensus.cc:385] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.310115 12766 raft_consensus.cc:740] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.311445 12768 tablet_bootstrap.cc:654] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.311409 12766 consensus_queue.cc:260] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.315384 12766 ts_tablet_manager.cc:1434] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:06:38.317122 12766 tablet_bootstrap.cc:492] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.326920 12766 tablet_bootstrap.cc:654] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.326651 12574 raft_consensus.cc:2468] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:38.328895 12689 leader_election.cc:304] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:38.330569 12770 raft_consensus.cc:2804] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.331482 12767 tablet_bootstrap.cc:492] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.341001 12770 raft_consensus.cc:697] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:38.342793 12770 consensus_queue.cc:237] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.293072 12781 leader_election.cc:290] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.355000 12767 tablet_bootstrap.cc:654] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.359938 12768 tablet_bootstrap.cc:492] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.360487 12766 tablet_bootstrap.cc:492] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.361159 12768 ts_tablet_manager.cc:1403] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.059s	user 0.036s	sys 0.020s
I20260709 13:06:38.361294 12766 ts_tablet_manager.cc:1403] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.044s	user 0.031s	sys 0.011s
I20260709 13:06:38.366791 12766 raft_consensus.cc:359] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.368245 12766 raft_consensus.cc:385] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.368731 12766 raft_consensus.cc:740] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.370086 12766 consensus_queue.cc:260] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.370107 12768 raft_consensus.cc:359] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.371341 12768 raft_consensus.cc:385] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.371726 12768 raft_consensus.cc:740] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.372712 12768 consensus_queue.cc:260] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.373525 12766 ts_tablet_manager.cc:1434] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:06:38.374904 12766 tablet_bootstrap.cc:492] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.374284 12470 catalog_manager.cc:5697] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:38.376116 12768 ts_tablet_manager.cc:1434] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:06:38.377417 12768 tablet_bootstrap.cc:492] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.384593 12766 tablet_bootstrap.cc:654] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.385092 12774 raft_consensus.cc:493] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.385756 12774 raft_consensus.cc:515] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.387033 12768 tablet_bootstrap.cc:654] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.411820 12774 leader_election.cc:290] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.424280 12775 raft_consensus.cc:493] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.425446 12775 raft_consensus.cc:515] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.374198 12767 tablet_bootstrap.cc:492] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.432349 12770 raft_consensus.cc:493] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.434028 12770 raft_consensus.cc:515] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.438396 12770 leader_election.cc:290] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.439471 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.442131 12767 ts_tablet_manager.cc:1403] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.111s	user 0.013s	sys 0.014s
W20260709 13:06:38.442034 12689 leader_election.cc:343] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.445443 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
W20260709 13:06:38.448546 12687 leader_election.cc:343] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.449383 12687 leader_election.cc:304] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.456627 12775 leader_election.cc:290] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.458586 12770 raft_consensus.cc:2749] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.475052 12767 raft_consensus.cc:359] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.480736 12767 raft_consensus.cc:385] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.484787 12767 raft_consensus.cc:740] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.489583 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:38.493851 12775 raft_consensus.cc:493] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.494837 12775 raft_consensus.cc:515] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
W20260709 13:06:38.496258 12537 leader_election.cc:343] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.498612 12775 leader_election.cc:290] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.504165 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.505554 12574 raft_consensus.cc:2468] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 0.
I20260709 13:06:38.509943 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1197098bc264479cba55fe28105b7432" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.511257 12574 raft_consensus.cc:2468] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 0.
I20260709 13:06:38.514110 12615 leader_election.cc:304] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:38.516693 12775 raft_consensus.cc:2804] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.517542 12775 raft_consensus.cc:493] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.518043 12775 raft_consensus.cc:3060] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.515715 12615 leader_election.cc:304] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:38.498423 12767 consensus_queue.cc:260] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.532780 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.538606 12766 tablet_bootstrap.cc:492] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.539654 12766 ts_tablet_manager.cc:1403] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.165s	user 0.037s	sys 0.106s
I20260709 13:06:38.542361 12768 tablet_bootstrap.cc:492] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.543365 12768 ts_tablet_manager.cc:1403] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.166s	user 0.027s	sys 0.057s
I20260709 13:06:38.544495 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
W20260709 13:06:38.547478 12612 leader_election.cc:343] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.548126 12768 raft_consensus.cc:359] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.549557 12768 raft_consensus.cc:385] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.548405 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1197098bc264479cba55fe28105b7432" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:38.549963 12768 raft_consensus.cc:740] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.551254 12802 raft_consensus.cc:2804] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.552649 12802 raft_consensus.cc:493] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.553153 12802 raft_consensus.cc:3060] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:38.552145 12612 leader_election.cc:343] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.557603 12766 raft_consensus.cc:359] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.558575 12766 raft_consensus.cc:385] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.558913 12766 raft_consensus.cc:740] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.551024 12768 consensus_queue.cc:260] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.561290 12770 raft_consensus.cc:493] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.561969 12770 raft_consensus.cc:515] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.565673 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
W20260709 13:06:38.567852 12689 leader_election.cc:343] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.568446 12768 ts_tablet_manager.cc:1434] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.005s
I20260709 13:06:38.570134 12770 leader_election.cc:290] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.559896 12766 consensus_queue.cc:260] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.570267 12802 raft_consensus.cc:515] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.571769 12768 tablet_bootstrap.cc:492] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.573628 12766 ts_tablet_manager.cc:1434] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.005s
I20260709 13:06:38.569828 12775 raft_consensus.cc:515] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.575155 12647 raft_consensus.cc:2393] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8c676f2c2264c40b7833e26cf972b7e in current term 1: Already voted for candidate 4d47e229db4042c1b76fd3976dd01e8c in this term.
I20260709 13:06:38.575538 12802 leader_election.cc:290] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.576699 12775 leader_election.cc:290] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.573823 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.582683 12648 raft_consensus.cc:2468] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:38.584232 12768 tablet_bootstrap.cc:654] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.584578 12687 leader_election.cc:304] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.578411 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:38.586341 12573 raft_consensus.cc:3060] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.578728 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1197098bc264479cba55fe28105b7432" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:38.590222 12574 raft_consensus.cc:3060] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.579741 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:38.575011 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1197098bc264479cba55fe28105b7432" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:38.591913 12723 raft_consensus.cc:3060] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.592613 12770 raft_consensus.cc:2804] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.593166 12770 raft_consensus.cc:493] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.593585 12770 raft_consensus.cc:3060] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.579906 12538 leader_election.cc:304] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8c676f2c2264c40b7833e26cf972b7e; no voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34
W20260709 13:06:38.601634 12612 leader_election.cc:343] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.602605 12781 raft_consensus.cc:493] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.603222 12781 raft_consensus.cc:515] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.605829 12781 leader_election.cc:290] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.580657 12766 tablet_bootstrap.cc:492] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.607388 12572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.608325 12572 raft_consensus.cc:2468] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:38.609689 12573 raft_consensus.cc:2468] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:38.609946 12689 leader_election.cc:304] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:38.610879 12774 raft_consensus.cc:2749] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.612911 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.613953 12648 raft_consensus.cc:2468] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:38.615109 12723 raft_consensus.cc:2468] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:38.563848 12767 ts_tablet_manager.cc:1434] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.121s	user 0.018s	sys 0.017s
I20260709 13:06:38.620497 12768 tablet_bootstrap.cc:492] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.616324 12781 raft_consensus.cc:2804] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.625910 12781 raft_consensus.cc:493] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.626365 12612 leader_election.cc:304] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:38.626444 12781 raft_consensus.cc:3060] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.628787 12770 raft_consensus.cc:515] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.616575 12615 leader_election.cc:304] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:38.631558 12802 raft_consensus.cc:2804] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.632162 12802 raft_consensus.cc:697] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:38.633157 12574 raft_consensus.cc:2468] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:38.633279 12803 raft_consensus.cc:2804] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.633739 12802 consensus_queue.cc:237] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.635488 12770 leader_election.cc:290] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.637041 12767 tablet_bootstrap.cc:492] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.639029 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:38.639879 12648 raft_consensus.cc:3060] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.632031 12766 tablet_bootstrap.cc:654] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.645855 12767 tablet_bootstrap.cc:654] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.638916 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:38.626830 12768 ts_tablet_manager.cc:1403] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.022s
I20260709 13:06:38.647194 12803 raft_consensus.cc:697] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
W20260709 13:06:38.648770 12689 leader_election.cc:343] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.648733 12803 consensus_queue.cc:237] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.651988 12768 raft_consensus.cc:359] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.653275 12768 raft_consensus.cc:385] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.653834 12768 raft_consensus.cc:740] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.654907 12768 consensus_queue.cc:260] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.657583 12781 raft_consensus.cc:515] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.664456 12781 leader_election.cc:290] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.665805 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:38.666831 12647 raft_consensus.cc:3060] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.667066 12648 raft_consensus.cc:2468] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:38.668915 12687 leader_election.cc:304] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.669692 12768 ts_tablet_manager.cc:1434] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:06:38.670710 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:38.671162 12768 tablet_bootstrap.cc:492] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.671486 12574 raft_consensus.cc:3060] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.672958 12470 catalog_manager.cc:5697] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.676613 12781 raft_consensus.cc:2804] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.678297 12781 raft_consensus.cc:697] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:38.682723 12766 tablet_bootstrap.cc:492] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.683429 12766 ts_tablet_manager.cc:1403] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.009s
I20260709 13:06:38.686713 12766 raft_consensus.cc:359] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.687374 12766 raft_consensus.cc:385] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.687693 12766 raft_consensus.cc:740] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.688129 12647 raft_consensus.cc:2468] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:38.688721 12766 consensus_queue.cc:260] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.690186 12687 leader_election.cc:304] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:38.679289 12781 consensus_queue.cc:237] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.691169 12770 raft_consensus.cc:2804] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.691777 12770 raft_consensus.cc:697] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:38.693173 12770 consensus_queue.cc:237] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.697840 12768 tablet_bootstrap.cc:654] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.698810 12766 ts_tablet_manager.cc:1434] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.006s
I20260709 13:06:38.700253 12766 tablet_bootstrap.cc:492] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:38.706063 12767 tablet_bootstrap.cc:492] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.706954 12767 ts_tablet_manager.cc:1403] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.009s
I20260709 13:06:38.708271 12766 tablet_bootstrap.cc:654] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.707063 12469 catalog_manager.cc:5697] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.711404 12767 raft_consensus.cc:359] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.712733 12767 raft_consensus.cc:385] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.714129 12767 raft_consensus.cc:740] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.719216 12767 consensus_queue.cc:260] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.722939 12767 ts_tablet_manager.cc:1434] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:06:38.722707 12471 catalog_manager.cc:5697] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:38.724958 12767 tablet_bootstrap.cc:492] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.715143 12574 raft_consensus.cc:2468] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:38.730021 12766 tablet_bootstrap.cc:492] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:38.714470 12768 tablet_bootstrap.cc:492] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.730795 12766 ts_tablet_manager.cc:1403] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.008s
I20260709 13:06:38.731035 12768 ts_tablet_manager.cc:1403] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.002s
I20260709 13:06:38.735390 12767 tablet_bootstrap.cc:654] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.735227 12766 raft_consensus.cc:359] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.736092 12766 raft_consensus.cc:385] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.736465 12766 raft_consensus.cc:740] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.736387 12768 raft_consensus.cc:359] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.737416 12768 raft_consensus.cc:385] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.737910 12768 raft_consensus.cc:740] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.737740 12766 consensus_queue.cc:260] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.738830 12768 consensus_queue.cc:260] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.741127 12768 ts_tablet_manager.cc:1434] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:38.741407 12766 ts_tablet_manager.cc:1434] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:38.742211 12768 tablet_bootstrap.cc:492] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:38.746522 12767 tablet_bootstrap.cc:492] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.747298 12767 ts_tablet_manager.cc:1403] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.000s
I20260709 13:06:38.749689 12768 tablet_bootstrap.cc:654] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.750268 12774 raft_consensus.cc:493] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.751191 12770 raft_consensus.cc:493] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.751008 12774 raft_consensus.cc:515] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.751742 12770 raft_consensus.cc:515] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.754825 12770 leader_election.cc:290] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:38.756999 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.757536 12767 raft_consensus.cc:359] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.758323 12767 raft_consensus.cc:385] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.758641 12767 raft_consensus.cc:740] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.759480 12767 consensus_queue.cc:260] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.760947 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3989aa91bcd34ca48467526cc431d712" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.762243 12767 ts_tablet_manager.cc:1434] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.008s
W20260709 13:06:38.763013 12538 leader_election.cc:343] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.763581 12767 tablet_bootstrap.cc:492] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.767892 12774 leader_election.cc:290] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.769973 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3989aa91bcd34ca48467526cc431d712" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:38.771265 12723 raft_consensus.cc:2468] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:38.773365 12537 leader_election.cc:304] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:38.773732 12767 tablet_bootstrap.cc:654] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.776170 12774 raft_consensus.cc:2804] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.776877 12774 raft_consensus.cc:493] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.777369 12774 raft_consensus.cc:3060] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.778491 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb54e72ac8946d0a99c1d2409745095" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.788021 12574 raft_consensus.cc:2393] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8da3d1ff50e74afbad7414d7e80b9b34 in current term 1: Already voted for candidate 4d47e229db4042c1b76fd3976dd01e8c in this term.
I20260709 13:06:38.785298 12471 catalog_manager.cc:5697] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:38.789963 12689 leader_election.cc:304] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.792066 12770 raft_consensus.cc:3060] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.804979 12770 raft_consensus.cc:2749] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:38.806164 12767 tablet_bootstrap.cc:492] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.806538 12768 tablet_bootstrap.cc:492] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:38.804987 12774 raft_consensus.cc:515] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.806842 12767 ts_tablet_manager.cc:1403] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.030s
I20260709 13:06:38.807129 12768 ts_tablet_manager.cc:1403] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.065s	user 0.045s	sys 0.012s
I20260709 13:06:38.810371 12768 raft_consensus.cc:359] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.810943 12768 raft_consensus.cc:385] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.810930 12767 raft_consensus.cc:359] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.811628 12768 raft_consensus.cc:740] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.811905 12767 raft_consensus.cc:385] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.811590 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3989aa91bcd34ca48467526cc431d712" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:38.813113 12768 consensus_queue.cc:260] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.813897 12767 raft_consensus.cc:740] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
W20260709 13:06:38.814379 12538 leader_election.cc:343] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:38.815114 12774 leader_election.cc:290] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.815002 12767 consensus_queue.cc:260] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.815235 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3989aa91bcd34ca48467526cc431d712" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:38.816220 12723 raft_consensus.cc:3060] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.817777 12767 ts_tablet_manager.cc:1434] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 13:06:38.819651 12767 tablet_bootstrap.cc:492] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.834008 12767 tablet_bootstrap.cc:654] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.843924 12723 raft_consensus.cc:2468] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:38.846325 12537 leader_election.cc:304] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:38.848992 12774 raft_consensus.cc:2804] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.849864 12774 raft_consensus.cc:697] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:38.851643 12774 consensus_queue.cc:237] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.860139 12818 raft_consensus.cc:493] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.860451 12768 ts_tablet_manager.cc:1434] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.053s	user 0.022s	sys 0.032s
I20260709 13:06:38.861016 12818 raft_consensus.cc:515] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.863577 12818 leader_election.cc:290] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.865103 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:38.865211 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:38.866405 12647 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:38.866475 12723 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:38.867551 12767 tablet_bootstrap.cc:492] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.868207 12767 ts_tablet_manager.cc:1403] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.022s
I20260709 13:06:38.868242 12537 leader_election.cc:304] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:38.872046 12774 raft_consensus.cc:2804] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:38.872011 12767 raft_consensus.cc:359] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.872936 12774 raft_consensus.cc:493] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.873607 12774 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.874481 12472 catalog_manager.cc:5697] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.873102 12767 raft_consensus.cc:385] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.876109 12767 raft_consensus.cc:740] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.877017 12767 consensus_queue.cc:260] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.879356 12767 ts_tablet_manager.cc:1434] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:06:38.880609 12767 tablet_bootstrap.cc:492] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:38.886409 12774 raft_consensus.cc:515] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.887637 12767 tablet_bootstrap.cc:654] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:38.890921 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:38.891804 12647 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.900687 12767 tablet_bootstrap.cc:492] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:38.901384 12767 ts_tablet_manager.cc:1403] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 13:06:38.903378 12647 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:38.904704 12538 leader_election.cc:304] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:38.904750 12767 raft_consensus.cc:359] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.904840 12770 consensus_queue.cc:1048] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:38.905773 12767 raft_consensus.cc:385] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:38.905915 12818 raft_consensus.cc:2804] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:38.906198 12767 raft_consensus.cc:740] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:38.907162 12818 raft_consensus.cc:697] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:38.907333 12767 consensus_queue.cc:260] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.908016 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:38.908865 12723 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.909343 12818 consensus_queue.cc:237] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.910216 12767 ts_tablet_manager.cc:1434] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:06:38.919746 12723 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:38.920480 12774 leader_election.cc:290] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.923997 12803 raft_consensus.cc:493] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.924731 12803 raft_consensus.cc:515] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:38.927455 12803 leader_election.cc:290] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:38.928572 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3989aa91bcd34ca48467526cc431d712" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:38.928941 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3989aa91bcd34ca48467526cc431d712" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:38.929939 12723 raft_consensus.cc:2393] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d47e229db4042c1b76fd3976dd01e8c in current term 1: Already voted for candidate b8c676f2c2264c40b7833e26cf972b7e in this term.
I20260709 13:06:38.929185 12647 raft_consensus.cc:3060] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.931545 12615 leader_election.cc:304] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c; no voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:38.933926 12803 raft_consensus.cc:3060] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:38.938277 12471 catalog_manager.cc:5697] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.943292 12770 consensus_queue.cc:1048] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:38.948939 12803 raft_consensus.cc:2749] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:39.020891 12818 raft_consensus.cc:493] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.021574 12818 raft_consensus.cc:515] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.023998 12818 leader_election.cc:290] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.025208 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5322ad4126d44f58989dbc5a195acd33" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:39.025492 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5322ad4126d44f58989dbc5a195acd33" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:39.026059 12647 raft_consensus.cc:2468] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:39.026575 12723 raft_consensus.cc:2468] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:39.027585 12538 leader_election.cc:304] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:39.028780 12818 raft_consensus.cc:2804] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:39.029191 12818 raft_consensus.cc:493] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.029521 12818 raft_consensus.cc:3060] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.038142 12818 raft_consensus.cc:515] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.040093 12818 leader_election.cc:290] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.041370 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5322ad4126d44f58989dbc5a195acd33" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:39.041255 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5322ad4126d44f58989dbc5a195acd33" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:39.042224 12723 raft_consensus.cc:3060] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.042348 12647 raft_consensus.cc:3060] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.044946 12818 raft_consensus.cc:493] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.045819 12818 raft_consensus.cc:515] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.049296 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:39.049763 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:39.050246 12648 raft_consensus.cc:2468] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:39.050571 12722 raft_consensus.cc:2468] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:39.049309 12818 leader_election.cc:290] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.051849 12723 raft_consensus.cc:2468] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:39.052122 12537 leader_election.cc:304] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:39.051849 12647 raft_consensus.cc:2468] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:39.053572 12818 raft_consensus.cc:2804] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:39.054066 12818 raft_consensus.cc:493] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.054019 12537 leader_election.cc:304] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:39.054766 12818 raft_consensus.cc:3060] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.055249 12774 raft_consensus.cc:2804] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:39.055744 12774 raft_consensus.cc:697] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:39.056658 12774 consensus_queue.cc:237] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.063199 12818 raft_consensus.cc:515] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.065757 12818 leader_election.cc:290] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.066996 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:39.067890 12647 raft_consensus.cc:3060] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.067216 12472 catalog_manager.cc:5697] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.067945 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:39.068756 12723 raft_consensus.cc:3060] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.072249 12574 raft_consensus.cc:3060] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:39.078279 12723 raft_consensus.cc:2468] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:39.079048 12647 raft_consensus.cc:2468] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:39.079980 12537 leader_election.cc:304] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:39.082242 12818 raft_consensus.cc:2804] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:39.085008 12818 raft_consensus.cc:697] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:39.085147 12770 consensus_queue.cc:1048] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.085911 12818 consensus_queue.cc:237] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
W20260709 13:06:39.091656 12518 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:39.096398 12770 consensus_queue.cc:1048] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.106310 12471 catalog_manager.cc:5697] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.116470 12770 consensus_queue.cc:1048] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.122970 12770 consensus_queue.cc:1048] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.128382 12802 consensus_queue.cc:1048] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.155793 12803 consensus_queue.cc:1048] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:39.231657 12803 consensus_queue.cc:1048] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:39.236805 12722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
mode: GRACEFUL
new_leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:41738
I20260709 13:06:39.237579 12722 raft_consensus.cc:606] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Received request to transfer leadership to TS b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:39.246764 12803 consensus_queue.cc:1048] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.286015 12573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:47704
I20260709 13:06:39.286897 12573 raft_consensus.cc:606] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:39.294301 12723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
mode: GRACEFUL
new_leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:41738
I20260709 13:06:39.295073 12723 raft_consensus.cc:606] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Received request to transfer leadership to TS 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:39.307385 12818 consensus_queue.cc:1048] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:39.323148 12820 consensus_queue.cc:1048] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.352185 12774 consensus_queue.cc:1048] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.369607 12774 consensus_queue.cc:1048] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.379499 12820 raft_consensus.cc:993] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:39.380606 12774 raft_consensus.cc:1081] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:39.382782 12647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:52214
I20260709 13:06:39.383502 12647 raft_consensus.cc:493] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:39.383970 12647 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.391407 12647 raft_consensus.cc:515] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.393498 12647 leader_election.cc:290] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.394428 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:39.394876 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:39.395148 12573 raft_consensus.cc:3055] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:39.395653 12573 raft_consensus.cc:740] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:39.395886 12723 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.396736 12573 consensus_queue.cc:260] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.398273 12573 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.406252 12723 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:39.407913 12612 leader_election.cc:304] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:39.408093 12573 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:39.409253 12803 raft_consensus.cc:2804] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:39.410073 12803 raft_consensus.cc:697] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:39.411188 12803 consensus_queue.cc:237] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.420588 12472 catalog_manager.cc:5697] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.461625 12774 consensus_queue.cc:1048] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:39.476368 12820 consensus_queue.cc:1048] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.525146 12820 consensus_queue.cc:1048] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:39.540498 12843 consensus_queue.cc:1048] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:39.559898 12781 raft_consensus.cc:993] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: : Instructing follower b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:39.560547 12789 raft_consensus.cc:1081] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Signalling peer b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:39.562345 12573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:47686
I20260709 13:06:39.562994 12573 raft_consensus.cc:493] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:39.563403 12573 raft_consensus.cc:3060] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.570902 12573 raft_consensus.cc:515] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.573305 12573 leader_election.cc:290] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.574522 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:39.574862 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8a84ad985aa4f64a366695c31ee67e3" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:39.575488 12647 raft_consensus.cc:3060] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.575542 12723 raft_consensus.cc:3055] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:39.576185 12723 raft_consensus.cc:740] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:39.577258 12723 consensus_queue.cc:260] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.578918 12723 raft_consensus.cc:3060] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.582434 12647 raft_consensus.cc:2468] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:39.584069 12538 leader_election.cc:304] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:39.585093 12843 raft_consensus.cc:2804] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:39.585512 12723 raft_consensus.cc:2468] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:39.585564 12843 raft_consensus.cc:697] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:39.586776 12843 consensus_queue.cc:237] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.594697 12472 catalog_manager.cc:5697] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 1 to 2, leader changed from 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195) to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.684255 12781 raft_consensus.cc:993] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: : Instructing follower 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:39.684975 12789 raft_consensus.cc:1081] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Signalling peer 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:39.687177 12647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:52202
I20260709 13:06:39.688297 12647 raft_consensus.cc:493] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:39.688742 12647 raft_consensus.cc:3060] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.695796 12647 raft_consensus.cc:515] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.699091 12647 leader_election.cc:290] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:39.700101 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:39.700328 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a9074ed8cc4d38b77ec07e44134c6e" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:39.701049 12573 raft_consensus.cc:3060] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.701104 12723 raft_consensus.cc:3055] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:39.701586 12723 raft_consensus.cc:740] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:39.702363 12723 consensus_queue.cc:260] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.703490 12723 raft_consensus.cc:3060] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:39.707930 12573 raft_consensus.cc:2468] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:39.710078 12615 leader_election.cc:304] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:39.711146 12806 raft_consensus.cc:2804] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:39.711371 12723 raft_consensus.cc:2468] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:39.711755 12806 raft_consensus.cc:697] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:39.712764 12806 consensus_queue.cc:237] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:39.721396 12472 catalog_manager.cc:5697] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 1 to 2, leader changed from 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195) to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.817870 12573 raft_consensus.cc:1275] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:39.819968 12806 consensus_queue.cc:1048] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:39.831459 12722 raft_consensus.cc:1275] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:39.834149 12802 consensus_queue.cc:1048] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:40.106761 12722 raft_consensus.cc:1275] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.108594 12806 consensus_queue.cc:1048] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:40.118151 12573 raft_consensus.cc:1275] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.120791 12803 consensus_queue.cc:1048] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:40.149399 12722 raft_consensus.cc:1275] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.152109 12818 consensus_queue.cc:1048] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:40.163347 12647 raft_consensus.cc:1275] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:40.166095 12818 consensus_queue.cc:1048] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:42.336625 12471 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56868:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:42.339195 12471 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:42.401409 12558 tablet_service.cc:1511] Processing CreateTablet for tablet 87635ece91c543a8b38f8723e6a7194b (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:42.402827 12638 tablet_service.cc:1511] Processing CreateTablet for tablet 87635ece91c543a8b38f8723e6a7194b (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:42.403328 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87635ece91c543a8b38f8723e6a7194b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.404412 12638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87635ece91c543a8b38f8723e6a7194b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.404817 12708 tablet_service.cc:1511] Processing CreateTablet for tablet 87635ece91c543a8b38f8723e6a7194b (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:42.406349 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87635ece91c543a8b38f8723e6a7194b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.410789 12563 tablet_service.cc:1511] Processing CreateTablet for tablet 46a85c7b727941fd8bb6357f03cafeae (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:42.412758 12563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a85c7b727941fd8bb6357f03cafeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.414546 12636 tablet_service.cc:1511] Processing CreateTablet for tablet 46a85c7b727941fd8bb6357f03cafeae (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:42.416601 12636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a85c7b727941fd8bb6357f03cafeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.428748 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 54bbfdc11bbb4a20beaa74e510e85dbe (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:42.430675 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54bbfdc11bbb4a20beaa74e510e85dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.438198 12706 tablet_service.cc:1511] Processing CreateTablet for tablet 54bbfdc11bbb4a20beaa74e510e85dbe (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:42.440415 12706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54bbfdc11bbb4a20beaa74e510e85dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.431924 12634 tablet_service.cc:1511] Processing CreateTablet for tablet 54bbfdc11bbb4a20beaa74e510e85dbe (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:42.442029 12634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54bbfdc11bbb4a20beaa74e510e85dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.428781 12710 tablet_service.cc:1511] Processing CreateTablet for tablet 46a85c7b727941fd8bb6357f03cafeae (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:42.444618 12867 tablet_bootstrap.cc:492] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:42.445659 12710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46a85c7b727941fd8bb6357f03cafeae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.462459 12868 tablet_bootstrap.cc:492] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:42.470817 12867 tablet_bootstrap.cc:654] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.474743 12631 tablet_service.cc:1511] Processing CreateTablet for tablet b67cd27f5e834e57928e85466f2d9967 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:42.476594 12631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67cd27f5e834e57928e85466f2d9967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.473704 12558 tablet_service.cc:1511] Processing CreateTablet for tablet b67cd27f5e834e57928e85466f2d9967 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:42.485447 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67cd27f5e834e57928e85466f2d9967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.495273 12709 tablet_service.cc:1511] Processing CreateTablet for tablet b67cd27f5e834e57928e85466f2d9967 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:42.503273 12709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67cd27f5e834e57928e85466f2d9967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.511893 12870 tablet_bootstrap.cc:492] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.514529 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3643c391ca4f0ba7e00eeca2ab243e (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:42.516283 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3643c391ca4f0ba7e00eeca2ab243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.528877 12636 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3643c391ca4f0ba7e00eeca2ab243e (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:42.530965 12636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3643c391ca4f0ba7e00eeca2ab243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.531890 12868 tablet_bootstrap.cc:654] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.537593 12710 tablet_service.cc:1511] Processing CreateTablet for tablet 2c3643c391ca4f0ba7e00eeca2ab243e (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:42.537945 12563 tablet_service.cc:1511] Processing CreateTablet for tablet 02b24b9c71614b81a916535be4665a6f (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:42.539389 12563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b24b9c71614b81a916535be4665a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.547231 12867 tablet_bootstrap.cc:492] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:42.548135 12867 ts_tablet_manager.cc:1403] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.104s	user 0.031s	sys 0.016s
I20260709 13:06:42.539357 12710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c3643c391ca4f0ba7e00eeca2ab243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.558723 12867 raft_consensus.cc:359] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.559741 12867 raft_consensus.cc:385] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.560182 12867 raft_consensus.cc:740] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.562740 12634 tablet_service.cc:1511] Processing CreateTablet for tablet 02b24b9c71614b81a916535be4665a6f (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:42.564380 12634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b24b9c71614b81a916535be4665a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.566334 12558 tablet_service.cc:1511] Processing CreateTablet for tablet c732de1b892946dc9d00762ccb69c3db (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:42.567353 12706 tablet_service.cc:1511] Processing CreateTablet for tablet 02b24b9c71614b81a916535be4665a6f (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:42.568148 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c732de1b892946dc9d00762ccb69c3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.571367 12867 consensus_queue.cc:260] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.574730 12708 tablet_service.cc:1511] Processing CreateTablet for tablet c732de1b892946dc9d00762ccb69c3db (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:42.576450 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c732de1b892946dc9d00762ccb69c3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.577590 12867 ts_tablet_manager.cc:1434] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:06:42.578888 12867 tablet_bootstrap.cc:492] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:42.572999 12631 tablet_service.cc:1511] Processing CreateTablet for tablet c732de1b892946dc9d00762ccb69c3db (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:42.586186 12631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c732de1b892946dc9d00762ccb69c3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.574704 12870 tablet_bootstrap.cc:654] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.586272 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 8a789d9dd6544615a3a5cb3c04876322 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:42.591231 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a789d9dd6544615a3a5cb3c04876322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.601333 12867 tablet_bootstrap.cc:654] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.606035 12711 tablet_service.cc:1511] Processing CreateTablet for tablet 8a789d9dd6544615a3a5cb3c04876322 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:42.611102 12713 tablet_service.cc:1511] Processing CreateTablet for tablet 090b0397d579459bb29962b23fe60a24 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:42.612510 12711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a789d9dd6544615a3a5cb3c04876322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.613338 12713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 090b0397d579459bb29962b23fe60a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.606035 12563 tablet_service.cc:1511] Processing CreateTablet for tablet 090b0397d579459bb29962b23fe60a24 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:42.615549 12563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 090b0397d579459bb29962b23fe60a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.615692 12638 tablet_service.cc:1511] Processing CreateTablet for tablet 090b0397d579459bb29962b23fe60a24 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:42.617244 12638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 090b0397d579459bb29962b23fe60a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.611205 12636 tablet_service.cc:1511] Processing CreateTablet for tablet 8a789d9dd6544615a3a5cb3c04876322 (DEFAULT_TABLE table=soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:42.626690 12636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a789d9dd6544615a3a5cb3c04876322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.629112 12855 raft_consensus.cc:493] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.629983 12855 raft_consensus.cc:515] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.633934 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:42.634119 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
W20260709 13:06:42.636092 12537 leader_election.cc:343] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:42.636169 12855 leader_election.cc:290] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
W20260709 13:06:42.637055 12538 leader_election.cc:343] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:42.645792 12538 leader_election.cc:304] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8c676f2c2264c40b7833e26cf972b7e; no voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:42.646270 12706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02b24b9c71614b81a916535be4665a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.646916 12855 raft_consensus.cc:2749] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:42.653466 12868 tablet_bootstrap.cc:492] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:42.654243 12868 ts_tablet_manager.cc:1403] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.192s	user 0.003s	sys 0.014s
I20260709 13:06:42.695503 12867 tablet_bootstrap.cc:492] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:42.696277 12867 ts_tablet_manager.cc:1403] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.118s	user 0.015s	sys 0.008s
I20260709 13:06:42.697893 12868 raft_consensus.cc:359] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.698833 12868 raft_consensus.cc:385] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.699120 12868 raft_consensus.cc:740] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.699971 12868 consensus_queue.cc:260] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.700209 12867 raft_consensus.cc:359] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.701318 12867 raft_consensus.cc:385] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.701776 12867 raft_consensus.cc:740] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.702663 12867 consensus_queue.cc:260] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.705192 12867 ts_tablet_manager.cc:1434] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:42.707659 12870 tablet_bootstrap.cc:492] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:42.708480 12870 ts_tablet_manager.cc:1403] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.197s	user 0.013s	sys 0.024s
I20260709 13:06:42.710749 12868 ts_tablet_manager.cc:1434] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 13:06:42.711953 12870 raft_consensus.cc:359] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.712886 12870 raft_consensus.cc:385] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.713248 12870 raft_consensus.cc:740] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.714211 12870 consensus_queue.cc:260] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.716993 12870 ts_tablet_manager.cc:1434] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:06:42.718112 12870 tablet_bootstrap.cc:492] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.712025 12868 tablet_bootstrap.cc:492] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:42.741353 12870 tablet_bootstrap.cc:654] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.742965 12867 tablet_bootstrap.cc:492] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:42.756201 12870 tablet_bootstrap.cc:492] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:42.757000 12870 ts_tablet_manager.cc:1403] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.014s
I20260709 13:06:42.760687 12870 raft_consensus.cc:359] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.761400 12870 raft_consensus.cc:385] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.761686 12870 raft_consensus.cc:740] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.762986 12868 tablet_bootstrap.cc:654] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.762563 12870 consensus_queue.cc:260] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.765578 12870 ts_tablet_manager.cc:1434] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:06:42.773931 12867 tablet_bootstrap.cc:654] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.778244 12870 tablet_bootstrap.cc:492] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.779037 12868 tablet_bootstrap.cc:492] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:42.779642 12868 ts_tablet_manager.cc:1403] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.068s	user 0.009s	sys 0.008s
I20260709 13:06:42.783317 12868 raft_consensus.cc:359] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.784229 12868 raft_consensus.cc:385] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.784713 12868 raft_consensus.cc:740] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.785712 12868 consensus_queue.cc:260] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.791536 12870 tablet_bootstrap.cc:654] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.800786 12868 ts_tablet_manager.cc:1434] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.007s
I20260709 13:06:42.802330 12868 tablet_bootstrap.cc:492] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:42.806144 12870 tablet_bootstrap.cc:492] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:42.806445 12867 tablet_bootstrap.cc:492] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:42.807006 12870 ts_tablet_manager.cc:1403] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.018s
I20260709 13:06:42.807001 12867 ts_tablet_manager.cc:1403] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.000s
I20260709 13:06:42.811595 12870 raft_consensus.cc:359] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.812776 12870 raft_consensus.cc:385] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.812469 12867 raft_consensus.cc:359] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.813305 12870 raft_consensus.cc:740] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.813715 12867 raft_consensus.cc:385] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.813867 12868 tablet_bootstrap.cc:654] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.814355 12867 raft_consensus.cc:740] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.814500 12870 consensus_queue.cc:260] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.815311 12867 consensus_queue.cc:260] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.817656 12870 ts_tablet_manager.cc:1434] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:42.819013 12870 tablet_bootstrap.cc:492] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.818888 12867 ts_tablet_manager.cc:1434] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:06:42.821354 12867 tablet_bootstrap.cc:492] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:42.828660 12870 tablet_bootstrap.cc:654] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.835722 12867 tablet_bootstrap.cc:654] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.836326 12868 tablet_bootstrap.cc:492] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:42.837030 12868 ts_tablet_manager.cc:1403] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.008s
I20260709 13:06:42.842015 12868 raft_consensus.cc:359] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.843429 12868 raft_consensus.cc:385] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.843739 12870 tablet_bootstrap.cc:492] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:42.844619 12870 ts_tablet_manager.cc:1403] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 13:06:42.843899 12868 raft_consensus.cc:740] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.847862 12867 tablet_bootstrap.cc:492] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:42.848647 12867 ts_tablet_manager.cc:1403] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.011s
I20260709 13:06:42.848421 12870 raft_consensus.cc:359] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.846959 12868 consensus_queue.cc:260] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.849541 12870 raft_consensus.cc:385] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.849910 12870 raft_consensus.cc:740] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.850819 12870 consensus_queue.cc:260] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.852218 12868 ts_tablet_manager.cc:1434] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 13:06:42.853781 12870 ts_tablet_manager.cc:1434] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:06:42.854693 12868 tablet_bootstrap.cc:492] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:42.855168 12870 tablet_bootstrap.cc:492] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.856848 12867 raft_consensus.cc:359] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.857824 12867 raft_consensus.cc:385] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.858206 12867 raft_consensus.cc:740] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.859402 12867 consensus_queue.cc:260] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.864121 12870 tablet_bootstrap.cc:654] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.865901 12867 ts_tablet_manager.cc:1434] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.017s	user 0.000s	sys 0.007s
I20260709 13:06:42.867481 12867 tablet_bootstrap.cc:492] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:42.868779 12868 tablet_bootstrap.cc:654] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.877732 12875 raft_consensus.cc:493] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.878471 12875 raft_consensus.cc:515] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.884092 12867 tablet_bootstrap.cc:654] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.886894 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:42.888051 12573 raft_consensus.cc:2468] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:42.892131 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:42.893684 12868 tablet_bootstrap.cc:492] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:42.893693 12867 tablet_bootstrap.cc:492] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:42.894618 12868 ts_tablet_manager.cc:1403] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.025s
I20260709 13:06:42.894750 12867 ts_tablet_manager.cc:1403] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 13:06:42.895967 12689 leader_election.cc:304] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:42.897924 12789 raft_consensus.cc:2804] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:42.898495 12789 raft_consensus.cc:493] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.898941 12789 raft_consensus.cc:3060] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:42.899147 12868 raft_consensus.cc:359] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.900175 12868 raft_consensus.cc:385] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.900527 12868 raft_consensus.cc:740] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.898972 12867 raft_consensus.cc:359] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.901399 12868 consensus_queue.cc:260] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.902163 12867 raft_consensus.cc:385] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.903043 12867 raft_consensus.cc:740] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.904117 12868 ts_tablet_manager.cc:1434] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:06:42.903923 12867 consensus_queue.cc:260] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.905145 12875 leader_election.cc:290] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:42.905711 12868 tablet_bootstrap.cc:492] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:42.906456 12867 ts_tablet_manager.cc:1434] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:06:42.893711 12647 raft_consensus.cc:2468] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:42.908720 12867 tablet_bootstrap.cc:492] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:42.914773 12870 tablet_bootstrap.cc:492] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:42.915609 12870 ts_tablet_manager.cc:1403] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.026s
I20260709 13:06:42.916762 12868 tablet_bootstrap.cc:654] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.917450 12867 tablet_bootstrap.cc:654] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.919624 12870 raft_consensus.cc:359] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.921136 12870 raft_consensus.cc:385] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.921550 12870 raft_consensus.cc:740] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.922513 12870 consensus_queue.cc:260] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.925417 12870 ts_tablet_manager.cc:1434] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:42.926741 12870 tablet_bootstrap.cc:492] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.936498 12870 tablet_bootstrap.cc:654] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:42.949631 12870 tablet_bootstrap.cc:492] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:42.950387 12870 ts_tablet_manager.cc:1403] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.010s
I20260709 13:06:42.928020 12789 raft_consensus.cc:515] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.960076 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:42.960428 12870 raft_consensus.cc:359] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.961035 12573 raft_consensus.cc:3060] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:42.961671 12870 raft_consensus.cc:385] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:42.962059 12870 raft_consensus.cc:740] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:42.963312 12870 consensus_queue.cc:260] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.966234 12870 ts_tablet_manager.cc:1434] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.007s
I20260709 13:06:42.967654 12870 tablet_bootstrap.cc:492] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:42.969008 12855 raft_consensus.cc:493] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.969717 12855 raft_consensus.cc:515] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:42.974457 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:42.975281 12722 raft_consensus.cc:2468] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:42.976509 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:42.977344 12647 raft_consensus.cc:2468] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:42.977972 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:42.978691 12648 raft_consensus.cc:3060] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:42.979148 12538 leader_election.cc:304] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:42.981768 12855 leader_election.cc:290] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:42.982479 12855 raft_consensus.cc:2804] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:42.983081 12855 raft_consensus.cc:493] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.983506 12855 raft_consensus.cc:3060] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:42.990602 12573 raft_consensus.cc:2468] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:42.997880 12689 leader_election.cc:304] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:42.999274 12875 raft_consensus.cc:2804] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:42.991803 12789 leader_election.cc:290] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.005934 12648 raft_consensus.cc:2468] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:43.007282 12877 raft_consensus.cc:493] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.011255 12877 raft_consensus.cc:515] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.012632 12867 tablet_bootstrap.cc:492] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:43.013592 12867 ts_tablet_manager.cc:1403] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.105s	user 0.039s	sys 0.042s
I20260709 13:06:43.015429 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.015786 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:43.016327 12574 raft_consensus.cc:2468] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 0.
I20260709 13:06:43.016474 12722 raft_consensus.cc:2468] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 0.
I20260709 13:06:43.017850 12615 leader_election.cc:304] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.018375 12877 leader_election.cc:290] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.019052 12803 raft_consensus.cc:2804] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.019449 12803 raft_consensus.cc:493] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.019256 12867 raft_consensus.cc:359] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.013522 12868 tablet_bootstrap.cc:492] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:43.019992 12803 raft_consensus.cc:3060] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.020831 12867 raft_consensus.cc:385] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.021250 12870 tablet_bootstrap.cc:654] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.021281 12867 raft_consensus.cc:740] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.022759 12867 consensus_queue.cc:260] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.023602 12875 raft_consensus.cc:697] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:43.024837 12875 consensus_queue.cc:237] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.030743 12868 ts_tablet_manager.cc:1403] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.125s	user 0.052s	sys 0.043s
I20260709 13:06:43.034849 12868 raft_consensus.cc:359] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.035943 12868 raft_consensus.cc:385] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.036358 12868 raft_consensus.cc:740] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.037287 12868 consensus_queue.cc:260] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.042333 12803 raft_consensus.cc:515] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.043694 12855 raft_consensus.cc:515] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.047183 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:43.047516 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.048059 12574 raft_consensus.cc:3060] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.048563 12648 raft_consensus.cc:3060] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.049160 12803 leader_election.cc:290] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.050967 12868 ts_tablet_manager.cc:1434] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.002s
I20260709 13:06:43.052626 12868 tablet_bootstrap.cc:492] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:43.052800 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.053615 12722 raft_consensus.cc:3060] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.058960 12574 raft_consensus.cc:2468] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:43.060815 12615 leader_election.cc:304] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.061328 12868 tablet_bootstrap.cc:654] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.062096 12803 raft_consensus.cc:2804] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.062806 12803 raft_consensus.cc:697] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:43.063252 12648 raft_consensus.cc:2468] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.063867 12803 consensus_queue.cc:237] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.065376 12538 leader_election.cc:304] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.076961 12875 raft_consensus.cc:493] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.078173 12892 raft_consensus.cc:493] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.077986 12875 raft_consensus.cc:515] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.081029 12855 leader_election.cc:290] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.083079 12875 leader_election.cc:290] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.084522 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.085239 12855 raft_consensus.cc:2804] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.085454 12574 raft_consensus.cc:2468] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:43.087937 12689 leader_election.cc:304] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.089888 12875 raft_consensus.cc:2804] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.090423 12875 raft_consensus.cc:493] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.090427 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.091423 12875 raft_consensus.cc:3060] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:43.093601 12537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34547, user_credentials={real_user=slave}} blocked reactor thread for 63730.9us
I20260709 13:06:43.085902 12855 raft_consensus.cc:697] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:43.096176 12855 consensus_queue.cc:237] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.102761 12868 tablet_bootstrap.cc:492] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:43.103721 12868 ts_tablet_manager.cc:1403] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.002s
I20260709 13:06:43.104799 12870 tablet_bootstrap.cc:492] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:43.106164 12870 ts_tablet_manager.cc:1403] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.139s	user 0.014s	sys 0.002s
I20260709 13:06:43.107345 12867 ts_tablet_manager.cc:1434] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.093s	user 0.016s	sys 0.024s
I20260709 13:06:43.109323 12868 raft_consensus.cc:359] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.110379 12868 raft_consensus.cc:385] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.109503 12867 tablet_bootstrap.cc:492] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:43.110929 12868 raft_consensus.cc:740] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.111574 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.112622 12723 raft_consensus.cc:3060] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.112134 12868 consensus_queue.cc:260] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.115619 12868 ts_tablet_manager.cc:1434] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.006s
I20260709 13:06:43.115159 12870 raft_consensus.cc:359] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.118011 12870 raft_consensus.cc:385] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.119102 12870 raft_consensus.cc:740] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.122207 12870 consensus_queue.cc:260] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.127188 12868 tablet_bootstrap.cc:492] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:43.121227 12867 tablet_bootstrap.cc:654] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.091893 12648 raft_consensus.cc:2468] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
W20260709 13:06:43.132620 12538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37275, user_credentials={real_user=slave}} blocked reactor thread for 67333.9us
I20260709 13:06:43.136754 12870 ts_tablet_manager.cc:1434] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.005s
W20260709 13:06:43.144840 12518 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:43.140705 12892 raft_consensus.cc:515] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.151289 12875 raft_consensus.cc:515] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.153349 12722 raft_consensus.cc:2468] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:43.155565 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:43.156476 12574 raft_consensus.cc:3060] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.157392 12892 leader_election.cc:290] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.127877 12723 raft_consensus.cc:2468] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.161379 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.162262 12648 raft_consensus.cc:2468] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:43.164014 12868 tablet_bootstrap.cc:654] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.168514 12538 leader_election.cc:304] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.170177 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.171077 12648 raft_consensus.cc:3060] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.173457 12875 leader_election.cc:290] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.175632 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:43.176366 12855 raft_consensus.cc:2804] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.177621 12855 raft_consensus.cc:493] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.178084 12855 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.181420 12472 catalog_manager.cc:5697] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.185899 12648 raft_consensus.cc:2468] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
W20260709 13:06:43.187240 12537 leader_election.cc:343] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:43.187913 12687 leader_election.cc:304] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:43.189926 12890 raft_consensus.cc:2804] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.190352 12574 raft_consensus.cc:2468] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 1.
I20260709 13:06:43.190447 12890 raft_consensus.cc:697] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:43.190503 12855 raft_consensus.cc:515] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.195772 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.196542 12648 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.197047 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.197368 12870 tablet_bootstrap.cc:492] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
W20260709 13:06:43.199414 12537 leader_election.cc:343] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:43.200101 12855 leader_election.cc:290] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.201617 12471 catalog_manager.cc:5697] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.191684 12890 consensus_queue.cc:237] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.204131 12470 catalog_manager.cc:5697] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:43.215690 12648 raft_consensus.cc:2468] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.216511 12789 raft_consensus.cc:493] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.217088 12789 raft_consensus.cc:515] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.217927 12538 leader_election.cc:304] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:43.219743 12894 raft_consensus.cc:2804] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.220182 12894 raft_consensus.cc:697] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:43.220633 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.221458 12894 consensus_queue.cc:237] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
W20260709 13:06:43.222426 12689 leader_election.cc:343] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:43.222529 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.221060 12789 leader_election.cc:290] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.220678 12870 tablet_bootstrap.cc:654] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:43.224246 12687 leader_election.cc:343] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:43.224970 12687 leader_election.cc:304] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.226827 12867 tablet_bootstrap.cc:492] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:43.227169 12789 raft_consensus.cc:2749] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:43.227787 12867 ts_tablet_manager.cc:1403] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.119s	user 0.028s	sys 0.008s
I20260709 13:06:43.229703 12868 tablet_bootstrap.cc:492] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:43.230405 12868 ts_tablet_manager.cc:1403] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.104s	user 0.018s	sys 0.009s
I20260709 13:06:43.235129 12867 raft_consensus.cc:359] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.236003 12867 raft_consensus.cc:385] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.235270 12868 raft_consensus.cc:359] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.236706 12868 raft_consensus.cc:385] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.237318 12868 raft_consensus.cc:740] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.236649 12867 raft_consensus.cc:740] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.238498 12868 consensus_queue.cc:260] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.239761 12867 consensus_queue.cc:260] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.242132 12868 ts_tablet_manager.cc:1434] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:06:43.243234 12867 ts_tablet_manager.cc:1434] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:06:43.244082 12868 tablet_bootstrap.cc:492] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:43.244728 12867 tablet_bootstrap.cc:492] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:43.254894 12868 tablet_bootstrap.cc:654] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.260581 12867 tablet_bootstrap.cc:654] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.269268 12894 raft_consensus.cc:493] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.270143 12894 raft_consensus.cc:515] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.273823 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.274979 12648 raft_consensus.cc:2468] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:43.275089 12894 leader_election.cc:290] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.274982 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
W20260709 13:06:43.277973 12537 leader_election.cc:343] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:43.279330 12538 leader_election.cc:304] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:43.280882 12894 raft_consensus.cc:2804] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.281330 12894 raft_consensus.cc:493] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.281693 12894 raft_consensus.cc:3060] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.291972 12894 raft_consensus.cc:515] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.287611 12472 catalog_manager.cc:5697] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:43.296218 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.297271 12648 raft_consensus.cc:3060] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.298163 12870 tablet_bootstrap.cc:492] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:43.301453 12870 ts_tablet_manager.cc:1403] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.104s	user 0.023s	sys 0.010s
I20260709 13:06:43.306931 12648 raft_consensus.cc:2468] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.308976 12538 leader_election.cc:304] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.310045 12855 raft_consensus.cc:2804] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.310602 12855 raft_consensus.cc:697] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:43.311640 12855 consensus_queue.cc:237] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.307304 12870 raft_consensus.cc:359] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.314970 12870 raft_consensus.cc:385] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.316294 12870 raft_consensus.cc:740] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.321928 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.318271 12894 leader_election.cc:290] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
W20260709 13:06:43.324221 12537 leader_election.cc:343] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:43.325182 12867 tablet_bootstrap.cc:492] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:43.325129 12868 tablet_bootstrap.cc:492] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:43.325472 12870 consensus_queue.cc:260] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.326320 12867 ts_tablet_manager.cc:1403] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.082s	user 0.040s	sys 0.032s
I20260709 13:06:43.326588 12868 ts_tablet_manager.cc:1403] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.083s	user 0.022s	sys 0.011s
I20260709 13:06:43.326838 12471 catalog_manager.cc:5697] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.329205 12870 ts_tablet_manager.cc:1434] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.005s
I20260709 13:06:43.330572 12870 tablet_bootstrap.cc:492] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: Bootstrap starting.
I20260709 13:06:43.330986 12867 raft_consensus.cc:359] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.331969 12867 raft_consensus.cc:385] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.332305 12867 raft_consensus.cc:740] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.331138 12868 raft_consensus.cc:359] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.334349 12868 raft_consensus.cc:385] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.333479 12867 consensus_queue.cc:260] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.334793 12868 raft_consensus.cc:740] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.336297 12868 consensus_queue.cc:260] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.337231 12867 ts_tablet_manager.cc:1434] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:06:43.338440 12867 tablet_bootstrap.cc:492] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: Bootstrap starting.
I20260709 13:06:43.338521 12870 tablet_bootstrap.cc:654] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.346807 12867 tablet_bootstrap.cc:654] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.343586 12868 ts_tablet_manager.cc:1434] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:06:43.349823 12868 tablet_bootstrap.cc:492] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: Bootstrap starting.
I20260709 13:06:43.353914 12870 tablet_bootstrap.cc:492] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: No bootstrap required, opened a new log
I20260709 13:06:43.355005 12870 ts_tablet_manager.cc:1403] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260709 13:06:43.357089 12868 tablet_bootstrap.cc:654] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:43.359567 12870 raft_consensus.cc:359] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.360837 12870 raft_consensus.cc:385] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.361271 12870 raft_consensus.cc:740] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.362618 12870 consensus_queue.cc:260] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.367714 12870 ts_tablet_manager.cc:1434] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:06:43.368999 12890 raft_consensus.cc:493] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.369894 12890 raft_consensus.cc:515] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.373180 12894 raft_consensus.cc:493] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.373993 12894 raft_consensus.cc:515] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.377341 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.377341 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.378271 12648 raft_consensus.cc:2468] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:43.378715 12574 raft_consensus.cc:2468] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:43.379982 12538 leader_election.cc:304] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.380694 12689 leader_election.cc:304] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.381115 12855 raft_consensus.cc:2804] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.381559 12855 raft_consensus.cc:493] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.381716 12789 raft_consensus.cc:2804] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.381989 12855 raft_consensus.cc:3060] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.382169 12789 raft_consensus.cc:493] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.382632 12789 raft_consensus.cc:3060] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.387157 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:43.392680 12894 leader_election.cc:290] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.393558 12867 tablet_bootstrap.cc:492] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: No bootstrap required, opened a new log
I20260709 13:06:43.394476 12867 ts_tablet_manager.cc:1403] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: Time spent bootstrapping tablet: real 0.056s	user 0.030s	sys 0.017s
I20260709 13:06:43.397810 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.397996 12855 raft_consensus.cc:515] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.398829 12648 raft_consensus.cc:2468] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 0.
I20260709 13:06:43.400112 12867 raft_consensus.cc:359] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.401233 12867 raft_consensus.cc:385] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.401451 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.401636 12867 raft_consensus.cc:740] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.400777 12890 leader_election.cc:290] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.402207 12648 raft_consensus.cc:3060] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.402781 12867 consensus_queue.cc:260] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.405238 12867 ts_tablet_manager.cc:1434] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 13:06:43.406939 12471 catalog_manager.cc:5697] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.409015 12789 raft_consensus.cc:515] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.410477 12722 raft_consensus.cc:2393] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8c676f2c2264c40b7833e26cf972b7e in current term 1: Already voted for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in this term.
I20260709 13:06:43.412256 12868 tablet_bootstrap.cc:492] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: No bootstrap required, opened a new log
I20260709 13:06:43.413913 12868 ts_tablet_manager.cc:1403] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.008s
I20260709 13:06:43.412400 12648 raft_consensus.cc:2468] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.416023 12538 leader_election.cc:304] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.419611 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.416249 12855 leader_election.cc:290] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.421227 12855 raft_consensus.cc:2804] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.421754 12855 raft_consensus.cc:697] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:43.419986 12868 raft_consensus.cc:359] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.422679 12855 consensus_queue.cc:237] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.423280 12868 raft_consensus.cc:385] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:43.423755 12868 raft_consensus.cc:740] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Initialized, Role: FOLLOWER
I20260709 13:06:43.424611 12868 consensus_queue.cc:260] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.427469 12868 ts_tablet_manager.cc:1434] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:06:43.431437 12789 leader_election.cc:290] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.431792 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:43.435940 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.437752 12648 raft_consensus.cc:2393] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8da3d1ff50e74afbad7414d7e80b9b34 in current term 1: Already voted for candidate b8c676f2c2264c40b7833e26cf972b7e in this term.
I20260709 13:06:43.439033 12687 leader_election.cc:304] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.440671 12789 raft_consensus.cc:2749] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:43.445781 12789 raft_consensus.cc:493] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.446559 12789 raft_consensus.cc:515] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.446657 12471 catalog_manager.cc:5697] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.448861 12789 leader_election.cc:290] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.450317 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.450299 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.451479 12877 raft_consensus.cc:493] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.451841 12648 raft_consensus.cc:2393] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8da3d1ff50e74afbad7414d7e80b9b34 in current term 1: Already voted for candidate b8c676f2c2264c40b7833e26cf972b7e in this term.
I20260709 13:06:43.452365 12877 raft_consensus.cc:515] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.454797 12689 leader_election.cc:304] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.455084 12877 leader_election.cc:290] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.455710 12789 raft_consensus.cc:3060] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.457955 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c732de1b892946dc9d00762ccb69c3db" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:43.459146 12723 raft_consensus.cc:2468] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 0.
I20260709 13:06:43.458099 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c732de1b892946dc9d00762ccb69c3db" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.460619 12612 leader_election.cc:304] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:43.461035 12574 raft_consensus.cc:2468] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 0.
I20260709 13:06:43.461594 12877 raft_consensus.cc:2804] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.462020 12877 raft_consensus.cc:493] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.462363 12877 raft_consensus.cc:3060] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.467702 12789 raft_consensus.cc:2749] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:43.471343 12877 raft_consensus.cc:515] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.473300 12877 leader_election.cc:290] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.474572 12574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c732de1b892946dc9d00762ccb69c3db" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:43.474761 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c732de1b892946dc9d00762ccb69c3db" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.475589 12723 raft_consensus.cc:3060] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.475665 12574 raft_consensus.cc:3060] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.478883 12789 raft_consensus.cc:493] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.479511 12789 raft_consensus.cc:515] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.482450 12789 leader_election.cc:290] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:43.483085 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e" is_pre_election: true
I20260709 13:06:43.483101 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.483883 12574 raft_consensus.cc:2468] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:43.484201 12648 raft_consensus.cc:2393] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8da3d1ff50e74afbad7414d7e80b9b34 in current term 1: Already voted for candidate b8c676f2c2264c40b7833e26cf972b7e in this term.
I20260709 13:06:43.485423 12723 raft_consensus.cc:2468] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 1.
I20260709 13:06:43.485524 12615 leader_election.cc:304] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.485483 12687 leader_election.cc:304] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.486894 12877 raft_consensus.cc:2804] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.487226 12789 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.487473 12877 raft_consensus.cc:697] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:43.488653 12877 consensus_queue.cc:237] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.497326 12789 raft_consensus.cc:2749] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:43.499465 12471 catalog_manager.cc:5697] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 1 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.516681 12877 consensus_queue.cc:1048] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.516681 12855 consensus_queue.cc:1048] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.539558 12894 consensus_queue.cc:1048] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.549273 12894 raft_consensus.cc:493] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.550146 12894 raft_consensus.cc:515] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.554730 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c" is_pre_election: true
I20260709 13:06:43.555020 12894 leader_election.cc:290] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.555322 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" is_pre_election: true
I20260709 13:06:43.555562 12647 raft_consensus.cc:2468] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:43.556110 12722 raft_consensus.cc:2468] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 0.
I20260709 13:06:43.557235 12538 leader_election.cc:304] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.558403 12877 consensus_queue.cc:1048] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:43.559229 12894 raft_consensus.cc:2804] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:43.559725 12894 raft_consensus.cc:493] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.560123 12894 raft_consensus.cc:3060] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.581498 12890 consensus_queue.cc:1048] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.582590 12894 raft_consensus.cc:515] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.587487 12649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.588577 12649 raft_consensus.cc:3060] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.596918 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.597800 12723 raft_consensus.cc:3060] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:43.600502 12894 leader_election.cc:290] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.602667 12649 raft_consensus.cc:2468] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.604467 12538 leader_election.cc:304] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.605720 12894 raft_consensus.cc:2804] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:43.607489 12894 raft_consensus.cc:697] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:43.607925 12723 raft_consensus.cc:2468] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 1.
I20260709 13:06:43.609170 12890 consensus_queue.cc:1048] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.608664 12894 consensus_queue.cc:237] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.625998 12890 consensus_queue.cc:1048] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.640407 12898 consensus_queue.cc:1048] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.650935 12890 consensus_queue.cc:1048] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.663221 12470 catalog_manager.cc:5697] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 1 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.691883 12855 consensus_queue.cc:1048] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.899967 12803 consensus_queue.cc:1048] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.914018 12647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
mode: GRACEFUL
new_leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:40524
I20260709 13:06:43.914439 12877 consensus_queue.cc:1048] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:43.914775 12647 raft_consensus.cc:606] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Received request to transfer leadership to TS b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.926975 12938 raft_consensus.cc:993] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: : Instructing follower b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:43.928740 12938 raft_consensus.cc:1081] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Signalling peer b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:43.927241 12721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
mode: GRACEFUL
new_leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:41738
I20260709 13:06:43.931020 12721 raft_consensus.cc:606] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Received request to transfer leadership to TS b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.933921 12574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:47688
I20260709 13:06:43.934691 12574 raft_consensus.cc:493] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:43.937376 12574 raft_consensus.cc:3060] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:43.938911 12721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
mode: GRACEFUL
new_leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:41738
I20260709 13:06:43.939713 12721 raft_consensus.cc:606] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Received request to transfer leadership to TS b8c676f2c2264c40b7833e26cf972b7e
I20260709 13:06:43.944602 12855 consensus_queue.cc:1048] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.949368 12894 consensus_queue.cc:1048] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.953138 12855 consensus_queue.cc:1048] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:43.953001 12574 raft_consensus.cc:515] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.956938 12574 leader_election.cc:290] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:43.958621 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:43.959604 12723 raft_consensus.cc:3060] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:43.962559 12574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:47704
I20260709 13:06:43.963515 12574 raft_consensus.cc:606] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:43.966462 12649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87635ece91c543a8b38f8723e6a7194b" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:43.967453 12649 raft_consensus.cc:3055] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:43.967932 12649 raft_consensus.cc:740] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:43.969306 12649 consensus_queue.cc:260] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.973212 12723 raft_consensus.cc:2468] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:43.974609 12649 raft_consensus.cc:3060] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:43.974679 12537 leader_election.cc:304] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:43.975803 12894 raft_consensus.cc:2804] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:43.976537 12894 raft_consensus.cc:697] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:43.977732 12894 consensus_queue.cc:237] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:43.985378 12649 raft_consensus.cc:2468] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:43.994761 12574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:47704
I20260709 13:06:43.995604 12574 raft_consensus.cc:606] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:43.999650 12471 catalog_manager.cc:5697] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 1 to 2, leader changed from 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194) to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.021819 12574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:47704
I20260709 13:06:44.022729 12574 raft_consensus.cc:606] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 4d47e229db4042c1b76fd3976dd01e8c
I20260709 13:06:44.023567 12894 consensus_queue.cc:1048] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:44.033500 12574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "090b0397d579459bb29962b23fe60a24"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:47704
I20260709 13:06:44.034252 12574 raft_consensus.cc:606] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:44.110188 12898 consensus_queue.cc:1048] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:44.115218 12930 raft_consensus.cc:993] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: : Instructing follower b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:44.115985 12931 raft_consensus.cc:1081] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Signalling peer b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:44.118517 12574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:47686
I20260709 13:06:44.119249 12574 raft_consensus.cc:493] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.119688 12574 raft_consensus.cc:3060] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.120613 12898 consensus_queue.cc:1048] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:44.137388 12574 raft_consensus.cc:515] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.141245 12649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:44.142115 12649 raft_consensus.cc:3060] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.142159 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54bbfdc11bbb4a20beaa74e510e85dbe" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:44.142964 12723 raft_consensus.cc:3055] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.143292 12723 raft_consensus.cc:740] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:44.144161 12723 consensus_queue.cc:260] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.145344 12723 raft_consensus.cc:3060] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.154924 12649 raft_consensus.cc:2468] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:44.155256 12574 leader_election.cc:290] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:44.156965 12538 leader_election.cc:304] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:44.158457 12894 raft_consensus.cc:2804] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.159075 12894 raft_consensus.cc:697] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:44.160408 12723 raft_consensus.cc:2468] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:44.160224 12894 consensus_queue.cc:237] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.161782 12898 raft_consensus.cc:993] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:44.162348 12898 raft_consensus.cc:1081] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:44.167784 12722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "090b0397d579459bb29962b23fe60a24"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:41720
I20260709 13:06:44.170271 12722 raft_consensus.cc:493] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.170935 12722 raft_consensus.cc:3060] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.171244 12471 catalog_manager.cc:5697] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 1 to 2, leader changed from 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195) to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.179350 12722 raft_consensus.cc:515] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.185688 12789 raft_consensus.cc:993] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: : Instructing follower b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:44.186445 12789 raft_consensus.cc:1081] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Signalling peer b8c676f2c2264c40b7833e26cf972b7e to start an election
I20260709 13:06:44.189239 12574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
 from {username='slave'} at 127.0.0.1:47686
I20260709 13:06:44.190058 12574 raft_consensus.cc:493] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.190706 12574 raft_consensus.cc:3060] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.198180 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:44.198282 12722 leader_election.cc:290] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:44.199097 12573 raft_consensus.cc:3055] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.200531 12573 raft_consensus.cc:740] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:44.201406 12573 consensus_queue.cc:260] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.202298 12574 raft_consensus.cc:515] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.202656 12573 raft_consensus.cc:3060] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.204656 12574 leader_election.cc:290] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Requested vote from peers 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:44.205520 12647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:44.206503 12647 raft_consensus.cc:3060] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.199509 12649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "090b0397d579459bb29962b23fe60a24" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:44.209237 12649 raft_consensus.cc:3060] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.219285 12649 raft_consensus.cc:2468] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 2.
I20260709 13:06:44.219185 12573 raft_consensus.cc:2468] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 2.
I20260709 13:06:44.219939 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b67cd27f5e834e57928e85466f2d9967" candidate_uuid: "b8c676f2c2264c40b7833e26cf972b7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:44.219997 12647 raft_consensus.cc:2468] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:44.220952 12722 raft_consensus.cc:3055] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.221342 12722 raft_consensus.cc:740] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:44.221396 12687 leader_election.cc:304] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:44.221753 12538 leader_election.cc:304] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:44.222424 12789 raft_consensus.cc:2804] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.223019 12789 raft_consensus.cc:697] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:44.222416 12722 consensus_queue.cc:260] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.222855 12894 raft_consensus.cc:2804] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.224377 12894 raft_consensus.cc:697] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Becoming Leader. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:44.224567 12722 raft_consensus.cc:3060] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.224213 12789 consensus_queue.cc:237] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.225414 12894 consensus_queue.cc:237] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.233681 12722 raft_consensus.cc:2468] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c676f2c2264c40b7833e26cf972b7e in term 2.
I20260709 13:06:44.237798 12471 catalog_manager.cc:5697] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.237462 12469 catalog_manager.cc:5697] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e reported cstate change: term changed from 1 to 2, leader changed from 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195) to b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193). New cstate: current_term: 2 leader_uuid: "b8c676f2c2264c40b7833e26cf972b7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.278945 12855 raft_consensus.cc:993] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:44.279683 12894 raft_consensus.cc:1081] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:44.281736 12649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:52214
I20260709 13:06:44.282406 12649 raft_consensus.cc:493] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.282753 12649 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.289979 12649 raft_consensus.cc:515] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.292266 12649 leader_election.cc:290] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:44.293622 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:44.294319 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:44.295154 12722 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.294463 12573 raft_consensus.cc:3055] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.295836 12573 raft_consensus.cc:740] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:44.297091 12573 consensus_queue.cc:260] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.298648 12573 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.302698 12722 raft_consensus.cc:2468] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:44.304230 12612 leader_election.cc:304] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:44.305248 12938 raft_consensus.cc:2804] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.305349 12573 raft_consensus.cc:2468] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:44.305915 12938 raft_consensus.cc:697] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:44.306954 12938 consensus_queue.cc:237] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.316680 12471 catalog_manager.cc:5697] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.484673 12649 raft_consensus.cc:1275] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.486430 12894 consensus_queue.cc:1048] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:44.497113 12722 raft_consensus.cc:1275] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.499464 12892 consensus_queue.cc:1048] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:44.506943 12892 raft_consensus.cc:993] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:44.507524 12892 raft_consensus.cc:1081] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:44.509784 12649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:52214
I20260709 13:06:44.510622 12649 raft_consensus.cc:493] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.511034 12649 raft_consensus.cc:3060] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.517874 12649 raft_consensus.cc:515] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.520381 12649 leader_election.cc:290] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:44.520757 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:44.521036 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02b24b9c71614b81a916535be4665a6f" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:44.521611 12573 raft_consensus.cc:3055] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.521780 12722 raft_consensus.cc:3060] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.521963 12573 raft_consensus.cc:740] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:44.522876 12573 consensus_queue.cc:260] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.524066 12573 raft_consensus.cc:3060] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.531693 12722 raft_consensus.cc:2468] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:44.531693 12573 raft_consensus.cc:2468] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:44.533519 12612 leader_election.cc:304] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:44.534749 12877 raft_consensus.cc:2804] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.535301 12877 raft_consensus.cc:697] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:44.535614 12855 raft_consensus.cc:993] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:44.536201 12894 raft_consensus.cc:1081] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 4d47e229db4042c1b76fd3976dd01e8c to start an election
I20260709 13:06:44.536307 12877 consensus_queue.cc:237] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.538651 12649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
 from {username='slave'} at 127.0.0.1:52214
I20260709 13:06:44.539254 12649 raft_consensus.cc:493] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.539711 12649 raft_consensus.cc:3060] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.545316 12471 catalog_manager.cc:5697] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.548280 12649 raft_consensus.cc:515] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.550833 12649 leader_election.cc:290] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547)
I20260709 13:06:44.552238 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
I20260709 13:06:44.552384 12573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a789d9dd6544615a3a5cb3c04876322" candidate_uuid: "4d47e229db4042c1b76fd3976dd01e8c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:44.553082 12722 raft_consensus.cc:3060] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.553491 12573 raft_consensus.cc:3055] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:44.553908 12573 raft_consensus.cc:740] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:44.554744 12573 consensus_queue.cc:260] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.556224 12573 raft_consensus.cc:3060] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:44.560949 12722 raft_consensus.cc:2468] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:44.562353 12612 leader_election.cc:304] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:44.562623 12573 raft_consensus.cc:2468] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d47e229db4042c1b76fd3976dd01e8c in term 2.
I20260709 13:06:44.563393 12877 raft_consensus.cc:2804] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:44.563941 12877 raft_consensus.cc:697] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Leader. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:44.564836 12877 consensus_queue.cc:237] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:44.569319 12722 raft_consensus.cc:1275] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.570998 12894 consensus_queue.cc:1048] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:44.574242 12471 catalog_manager.cc:5697] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194). New cstate: current_term: 2 leader_uuid: "4d47e229db4042c1b76fd3976dd01e8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:44.585947 12649 raft_consensus.cc:1275] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.587915 12898 consensus_queue.cc:1048] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:44.762696 12722 raft_consensus.cc:1275] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.764853 12877 consensus_queue.cc:1048] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:44.776072 12573 raft_consensus.cc:1275] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.777788 12938 consensus_queue.cc:1048] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:44.790865 12649 raft_consensus.cc:1275] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.794476 12789 consensus_queue.cc:1048] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:44.810158 12722 raft_consensus.cc:1275] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.812129 12898 consensus_queue.cc:1048] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:44.818908 12649 raft_consensus.cc:1275] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer b8c676f2c2264c40b7833e26cf972b7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.821487 12898 consensus_queue.cc:1048] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:44.829347 12573 raft_consensus.cc:1275] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:44.832952 12950 consensus_queue.cc:1048] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:45.051824 12723 raft_consensus.cc:1275] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.053371 12938 consensus_queue.cc:1048] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.067023 12573 raft_consensus.cc:1275] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.069679 12955 consensus_queue.cc:1048] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:45.094898 12723 raft_consensus.cc:1275] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.096616 12955 consensus_queue.cc:1048] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.109784 12573 raft_consensus.cc:1275] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 4d47e229db4042c1b76fd3976dd01e8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:45.113175 12955 consensus_queue.cc:1048] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:47.039654 12469 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56868:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:47.074426 12469 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56868:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:47.126999   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:06:47.413288 12985 log.cc:927] Time spent T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: Append to log took a long time: real 0.130s	user 0.000s	sys 0.002s
W20260709 13:06:47.467900 12991 log.cc:927] Time spent T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 13:06:47.437062 13005 tablet.cc:1724] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:47.525117 12990 log.cc:927] Time spent T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
W20260709 13:06:47.417035 12612 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:34547, user_credentials={real_user=slave}} blocked reactor thread for 78326.7us
W20260709 13:06:47.568966 12997 log.cc:927] Time spent T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 13:06:47.542901 12992 log.cc:927] Time spent T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:06:47.681943 13001 log.cc:927] Time spent T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: Append to log took a long time: real 0.170s	user 0.003s	sys 0.000s
W20260709 13:06:47.689495 12675 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:02b24b9c71614b81a916535be4665a6f)
W20260709 13:06:47.690857 12675 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:02b24b9c71614b81a916535be4665a6f)
I20260709 13:06:47.725508 13026 tablet.cc:1724] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:47.729095 13019 log.cc:927] Time spent T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 13:06:47.717387 13023 tablet.cc:1724] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.709895 13016 tablet.cc:1724] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.776608 13027 tablet.cc:1724] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.778143 13031 tablet.cc:1724] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.782958 13034 tablet.cc:1724] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.792075 13024 tablet.cc:1724] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.796909 13030 tablet.cc:1724] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:47.805900 13021 log.cc:927] Time spent T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:06:47.791661 13028 tablet.cc:1724] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.808758 13025 tablet.cc:1724] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.843039 13030 tablet.cc:1724] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.890693 13035 tablet.cc:1724] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.889778 12469 catalog_manager.cc:5789] TS 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195:34547) does not have the latest schema for tablet 090b0397d579459bb29962b23fe60a24 (table soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]). Expected version 1 got 0
W20260709 13:06:47.904708 12600 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:02b24b9c71614b81a916535be4665a6f)
I20260709 13:06:47.928850 12647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
mode: GRACEFUL
new_leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:40534
I20260709 13:06:47.928958 13028 tablet.cc:1724] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.930364 12647 raft_consensus.cc:606] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Received request to transfer leadership to TS 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:47.938508 13005 tablet.cc:1724] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.941406 13030 tablet.cc:1724] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.954197 12471 catalog_manager.cc:5789] TS 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275) does not have the latest schema for tablet c732de1b892946dc9d00762ccb69c3db (table soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]). Expected version 1 got 0
I20260709 13:06:47.919971 13034 tablet.cc:1724] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.972841 13031 tablet.cc:1724] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.982157 13023 tablet.cc:1724] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.986871 13024 tablet.cc:1724] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.972048 13025 tablet.cc:1724] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.990072 13031 tablet.cc:1724] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.989914 13030 tablet.cc:1724] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:47.996170 12471 catalog_manager.cc:5789] TS 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275) does not have the latest schema for tablet 8a789d9dd6544615a3a5cb3c04876322 (table soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]). Expected version 1 got 0
I20260709 13:06:47.980018 13027 tablet.cc:1724] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:48.007416 13026 tablet.cc:1724] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:48.018575 12472 catalog_manager.cc:5789] TS b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497) does not have the latest schema for tablet 54bbfdc11bbb4a20beaa74e510e85dbe (table soft_deleted_table [id=f503e886aa584594a531198b1216fa9a]). Expected version 1 got 0
I20260709 13:06:48.024748 13034 tablet.cc:1724] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:48.026906 13005 tablet.cc:1724] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:48.096918 12571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:44358
I20260709 13:06:48.097728 12571 raft_consensus.cc:606] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:48.099812   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:06:48.101063   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:48.117580 12571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829"
dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
mode: GRACEFUL
new_leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:44358
I20260709 13:06:48.118525 12571 raft_consensus.cc:606] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Received request to transfer leadership to TS 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:48.122449 12648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9"
dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
mode: GRACEFUL
new_leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:40534
I20260709 13:06:48.123236 12648 raft_consensus.cc:606] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Received request to transfer leadership to TS 8da3d1ff50e74afbad7414d7e80b9b34
I20260709 13:06:48.125031   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:48.125690   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:48.127894 12469 catalog_manager.cc:6242] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339: soft_deleted_table [id=f503e886aa584594a531198b1216fa9a] alter complete (version 1)
I20260709 13:06:48.152621 13010 raft_consensus.cc:993] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: : Instructing follower 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.153201 12998 raft_consensus.cc:1081] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Signalling peer 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.155395 12717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:41736
I20260709 13:06:48.156117 12717 raft_consensus.cc:493] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:48.156488 12717 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.163873 12717 raft_consensus.cc:515] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.166708 12717 leader_election.cc:290] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 3 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:48.167822 12572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:48.168815 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c3643c391ca4f0ba7e00eeca2ab243e" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:48.168859 12572 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.169546 12648 raft_consensus.cc:3055] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:48.170089 12648 raft_consensus.cc:740] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:48.171409 12648 consensus_queue.cc:260] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.173096 12648 raft_consensus.cc:3060] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.179484 12572 raft_consensus.cc:2468] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 3.
I20260709 13:06:48.181591 12689 leader_election.cc:304] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:48.182798 12648 raft_consensus.cc:2468] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 3.
I20260709 13:06:48.182979 12963 raft_consensus.cc:2804] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:48.183869 12963 raft_consensus.cc:697] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:48.185560 12963 consensus_queue.cc:237] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.197387 12469 catalog_manager.cc:5697] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 2 to 3, leader changed from 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194) to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.224215 13007 raft_consensus.cc:993] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.225176 12962 raft_consensus.cc:1081] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.227329 12717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:41720
I20260709 13:06:48.228008 12717 raft_consensus.cc:493] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:48.228416 12717 raft_consensus.cc:3060] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:48.237273 12717 raft_consensus.cc:515] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.239939 12717 leader_election.cc:290] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:48.241230 12572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:48.241917 12572 raft_consensus.cc:3055] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:48.242190 12572 raft_consensus.cc:740] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:48.242645 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46a85c7b727941fd8bb6357f03cafeae" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:48.243356 12648 raft_consensus.cc:3060] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:48.243254 12572 consensus_queue.cc:260] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.244793 12572 raft_consensus.cc:3060] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:48.250524 12648 raft_consensus.cc:2468] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 2.
I20260709 13:06:48.251267 12572 raft_consensus.cc:2468] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 2.
I20260709 13:06:48.252243 12687 leader_election.cc:304] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:48.253474 12963 raft_consensus.cc:2804] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:48.254124 12963 raft_consensus.cc:697] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:48.254861 12963 consensus_queue.cc:237] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.263826 12469 catalog_manager.cc:5697] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.359504 12962 raft_consensus.cc:993] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: : Instructing follower 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.360093 13007 raft_consensus.cc:1081] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Signalling peer 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.362318 12717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:41720
I20260709 13:06:48.362967 12717 raft_consensus.cc:493] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:48.363466 12717 raft_consensus.cc:3060] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:48.371069 12717 raft_consensus.cc:515] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.374373 12717 leader_election.cc:290] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 2 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:48.376519 13010 raft_consensus.cc:993] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: : Instructing follower 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.377275 12998 raft_consensus.cc:1081] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Signalling peer 8da3d1ff50e74afbad7414d7e80b9b34 to start an election
I20260709 13:06:48.377223 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:48.377573 12572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d507a98e09764fcbb8e0c5b00e90d829" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:48.378222 12648 raft_consensus.cc:3060] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:48.378506 12572 raft_consensus.cc:3055] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:48.379060 12572 raft_consensus.cc:740] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8c676f2c2264c40b7833e26cf972b7e, State: Running, Role: LEADER
I20260709 13:06:48.379518 12717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9"
dest_uuid: "8da3d1ff50e74afbad7414d7e80b9b34"
 from {username='slave'} at 127.0.0.1:41736
I20260709 13:06:48.380250 12717 raft_consensus.cc:493] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:48.380761 12717 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.380575 12572 consensus_queue.cc:260] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.382447 12572 raft_consensus.cc:3060] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:48.388677 12648 raft_consensus.cc:2468] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 2.
I20260709 13:06:48.390813 12687 leader_election.cc:304] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d47e229db4042c1b76fd3976dd01e8c, 8da3d1ff50e74afbad7414d7e80b9b34; no voters: 
I20260709 13:06:48.391372 12572 raft_consensus.cc:2468] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 2.
I20260709 13:06:48.391302 12717 raft_consensus.cc:515] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.392091 12963 raft_consensus.cc:2804] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:48.392833 12963 raft_consensus.cc:697] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:48.394331 12717 leader_election.cc:290] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 3 election: Requested vote from peers b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497), 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275)
I20260709 13:06:48.394006 12963 consensus_queue.cc:237] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.395795 12572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b8c676f2c2264c40b7833e26cf972b7e"
I20260709 13:06:48.397150 12572 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.399135 12648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebf63419e0c04cb98a73d13ee04340a9" candidate_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d47e229db4042c1b76fd3976dd01e8c"
I20260709 13:06:48.399955 12648 raft_consensus.cc:3055] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:48.401491 12648 raft_consensus.cc:740] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d47e229db4042c1b76fd3976dd01e8c, State: Running, Role: LEADER
I20260709 13:06:48.402383 12648 consensus_queue.cc:260] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.403659 12648 raft_consensus.cc:3060] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:48.410130 12572 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 3.
I20260709 13:06:48.411567 12689 leader_election.cc:304] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da3d1ff50e74afbad7414d7e80b9b34, b8c676f2c2264c40b7833e26cf972b7e; no voters: 
I20260709 13:06:48.411577 12469 catalog_manager.cc:5697] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 1 to 2, leader changed from b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193) to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.412981 12963 raft_consensus.cc:2804] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:48.413282 12648 raft_consensus.cc:2468] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da3d1ff50e74afbad7414d7e80b9b34 in term 3.
I20260709 13:06:48.413569 12963 raft_consensus.cc:697] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 LEADER]: Becoming Leader. State: Replica: 8da3d1ff50e74afbad7414d7e80b9b34, State: Running, Role: LEADER
I20260709 13:06:48.414827 12963 consensus_queue.cc:237] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } }
I20260709 13:06:48.429795 12469 catalog_manager.cc:5697] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 reported cstate change: term changed from 2 to 3, leader changed from 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194) to 8da3d1ff50e74afbad7414d7e80b9b34 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da3d1ff50e74afbad7414d7e80b9b34" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:48.685497 12572 raft_consensus.cc:1275] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:48.687122 12963 consensus_queue.cc:1048] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:48.698009 12648 raft_consensus.cc:1275] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:48.700227 13047 consensus_queue.cc:1048] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:48.754343 12648 raft_consensus.cc:1275] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:48.756210 13004 consensus_queue.cc:1048] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:48.758440 12572 raft_consensus.cc:1275] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:48.760488 13004 consensus_queue.cc:1048] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:48.814275 12572 raft_consensus.cc:1275] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:48.815858 13051 consensus_queue.cc:1048] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:48.828115 12648 raft_consensus.cc:1275] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:48.831707 13004 consensus_queue.cc:1048] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:48.946343 12647 raft_consensus.cc:1275] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:48.948148 13004 consensus_queue.cc:1048] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d47e229db4042c1b76fd3976dd01e8c" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.961561 12572 raft_consensus.cc:1275] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Refusing update from remote peer 8da3d1ff50e74afbad7414d7e80b9b34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:48.963469 13051 consensus_queue.cc:1048] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c676f2c2264c40b7833e26cf972b7e" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:49.213321   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:06:49.219971   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:06:49.220738   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:49.229696   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:06:49.230700   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:50.271350   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:06:50.279143 12689 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
W20260709 13:06:50.284200 12689 consensus_peers.cc:597] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.321251   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:50.322381   431 tablet_replica.cc:333] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.323405   431 raft_consensus.cc:2243] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.324061   431 raft_consensus.cc:2272] T 090b0397d579459bb29962b23fe60a24 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.327310   431 tablet_replica.cc:333] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.328137   431 raft_consensus.cc:2243] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.328835   431 raft_consensus.cc:2272] T c732de1b892946dc9d00762ccb69c3db P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.331720   431 tablet_replica.cc:333] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.332695   431 raft_consensus.cc:2243] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.333400   431 raft_consensus.cc:2272] T 02b24b9c71614b81a916535be4665a6f P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.336237   431 tablet_replica.cc:333] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
W20260709 13:06:50.336249 12689 consensus_peers.cc:597] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.337550   431 raft_consensus.cc:2243] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.338224   431 raft_consensus.cc:2272] T 2c3643c391ca4f0ba7e00eeca2ab243e P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.341143   431 tablet_replica.cc:333] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.342000   431 raft_consensus.cc:2243] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.343014   431 raft_consensus.cc:2272] T b67cd27f5e834e57928e85466f2d9967 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.345937   431 tablet_replica.cc:333] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.346773   431 raft_consensus.cc:2243] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.347811   431 raft_consensus.cc:2272] T 54bbfdc11bbb4a20beaa74e510e85dbe P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.353878   431 tablet_replica.cc:333] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.354854   431 raft_consensus.cc:2243] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.356076   431 raft_consensus.cc:2272] T 87635ece91c543a8b38f8723e6a7194b P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.360596   431 tablet_replica.cc:333] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.361806   431 raft_consensus.cc:2243] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.362732   431 raft_consensus.cc:2272] T 77a9074ed8cc4d38b77ec07e44134c6e P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:50.363365 12615 consensus_peers.cc:597] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:50.364265 12615 consensus_peers.cc:597] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.366410   431 tablet_replica.cc:333] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.367250   431 raft_consensus.cc:2243] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.368422   431 raft_consensus.cc:2272] T c8a84ad985aa4f64a366695c31ee67e3 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.372571   431 tablet_replica.cc:333] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.373279   431 raft_consensus.cc:2243] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.373790   431 raft_consensus.cc:2272] T ebf63419e0c04cb98a73d13ee04340a9 P b8c676f2c2264c40b7833e26cf972b7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.376844   431 tablet_replica.cc:333] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.377866   431 raft_consensus.cc:2243] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.378762   431 raft_consensus.cc:2272] T 8a789d9dd6544615a3a5cb3c04876322 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.382027   431 tablet_replica.cc:333] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.382764   431 raft_consensus.cc:2243] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.383365   431 raft_consensus.cc:2272] T 3af3a07da5e8476f9005e2ded8f90fdb P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.386494   431 tablet_replica.cc:333] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.387270   431 raft_consensus.cc:2243] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.388370   431 raft_consensus.cc:2272] T 5322ad4126d44f58989dbc5a195acd33 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.391155   431 tablet_replica.cc:333] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.391844   431 raft_consensus.cc:2243] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.392379   431 raft_consensus.cc:2272] T d507a98e09764fcbb8e0c5b00e90d829 P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.396550   431 tablet_replica.cc:333] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.397691   431 raft_consensus.cc:2243] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.398649   431 raft_consensus.cc:2272] T 1197098bc264479cba55fe28105b7432 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:50.399988 12689 consensus_peers.cc:597] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.402026   431 tablet_replica.cc:333] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.402887   431 raft_consensus.cc:2243] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.404521   431 raft_consensus.cc:2272] T 3989aa91bcd34ca48467526cc431d712 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.407364   431 tablet_replica.cc:333] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.408041   431 raft_consensus.cc:2243] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.408728   431 raft_consensus.cc:2272] T 46a85c7b727941fd8bb6357f03cafeae P b8c676f2c2264c40b7833e26cf972b7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.412252   431 tablet_replica.cc:333] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e: stopping tablet replica
I20260709 13:06:50.413079   431 raft_consensus.cc:2243] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.413746   431 raft_consensus.cc:2272] T bbb54e72ac8946d0a99c1d2409745095 P b8c676f2c2264c40b7833e26cf972b7e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:50.441221 12689 consensus_peers.cc:597] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:50.459230 12539 connection.cc:466] server connection from 127.0.0.1:47686 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47686 (request call id 106) could send its response
W20260709 13:06:50.460438 12689 consensus_peers.cc:597] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Network error: recv got EOF from 127.0.107.193:45497 (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:50.469949 12689 consensus_peers.cc:597] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45497: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.476814   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:50.521263 12615 consensus_peers.cc:597] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45497: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.534251   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:50.536890 12615 consensus_peers.cc:597] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45497: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:50.542186 12687 consensus_peers.cc:597] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Couldn't send request to peer 4d47e229db4042c1b76fd3976dd01e8c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:50.578171 12615 consensus_peers.cc:597] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45497: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.581678   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:50.582669   431 tablet_replica.cc:333] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.583866   431 raft_consensus.cc:2243] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.584671   431 raft_consensus.cc:2272] T 090b0397d579459bb29962b23fe60a24 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.587437   431 tablet_replica.cc:333] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.588068   431 raft_consensus.cc:2243] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.589049   431 raft_consensus.cc:2272] T 02b24b9c71614b81a916535be4665a6f P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.591900   431 tablet_replica.cc:333] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.592835   431 raft_consensus.cc:2243] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.593935   431 raft_consensus.cc:2272] T c732de1b892946dc9d00762ccb69c3db P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.596747   431 tablet_replica.cc:333] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.597579   431 raft_consensus.cc:2243] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.598223   431 raft_consensus.cc:2272] T 2c3643c391ca4f0ba7e00eeca2ab243e P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.601100   431 tablet_replica.cc:333] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.601881   431 raft_consensus.cc:2243] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.602442   431 raft_consensus.cc:2272] T b67cd27f5e834e57928e85466f2d9967 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.605221   431 tablet_replica.cc:333] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.605898   431 raft_consensus.cc:2243] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.606508   431 raft_consensus.cc:2272] T 54bbfdc11bbb4a20beaa74e510e85dbe P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.609272   431 tablet_replica.cc:333] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.610244   431 raft_consensus.cc:2243] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.610921   431 raft_consensus.cc:2272] T 87635ece91c543a8b38f8723e6a7194b P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.614054   431 tablet_replica.cc:333] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.614962   431 raft_consensus.cc:2243] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.616223   431 raft_consensus.cc:2272] T 8a789d9dd6544615a3a5cb3c04876322 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.619496   431 tablet_replica.cc:333] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
W20260709 13:06:50.620215 12615 consensus_peers.cc:597] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c -> Peer b8c676f2c2264c40b7833e26cf972b7e (127.0.107.193:45497): Couldn't send request to peer b8c676f2c2264c40b7833e26cf972b7e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:45497: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.620440   431 raft_consensus.cc:2243] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.621232   431 raft_consensus.cc:2272] T 3af3a07da5e8476f9005e2ded8f90fdb P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.623759   431 tablet_replica.cc:333] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.624482   431 raft_consensus.cc:2243] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.624966   431 raft_consensus.cc:2272] T 5322ad4126d44f58989dbc5a195acd33 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.627509   431 tablet_replica.cc:333] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.628140   431 raft_consensus.cc:2243] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.628726   431 raft_consensus.cc:2272] T 3989aa91bcd34ca48467526cc431d712 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.631917   431 tablet_replica.cc:333] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.632851   431 raft_consensus.cc:2243] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.634143   431 raft_consensus.cc:2272] T 1197098bc264479cba55fe28105b7432 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:50.637135 12687 consensus_peers.cc:597] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 -> Peer 4d47e229db4042c1b76fd3976dd01e8c (127.0.107.194:37275): Couldn't send request to peer 4d47e229db4042c1b76fd3976dd01e8c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.638244   431 tablet_replica.cc:333] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.639083   431 raft_consensus.cc:2243] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.639631   431 raft_consensus.cc:2272] T d507a98e09764fcbb8e0c5b00e90d829 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.642616   431 tablet_replica.cc:333] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.643374   431 raft_consensus.cc:2243] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.643904   431 raft_consensus.cc:2272] T ebf63419e0c04cb98a73d13ee04340a9 P 4d47e229db4042c1b76fd3976dd01e8c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.646889   431 tablet_replica.cc:333] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.647557   431 raft_consensus.cc:2243] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.648680   431 raft_consensus.cc:2272] T 77a9074ed8cc4d38b77ec07e44134c6e P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.651772   431 tablet_replica.cc:333] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.652647   431 raft_consensus.cc:2243] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.653205   431 raft_consensus.cc:2272] T c8a84ad985aa4f64a366695c31ee67e3 P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.656273   431 tablet_replica.cc:333] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.656971   431 raft_consensus.cc:2243] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.657455   431 raft_consensus.cc:2272] T 46a85c7b727941fd8bb6357f03cafeae P 4d47e229db4042c1b76fd3976dd01e8c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.659904   431 tablet_replica.cc:333] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c: stopping tablet replica
I20260709 13:06:50.660555   431 raft_consensus.cc:2243] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.661512   431 raft_consensus.cc:2272] T bbb54e72ac8946d0a99c1d2409745095 P 4d47e229db4042c1b76fd3976dd01e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.709728   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:06:50.757937   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:50.803710   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:50.804795   431 tablet_replica.cc:333] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.805883   431 raft_consensus.cc:2243] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.806864   431 raft_consensus.cc:2272] T 02b24b9c71614b81a916535be4665a6f P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.809464   431 tablet_replica.cc:333] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.810254   431 raft_consensus.cc:2243] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.811570   431 raft_consensus.cc:2272] T 2c3643c391ca4f0ba7e00eeca2ab243e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.814201   431 tablet_replica.cc:333] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.814857   431 raft_consensus.cc:2243] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.815326   431 raft_consensus.cc:2272] T c732de1b892946dc9d00762ccb69c3db P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.817713   431 tablet_replica.cc:333] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.818447   431 raft_consensus.cc:2243] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.819633   431 raft_consensus.cc:2272] T 090b0397d579459bb29962b23fe60a24 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.822402   431 tablet_replica.cc:333] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.823062   431 raft_consensus.cc:2243] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.823586   431 raft_consensus.cc:2272] T b67cd27f5e834e57928e85466f2d9967 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.826198   431 tablet_replica.cc:333] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.826835   431 raft_consensus.cc:2243] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.827790   431 raft_consensus.cc:2272] T 46a85c7b727941fd8bb6357f03cafeae P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.831095   431 tablet_replica.cc:333] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.831943   431 raft_consensus.cc:2243] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.832574   431 raft_consensus.cc:2272] T 54bbfdc11bbb4a20beaa74e510e85dbe P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.835086   431 tablet_replica.cc:333] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.835808   431 raft_consensus.cc:2243] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.836386   431 raft_consensus.cc:2272] T 87635ece91c543a8b38f8723e6a7194b P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.838884   431 tablet_replica.cc:333] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.839515   431 raft_consensus.cc:2243] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.840018   431 raft_consensus.cc:2272] T bbb54e72ac8946d0a99c1d2409745095 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.842798   431 tablet_replica.cc:333] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.843487   431 raft_consensus.cc:2243] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.844512   431 raft_consensus.cc:2272] T ebf63419e0c04cb98a73d13ee04340a9 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.847213   431 tablet_replica.cc:333] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.847851   431 raft_consensus.cc:2243] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.848456   431 raft_consensus.cc:2272] T c8a84ad985aa4f64a366695c31ee67e3 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.851073   431 tablet_replica.cc:333] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.851986   431 raft_consensus.cc:2243] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.852787   431 raft_consensus.cc:2272] T 77a9074ed8cc4d38b77ec07e44134c6e P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.855600   431 tablet_replica.cc:333] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.856326   431 raft_consensus.cc:2243] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.857133   431 raft_consensus.cc:2272] T 8a789d9dd6544615a3a5cb3c04876322 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.859762   431 tablet_replica.cc:333] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.860498   431 raft_consensus.cc:2243] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.861090   431 raft_consensus.cc:2272] T 5322ad4126d44f58989dbc5a195acd33 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.864099   431 tablet_replica.cc:333] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.864802   431 raft_consensus.cc:2243] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.865851   431 raft_consensus.cc:2272] T 3af3a07da5e8476f9005e2ded8f90fdb P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.869004   431 tablet_replica.cc:333] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.870195   431 raft_consensus.cc:2243] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.870939   431 raft_consensus.cc:2272] T 3989aa91bcd34ca48467526cc431d712 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.873484   431 tablet_replica.cc:333] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.874018   431 raft_consensus.cc:2243] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.874827   431 raft_consensus.cc:2272] T 1197098bc264479cba55fe28105b7432 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.879137   431 tablet_replica.cc:333] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34: stopping tablet replica
I20260709 13:06:50.880193   431 raft_consensus.cc:2243] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:50.881299   431 raft_consensus.cc:2272] T d507a98e09764fcbb8e0c5b00e90d829 P 8da3d1ff50e74afbad7414d7e80b9b34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.938838   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:50.989295   431 master.cc:562] Master@127.0.107.254:36045 shutting down...
I20260709 13:06:51.019519   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:51.020645   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:51.021188   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P b4d2da7b000443c5b6a2a1e688d1a339: stopping tablet replica
I20260709 13:06:51.045651   431 master.cc:584] Master@127.0.107.254:36045 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (14223 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:06:51.115330   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:36441
I20260709 13:06:51.117025   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:51.124027 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:51.124044 13063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.132550   431 server_base.cc:1061] running on GCE node
W20260709 13:06:51.132416 13065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.133878   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:51.134125   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:51.134317   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602411134302 us; error 0 us; skew 500 ppm
I20260709 13:06:51.134918   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:51.138036   431 webserver.cc:533] Webserver started at http://127.0.107.254:37145/ using document root <none> and password file <none>
I20260709 13:06:51.138751   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:51.138954   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:51.139328   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:51.140779   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/master-0-root/instance:
uuid: "7b2b5f93978b44e7a360e4285d523c0f"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.146277   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:06:51.150997 13070 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.152088   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:51.152604   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/master-0-root
uuid: "7b2b5f93978b44e7a360e4285d523c0f"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.153090   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:51.171149   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:51.172610   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:51.225761   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:36441
I20260709 13:06:51.225888 13121 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:36441 every 8 connection(s)
I20260709 13:06:51.231068 13122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.247860 13122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f: Bootstrap starting.
I20260709 13:06:51.255805 13122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.262013 13122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f: No bootstrap required, opened a new log
I20260709 13:06:51.264930 13122 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2b5f93978b44e7a360e4285d523c0f" member_type: VOTER }
I20260709 13:06:51.265529 13122 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.265813 13122 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b2b5f93978b44e7a360e4285d523c0f, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.266609 13122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2b5f93978b44e7a360e4285d523c0f" member_type: VOTER }
I20260709 13:06:51.267195 13122 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:51.267467 13122 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:51.267766 13122 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.275163 13122 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2b5f93978b44e7a360e4285d523c0f" member_type: VOTER }
I20260709 13:06:51.275992 13122 leader_election.cc:304] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b2b5f93978b44e7a360e4285d523c0f; no voters: 
I20260709 13:06:51.277628 13122 leader_election.cc:290] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:51.278215 13125 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.280009 13125 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 1 LEADER]: Becoming Leader. State: Replica: 7b2b5f93978b44e7a360e4285d523c0f, State: Running, Role: LEADER
I20260709 13:06:51.280997 13125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2b5f93978b44e7a360e4285d523c0f" member_type: VOTER }
I20260709 13:06:51.281750 13122 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:51.286705 13127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b2b5f93978b44e7a360e4285d523c0f. Latest consensus state: current_term: 1 leader_uuid: "7b2b5f93978b44e7a360e4285d523c0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2b5f93978b44e7a360e4285d523c0f" member_type: VOTER } }
I20260709 13:06:51.287834 13127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:51.287493 13126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b2b5f93978b44e7a360e4285d523c0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2b5f93978b44e7a360e4285d523c0f" member_type: VOTER } }
I20260709 13:06:51.289712 13126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:51.291455 13132 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:51.298470 13132 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:51.303565   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:51.310813 13132 catalog_manager.cc:1383] Generated new cluster ID: 21d77dd4679a4b3188b6426548e593ce
I20260709 13:06:51.311254 13132 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:51.334105 13132 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:51.335922 13132 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:51.353271 13132 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f: Generated new TSK 0
I20260709 13:06:51.354172 13132 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:51.374977   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:51.383477 13143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:51.385141 13144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:51.386358 13146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.387050   431 server_base.cc:1061] running on GCE node
I20260709 13:06:51.388010   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:51.388438   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:51.388677   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602411388658 us; error 0 us; skew 500 ppm
I20260709 13:06:51.389429   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:51.392863   431 webserver.cc:533] Webserver started at http://127.0.107.193:36793/ using document root <none> and password file <none>
I20260709 13:06:51.393464   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:51.393654   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:51.393920   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:51.395236   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-0-root/instance:
uuid: "6d05e27c97034a909efec0f5dd6b4b31"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.401062   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:06:51.405707 13151 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.407596   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:51.407987   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-0-root
uuid: "6d05e27c97034a909efec0f5dd6b4b31"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.408298   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:51.425369   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:51.426862   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:51.428879   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:51.432056   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:51.432327   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.432672   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:51.432881   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.492112   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42139
I20260709 13:06:51.492291 13213 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42139 every 8 connection(s)
I20260709 13:06:51.498373   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:51.510339 13219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:51.510313 13218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.516003   431 server_base.cc:1061] running on GCE node
W20260709 13:06:51.517072 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.518888   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:51.519197   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:51.519402   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602411519382 us; error 0 us; skew 500 ppm
I20260709 13:06:51.520062   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:51.523082 13214 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36441
I20260709 13:06:51.523546 13214 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:51.524394   431 webserver.cc:533] Webserver started at http://127.0.107.194:40223/ using document root <none> and password file <none>
I20260709 13:06:51.524647 13214 heartbeater.cc:507] Master 127.0.107.254:36441 requested a full tablet report, sending...
I20260709 13:06:51.525415   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:51.525728   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:51.526060   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:51.527604 13087 ts_manager.cc:194] Registered new tserver with Master: 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139)
I20260709 13:06:51.527618   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-1-root/instance:
uuid: "eecc19d37a4940b6997f35cd9ffb574f"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.530205 13087 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53866
I20260709 13:06:51.534471   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:51.539520 13226 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.540886   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:51.541293   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-1-root
uuid: "eecc19d37a4940b6997f35cd9ffb574f"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.541656   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:51.564603   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:51.566128   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:51.567955   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:51.571146   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:51.571456   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.571991   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:51.572435   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.002s	sys 0.000s
I20260709 13:06:51.629784   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:46015
I20260709 13:06:51.629858 13288 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:46015 every 8 connection(s)
I20260709 13:06:51.636191   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:51.646996 13293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:51.647493 13292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.652436   431 server_base.cc:1061] running on GCE node
I20260709 13:06:51.653522 13289 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36441
I20260709 13:06:51.654309 13289 heartbeater.cc:461] Registering TS with master...
W20260709 13:06:51.654184 13295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:51.655774 13289 heartbeater.cc:507] Master 127.0.107.254:36441 requested a full tablet report, sending...
I20260709 13:06:51.656402   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:51.656812   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:51.657042   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602411657015 us; error 0 us; skew 500 ppm
I20260709 13:06:51.657972   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:51.659298 13087 ts_manager.cc:194] Registered new tserver with Master: eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:51.661877 13087 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53878
I20260709 13:06:51.662000   431 webserver.cc:533] Webserver started at http://127.0.107.195:37173/ using document root <none> and password file <none>
I20260709 13:06:51.662886   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:51.663162   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:51.663575   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:51.666337   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-2-root/instance:
uuid: "ff2c85391bff48d5bafcf26c004ad30e"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.675961   431 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:51.680756 13300 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.681926   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:51.682339   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-2-root
uuid: "ff2c85391bff48d5bafcf26c004ad30e"
format_stamp: "Formatted at 2026-07-09 13:06:51 on dist-test-slave-ftjc"
I20260709 13:06:51.682703   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskWdLg9H/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602002558854-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:51.718972   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:51.720726   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:51.722841   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:51.725742   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:51.726058   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.726358   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:51.726562   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:51.790704   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36547
I20260709 13:06:51.791213 13362 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36547 every 8 connection(s)
I20260709 13:06:51.811321 13363 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36441
I20260709 13:06:51.811980 13363 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:51.813153 13363 heartbeater.cc:507] Master 127.0.107.254:36441 requested a full tablet report, sending...
I20260709 13:06:51.816172 13087 ts_manager.cc:194] Registered new tserver with Master: ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:51.817080   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021773676s
I20260709 13:06:51.818140 13087 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53882
I20260709 13:06:51.855350 13087 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53894:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:51.858765 13087 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:51.956454 13179 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd9455d9af4433d9e2f66e3018c86e8 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:51.958777 13179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd9455d9af4433d9e2f66e3018c86e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.962473 13178 tablet_service.cc:1511] Processing CreateTablet for tablet 68a72ac98d9f4748bc8466b080e758e4 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:51.962718 13328 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd9455d9af4433d9e2f66e3018c86e8 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:51.960495 13177 tablet_service.cc:1511] Processing CreateTablet for tablet 34e536fb23464ff3b1396e46945ac5ea (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:51.960997 13176 tablet_service.cc:1511] Processing CreateTablet for tablet 940faeaa383542e1ac59b86fa4167a8b (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:51.964480 13178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a72ac98d9f4748bc8466b080e758e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.964783 13328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd9455d9af4433d9e2f66e3018c86e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.965785 13176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940faeaa383542e1ac59b86fa4167a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.969486 13175 tablet_service.cc:1511] Processing CreateTablet for tablet 1b6fbf23640a4e858af7f28bddd6ad2e (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:51.973763 13326 tablet_service.cc:1511] Processing CreateTablet for tablet 34e536fb23464ff3b1396e46945ac5ea (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:51.974614 13327 tablet_service.cc:1511] Processing CreateTablet for tablet 68a72ac98d9f4748bc8466b080e758e4 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:51.975698 13326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34e536fb23464ff3b1396e46945ac5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.977375 13324 tablet_service.cc:1511] Processing CreateTablet for tablet 1b6fbf23640a4e858af7f28bddd6ad2e (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:51.979017 13324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b6fbf23640a4e858af7f28bddd6ad2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.981752 13325 tablet_service.cc:1511] Processing CreateTablet for tablet 940faeaa383542e1ac59b86fa4167a8b (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:51.983554 13325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940faeaa383542e1ac59b86fa4167a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.984987 13327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a72ac98d9f4748bc8466b080e758e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.988991 13175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b6fbf23640a4e858af7f28bddd6ad2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.994791 13177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34e536fb23464ff3b1396e46945ac5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:51.998270 13322 tablet_service.cc:1511] Processing CreateTablet for tablet ba35c9ea187e482cbeb39f6b219c7030 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:52.001123 13321 tablet_service.cc:1511] Processing CreateTablet for tablet 383dda1d741941398461d227aee2b621 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:52.002370 13323 tablet_service.cc:1511] Processing CreateTablet for tablet 30321d144d6140b0a45e60b81c9379d3 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:52.000165 13322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba35c9ea187e482cbeb39f6b219c7030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.008155 13323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30321d144d6140b0a45e60b81c9379d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.010424 13174 tablet_service.cc:1511] Processing CreateTablet for tablet 30321d144d6140b0a45e60b81c9379d3 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:52.015314 13173 tablet_service.cc:1511] Processing CreateTablet for tablet ba35c9ea187e482cbeb39f6b219c7030 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:52.018419 13174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30321d144d6140b0a45e60b81c9379d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.021317 13321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 383dda1d741941398461d227aee2b621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.024187 13173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba35c9ea187e482cbeb39f6b219c7030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.071400 13381 tablet_bootstrap.cc:492] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.074267 13328 tablet_service.cc:1511] Processing CreateTablet for tablet caef7e2d7c474ee1a9b8f91327426099 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:52.076475 13328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caef7e2d7c474ee1a9b8f91327426099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.080369 13381 tablet_bootstrap.cc:654] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.083081 13172 tablet_service.cc:1511] Processing CreateTablet for tablet 383dda1d741941398461d227aee2b621 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:52.087174 13172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 383dda1d741941398461d227aee2b621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.084651 13171 tablet_service.cc:1511] Processing CreateTablet for tablet caef7e2d7c474ee1a9b8f91327426099 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:52.090534 13171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caef7e2d7c474ee1a9b8f91327426099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.123031 13254 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd9455d9af4433d9e2f66e3018c86e8 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:52.125527 13253 tablet_service.cc:1511] Processing CreateTablet for tablet 68a72ac98d9f4748bc8466b080e758e4 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:52.127362 13253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a72ac98d9f4748bc8466b080e758e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.127691 13246 tablet_service.cc:1511] Processing CreateTablet for tablet caef7e2d7c474ee1a9b8f91327426099 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:52.129706 13252 tablet_service.cc:1511] Processing CreateTablet for tablet 34e536fb23464ff3b1396e46945ac5ea (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:52.130333 13246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caef7e2d7c474ee1a9b8f91327426099. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.132581 13251 tablet_service.cc:1511] Processing CreateTablet for tablet 940faeaa383542e1ac59b86fa4167a8b (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:52.134603 13251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940faeaa383542e1ac59b86fa4167a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.137753 13250 tablet_service.cc:1511] Processing CreateTablet for tablet 1b6fbf23640a4e858af7f28bddd6ad2e (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:52.139842 13250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b6fbf23640a4e858af7f28bddd6ad2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.141640 13248 tablet_service.cc:1511] Processing CreateTablet for tablet ba35c9ea187e482cbeb39f6b219c7030 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:52.143683 13248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba35c9ea187e482cbeb39f6b219c7030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.144495 13381 tablet_bootstrap.cc:492] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.125653 13249 tablet_service.cc:1511] Processing CreateTablet for tablet 30321d144d6140b0a45e60b81c9379d3 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:52.155709 13381 ts_tablet_manager.cc:1403] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.087s	user 0.016s	sys 0.057s
I20260709 13:06:52.145462 13247 tablet_service.cc:1511] Processing CreateTablet for tablet 383dda1d741941398461d227aee2b621 (DEFAULT_TABLE table=test-workload [id=7e28ce1ba670438d9700d2b297190640]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:52.152064 13254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd9455d9af4433d9e2f66e3018c86e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.159874 13381 raft_consensus.cc:359] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.160786 13247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 383dda1d741941398461d227aee2b621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.161689 13252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34e536fb23464ff3b1396e46945ac5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.164759 13249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30321d144d6140b0a45e60b81c9379d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.160765 13381 raft_consensus.cc:385] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.172884 13381 raft_consensus.cc:740] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.174073 13381 consensus_queue.cc:260] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.185174 13383 tablet_bootstrap.cc:492] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.199267 13383 tablet_bootstrap.cc:654] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.208412 13383 tablet_bootstrap.cc:492] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.209272 13383 ts_tablet_manager.cc:1403] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 13:06:52.213255 13383 raft_consensus.cc:359] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.214222 13383 raft_consensus.cc:385] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.214596 13383 raft_consensus.cc:740] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.215770 13383 consensus_queue.cc:260] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.225108 13214 heartbeater.cc:499] Master 127.0.107.254:36441 was elected leader, sending a full tablet report...
I20260709 13:06:52.227612 13383 ts_tablet_manager.cc:1434] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.018s	user 0.013s	sys 0.001s
I20260709 13:06:52.229454 13383 tablet_bootstrap.cc:492] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.237172 13383 tablet_bootstrap.cc:654] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:52.247893 13215 tablet.cc:2406] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:52.252300 13383 tablet_bootstrap.cc:492] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.253243 13383 ts_tablet_manager.cc:1403] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.005s
I20260709 13:06:52.257253 13383 raft_consensus.cc:359] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.258575 13383 raft_consensus.cc:385] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.259086 13383 raft_consensus.cc:740] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.260180 13383 consensus_queue.cc:260] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.264123 13383 ts_tablet_manager.cc:1434] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:52.265551 13383 tablet_bootstrap.cc:492] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.273717 13383 tablet_bootstrap.cc:654] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.277824 13363 heartbeater.cc:499] Master 127.0.107.254:36441 was elected leader, sending a full tablet report...
I20260709 13:06:52.281416 13381 ts_tablet_manager.cc:1434] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.125s	user 0.015s	sys 0.025s
I20260709 13:06:52.282769 13381 tablet_bootstrap.cc:492] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.287209 13383 tablet_bootstrap.cc:492] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.288502 13383 ts_tablet_manager.cc:1403] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.005s
I20260709 13:06:52.291848 13381 tablet_bootstrap.cc:654] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.300053 13387 raft_consensus.cc:493] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.300822 13387 raft_consensus.cc:515] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:06:52.296258 13135 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:52.292840 13383 raft_consensus.cc:359] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.317183 13383 raft_consensus.cc:385] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.317833 13383 raft_consensus.cc:740] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.338109 13387 leader_election.cc:290] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.338397 13381 tablet_bootstrap.cc:492] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.339382 13381 ts_tablet_manager.cc:1403] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.014s
I20260709 13:06:52.340709 13385 tablet_bootstrap.cc:492] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:52.344761 13381 raft_consensus.cc:359] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.345690 13381 raft_consensus.cc:385] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.346078 13381 raft_consensus.cc:740] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.347260 13381 consensus_queue.cc:260] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.350185 13381 ts_tablet_manager.cc:1434] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 13:06:52.351752 13381 tablet_bootstrap.cc:492] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.352888 13385 tablet_bootstrap.cc:654] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.352587 13383 consensus_queue.cc:260] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.355489 13383 ts_tablet_manager.cc:1434] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.001s
I20260709 13:06:52.380635 13383 tablet_bootstrap.cc:492] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.381649 13381 tablet_bootstrap.cc:654] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.469609 13387 raft_consensus.cc:493] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.470445 13387 raft_consensus.cc:515] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.496240 13387 leader_election.cc:290] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.496560 13383 tablet_bootstrap.cc:654] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.501166 13381 tablet_bootstrap.cc:492] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.501842 13381 ts_tablet_manager.cc:1403] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.150s	user 0.018s	sys 0.002s
I20260709 13:06:52.506942 13385 tablet_bootstrap.cc:492] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:52.506780 13381 raft_consensus.cc:359] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.509262 13381 raft_consensus.cc:385] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.509918 13381 raft_consensus.cc:740] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.507900 13385 ts_tablet_manager.cc:1403] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.168s	user 0.028s	sys 0.047s
I20260709 13:06:52.512619 13381 consensus_queue.cc:260] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.514377 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.515494 13381 ts_tablet_manager.cc:1434] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:52.515450 13385 raft_consensus.cc:359] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.516919 13385 raft_consensus.cc:385] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.517443 13381 tablet_bootstrap.cc:492] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.517455 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:52.518622 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
W20260709 13:06:52.506781 13071 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:46015, user_credentials={real_user=slave}} blocked reactor thread for 97423.4us
I20260709 13:06:52.517495 13385 raft_consensus.cc:740] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
W20260709 13:06:52.521710 13152 leader_election.cc:343] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Not found: Tablet not found: 30321d144d6140b0a45e60b81c9379d3
W20260709 13:06:52.523150 13152 leader_election.cc:343] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.524422 13152 leader_election.cc:343] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.525475 13152 leader_election.cc:304] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31; no voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:52.526836 13381 tablet_bootstrap.cc:654] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.527700 13387 raft_consensus.cc:2749] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:52.528234 13152 leader_election.cc:343] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.528904 13152 leader_election.cc:304] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31; no voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:52.529109 13385 consensus_queue.cc:260] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.530553 13387 raft_consensus.cc:2749] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.531690 13383 tablet_bootstrap.cc:492] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.532407 13383 ts_tablet_manager.cc:1403] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.167s	user 0.033s	sys 0.059s
I20260709 13:06:52.536854 13289 heartbeater.cc:499] Master 127.0.107.254:36441 was elected leader, sending a full tablet report...
I20260709 13:06:52.536175 13383 raft_consensus.cc:359] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.538941 13383 raft_consensus.cc:385] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.539449 13383 raft_consensus.cc:740] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.539994 13381 tablet_bootstrap.cc:492] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.541201 13381 ts_tablet_manager.cc:1403] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.002s
I20260709 13:06:52.540735 13383 consensus_queue.cc:260] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.536182 13385 ts_tablet_manager.cc:1434] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:06:52.542899 13385 tablet_bootstrap.cc:492] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:52.544075 13383 ts_tablet_manager.cc:1434] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 13:06:52.545215 13381 raft_consensus.cc:359] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.546274 13381 raft_consensus.cc:385] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.546706 13381 raft_consensus.cc:740] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.547878 13381 consensus_queue.cc:260] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.552043 13381 ts_tablet_manager.cc:1434] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:06:52.553787 13385 tablet_bootstrap.cc:654] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.553745 13387 raft_consensus.cc:493] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.554592 13381 tablet_bootstrap.cc:492] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.554716 13387 raft_consensus.cc:515] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.557581 13384 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.559392 13383 tablet_bootstrap.cc:492] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.560163 13387 leader_election.cc:290] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.560904 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.563102 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:52.564219 13337 raft_consensus.cc:2468] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:52.566104 13381 tablet_bootstrap.cc:654] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.566597 13152 leader_election.cc:304] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
W20260709 13:06:52.567906 13152 leader_election.cc:343] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.569199 13387 raft_consensus.cc:2804] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.569876 13387 raft_consensus.cc:493] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.570479 13387 raft_consensus.cc:3060] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.558213 13384 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.581151 13383 tablet_bootstrap.cc:654] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.582624 13387 raft_consensus.cc:515] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.591395 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:52.594511 13387 leader_election.cc:290] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
W20260709 13:06:52.597090 13152 leader_election.cc:343] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.599498 13384 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:52.600003 13385 tablet_bootstrap.cc:492] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:52.604305 13385 ts_tablet_manager.cc:1403] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.003s
I20260709 13:06:52.607110 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:52.608120 13337 raft_consensus.cc:3060] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.609403 13383 tablet_bootstrap.cc:492] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.610376 13385 raft_consensus.cc:359] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.611851 13385 raft_consensus.cc:385] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.610423 13383 ts_tablet_manager.cc:1403] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.051s	user 0.018s	sys 0.007s
I20260709 13:06:52.613206 13381 tablet_bootstrap.cc:492] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.614023 13381 ts_tablet_manager.cc:1403] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.061s	user 0.019s	sys 0.030s
I20260709 13:06:52.616408 13383 raft_consensus.cc:359] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.617828 13381 raft_consensus.cc:359] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.618481 13383 raft_consensus.cc:385] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.612283 13385 raft_consensus.cc:740] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.618975 13381 raft_consensus.cc:385] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.619395 13381 raft_consensus.cc:740] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.619959 13385 consensus_queue.cc:260] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.620424 13381 consensus_queue.cc:260] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.622718 13385 ts_tablet_manager.cc:1434] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:06:52.623359 13381 ts_tablet_manager.cc:1434] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:52.624778 13385 tablet_bootstrap.cc:492] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:52.625268 13381 tablet_bootstrap.cc:492] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.623142 13383 raft_consensus.cc:740] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.631014 13383 consensus_queue.cc:260] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.633111 13337 raft_consensus.cc:2468] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:52.633944 13381 tablet_bootstrap.cc:654] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.641010 13152 leader_election.cc:304] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:52.642069 13387 raft_consensus.cc:2804] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.635437 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.637399 13383 ts_tablet_manager.cc:1434] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.002s
I20260709 13:06:52.645572 13387 raft_consensus.cc:697] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:52.645931 13385 tablet_bootstrap.cc:654] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.647024 13387 consensus_queue.cc:237] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.653489 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
W20260709 13:06:52.655671 13301 leader_election.cc:343] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.656507 13303 leader_election.cc:343] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.661000 13383 tablet_bootstrap.cc:492] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.657148 13303 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:52.663678 13384 raft_consensus.cc:2749] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.666617 13381 tablet_bootstrap.cc:492] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.667418 13381 ts_tablet_manager.cc:1403] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.015s
I20260709 13:06:52.669098 13086 catalog_manager.cc:5697] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.671377 13381 raft_consensus.cc:359] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.672410 13381 raft_consensus.cc:385] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.672746 13383 tablet_bootstrap.cc:654] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.672781 13381 raft_consensus.cc:740] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.674100 13381 consensus_queue.cc:260] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.676026 13385 tablet_bootstrap.cc:492] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:52.677105 13385 ts_tablet_manager.cc:1403] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.003s
I20260709 13:06:52.677956 13381 ts_tablet_manager.cc:1434] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.001s
I20260709 13:06:52.679745 13381 tablet_bootstrap.cc:492] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.680814 13385 raft_consensus.cc:359] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.681941 13385 raft_consensus.cc:385] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.683290 13385 raft_consensus.cc:740] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.684383 13385 consensus_queue.cc:260] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.685818 13397 raft_consensus.cc:493] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.686596 13397 raft_consensus.cc:515] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.691898 13381 tablet_bootstrap.cc:654] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.702013 13397 leader_election.cc:290] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.704063 13397 raft_consensus.cc:493] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.706453 13397 raft_consensus.cc:515] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.707588 13381 tablet_bootstrap.cc:492] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.708578 13381 ts_tablet_manager.cc:1403] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.029s	user 0.024s	sys 0.004s
I20260709 13:06:52.714068 13381 raft_consensus.cc:359] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.715327 13381 raft_consensus.cc:385] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.715762 13381 raft_consensus.cc:740] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.717137 13381 consensus_queue.cc:260] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.720276 13381 ts_tablet_manager.cc:1434] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:52.721728 13381 tablet_bootstrap.cc:492] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.729264 13397 leader_election.cc:290] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.730530 13397 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.731308 13397 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.732754 13381 tablet_bootstrap.cc:654] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.742583 13383 tablet_bootstrap.cc:492] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.743196 13383 ts_tablet_manager.cc:1403] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.058s
I20260709 13:06:52.746703 13383 raft_consensus.cc:359] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.747202 13381 tablet_bootstrap.cc:492] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.747624 13383 raft_consensus.cc:385] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.747982 13383 raft_consensus.cc:740] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.748066 13381 ts_tablet_manager.cc:1403] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.017s
I20260709 13:06:52.749179 13383 consensus_queue.cc:260] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.752456 13381 raft_consensus.cc:359] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.753692 13381 raft_consensus.cc:385] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.754019 13381 raft_consensus.cc:740] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.755862 13381 consensus_queue.cc:260] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.757148 13383 ts_tablet_manager.cc:1434] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 13:06:52.758577 13383 tablet_bootstrap.cc:492] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.759104 13381 ts_tablet_manager.cc:1434] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 13:06:52.760449 13381 tablet_bootstrap.cc:492] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:52.760854 13385 ts_tablet_manager.cc:1434] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.083s	user 0.010s	sys 0.000s
I20260709 13:06:52.762501 13385 tablet_bootstrap.cc:492] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:52.767920 13381 tablet_bootstrap.cc:654] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.768184 13383 tablet_bootstrap.cc:654] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.781420 13383 tablet_bootstrap.cc:492] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.782124 13383 ts_tablet_manager.cc:1403] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 13:06:52.781419 13381 tablet_bootstrap.cc:492] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:52.784133 13381 ts_tablet_manager.cc:1403] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.003s
I20260709 13:06:52.791919 13385 tablet_bootstrap.cc:654] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.792536 13383 raft_consensus.cc:359] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.797201 13381 raft_consensus.cc:359] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.798274 13381 raft_consensus.cc:385] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.798686 13381 raft_consensus.cc:740] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.799688 13397 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.794054 13383 raft_consensus.cc:385] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.800499 13383 raft_consensus.cc:740] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.801713 13383 consensus_queue.cc:260] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.799652 13381 consensus_queue.cc:260] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.804114 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.805023 13383 ts_tablet_manager.cc:1434] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:06:52.805886 13385 tablet_bootstrap.cc:492] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:52.806315 13383 tablet_bootstrap.cc:492] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.806519 13385 ts_tablet_manager.cc:1403] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.006s
I20260709 13:06:52.807294 13381 ts_tablet_manager.cc:1434] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 13:06:52.810456 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.811909 13189 raft_consensus.cc:2468] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:52.810518 13384 raft_consensus.cc:493] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.813805 13385 raft_consensus.cc:359] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.813877 13384 raft_consensus.cc:515] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.814908 13385 raft_consensus.cc:385] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.815469 13385 raft_consensus.cc:740] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.816488 13383 tablet_bootstrap.cc:654] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.817883 13229 leader_election.cc:304] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:52.819001 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.820111 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.821165 13189 raft_consensus.cc:2468] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:52.822399 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.816541 13385 consensus_queue.cc:260] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.824142 13303 leader_election.cc:304] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:52.820187 13188 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
W20260709 13:06:52.827158 13301 leader_election.cc:343] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.826701 13397 raft_consensus.cc:2804] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.828228 13397 raft_consensus.cc:493] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.825940 13384 leader_election.cc:290] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:52.828897 13229 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:52.829687 13384 raft_consensus.cc:2804] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.830226 13384 raft_consensus.cc:493] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.828897 13397 raft_consensus.cc:3060] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.830919 13384 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.830286 13411 raft_consensus.cc:2804] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.837329 13387 raft_consensus.cc:493] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.839030 13387 raft_consensus.cc:515] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.837030 13411 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.845685 13411 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.846103 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.847362 13264 raft_consensus.cc:2468] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:52.841687 13387 leader_election.cc:290] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.849663 13152 leader_election.cc:304] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:52.850663 13387 raft_consensus.cc:2804] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.850803 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:52.851755 13387 raft_consensus.cc:493] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.852211 13337 raft_consensus.cc:2468] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:52.852268 13387 raft_consensus.cc:3060] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.856128 13397 raft_consensus.cc:515] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.858075 13411 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:52.861419 13383 tablet_bootstrap.cc:492] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:52.862176 13383 ts_tablet_manager.cc:1403] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.015s
I20260709 13:06:52.862643 13385 ts_tablet_manager.cc:1434] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.003s
I20260709 13:06:52.864053 13385 tablet_bootstrap.cc:492] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:52.867583 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:52.869213 13337 raft_consensus.cc:2393] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eecc19d37a4940b6997f35cd9ffb574f in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:52.870805 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:52.873044 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:52.874282 13336 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:52.875614 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:52.875793 13383 raft_consensus.cc:359] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.862480 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:52.877465 13383 raft_consensus.cc:385] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.877604 13189 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.877918 13383 raft_consensus.cc:740] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.876739 13387 raft_consensus.cc:515] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.879118 13383 consensus_queue.cc:260] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.881685 13383 ts_tablet_manager.cc:1434] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.010s
I20260709 13:06:52.882560 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:52.883299 13264 raft_consensus.cc:3060] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.883371 13383 tablet_bootstrap.cc:492] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:52.885783 13188 raft_consensus.cc:3060] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.889405 13384 raft_consensus.cc:515] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.891463 13383 tablet_bootstrap.cc:654] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.892849 13384 leader_election.cc:290] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:52.882578 13385 tablet_bootstrap.cc:654] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.893941 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:52.895220 13227 leader_election.cc:304] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
W20260709 13:06:52.895937 13301 leader_election.cc:343] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.896809 13415 raft_consensus.cc:3060] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.897678 13264 raft_consensus.cc:2468] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:52.899657 13152 leader_election.cc:304] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:52.872082 13338 raft_consensus.cc:2468] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:52.901966 13405 raft_consensus.cc:2804] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.902439 13384 raft_consensus.cc:493] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.903012 13384 raft_consensus.cc:515] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.903520 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:52.908823 13422 raft_consensus.cc:493] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.909303 13384 leader_election.cc:290] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:52.909893 13422 raft_consensus.cc:515] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.911330 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.913085 13263 raft_consensus.cc:2393] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff2c85391bff48d5bafcf26c004ad30e in current term 1: Already voted for candidate eecc19d37a4940b6997f35cd9ffb574f in this term.
I20260709 13:06:52.914414 13422 leader_election.cc:290] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:52.915860 13336 raft_consensus.cc:3060] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.918542 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
W20260709 13:06:52.920770 13301 leader_election.cc:343] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.925665 13422 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.928563 13405 raft_consensus.cc:697] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:52.930701 13405 consensus_queue.cc:237] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.926580 13422 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:52.938573 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:52.939261 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:52.939635 13338 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:52.941635 13301 leader_election.cc:343] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.947923 13387 leader_election.cc:290] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.952896 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:52.954499 13187 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.957212 13411 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.966595 13186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.968755 13185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.969996 13185 raft_consensus.cc:2468] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:52.971940 13184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:52.968772 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:52.973416 13422 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:52.973809 13337 raft_consensus.cc:3060] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.867653 13397 leader_election.cc:290] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:52.977219 13187 raft_consensus.cc:2468] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:52.979856 13303 leader_election.cc:304] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:52.981670 13303 leader_election.cc:304] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
W20260709 13:06:52.983098 13303 leader_election.cc:343] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.983194 13188 raft_consensus.cc:2468] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:52.983652 13303 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:52.984025 13415 raft_consensus.cc:2749] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.986457 13337 raft_consensus.cc:2468] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:52.987334 13384 raft_consensus.cc:2749] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.984062 13422 raft_consensus.cc:2804] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.988741 13422 raft_consensus.cc:493] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.989275 13422 raft_consensus.cc:3060] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.994197 13303 leader_election.cc:304] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:52.994629 13189 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:52.995465 13384 raft_consensus.cc:2804] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.997679 13229 leader_election.cc:304] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:52.997834 13383 tablet_bootstrap.cc:492] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:53.001852 13383 ts_tablet_manager.cc:1403] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.119s	user 0.035s	sys 0.065s
I20260709 13:06:53.003052 13415 raft_consensus.cc:2804] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.003759 13229 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:53.004136 13415 raft_consensus.cc:697] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.005095 13411 raft_consensus.cc:2804] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.005642 13411 raft_consensus.cc:697] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.005293 13415 consensus_queue.cc:237] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.007030 13383 raft_consensus.cc:359] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.007859 13383 raft_consensus.cc:385] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.008142 13383 raft_consensus.cc:740] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.999632 13086 catalog_manager.cc:5697] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.008985 13383 consensus_queue.cc:260] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.012770 13383 ts_tablet_manager.cc:1434] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:06:53.014307 13411 consensus_queue.cc:237] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:53.016193 13336 raft_consensus.cc:2468] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:53.016695 13385 tablet_bootstrap.cc:492] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:53.017154 13423 raft_consensus.cc:2749] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.017729 13385 ts_tablet_manager.cc:1403] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.154s	user 0.032s	sys 0.054s
I20260709 13:06:53.023249 13384 raft_consensus.cc:697] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.024514 13384 consensus_queue.cc:237] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.025456 13338 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:53.032109 13422 raft_consensus.cc:515] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.036785 13385 raft_consensus.cc:359] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.037803 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.038173 13385 raft_consensus.cc:385] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.038738 13385 raft_consensus.cc:740] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.040840 13422 leader_election.cc:290] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:53.041639 13422 raft_consensus.cc:493] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.041395 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.042138 13385 consensus_queue.cc:260] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:06:53.044252 13301 leader_election.cc:343] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.045311 13422 raft_consensus.cc:515] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.046149 13385 ts_tablet_manager.cc:1434] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:06:53.047631 13385 tablet_bootstrap.cc:492] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:53.049149 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:53.050522 13189 raft_consensus.cc:2468] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:53.038513 13188 raft_consensus.cc:3060] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.051704 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:53.052388 13303 leader_election.cc:304] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.053610 13428 raft_consensus.cc:2804] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.054379 13428 raft_consensus.cc:493] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.054834 13428 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:53.054926 13301 leader_election.cc:343] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.055999 13422 leader_election.cc:290] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:53.062651 13385 tablet_bootstrap.cc:654] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.068215 13085 catalog_manager.cc:5697] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.070410 13086 catalog_manager.cc:5697] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 0 to 1, leader changed from <none> to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.087715 13188 raft_consensus.cc:2468] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:53.089511 13303 leader_election.cc:304] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:53.090413 13385 tablet_bootstrap.cc:492] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:53.090708 13384 raft_consensus.cc:2804] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.091169 13385 ts_tablet_manager.cc:1403] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.028s
I20260709 13:06:53.091331 13384 raft_consensus.cc:697] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.073019 13428 raft_consensus.cc:515] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.092976 13384 consensus_queue.cc:237] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.095221 13385 raft_consensus.cc:359] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.096242 13385 raft_consensus.cc:385] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.096642 13385 raft_consensus.cc:740] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.097651 13385 consensus_queue.cc:260] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.098652 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.100257 13188 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.100488 13385 ts_tablet_manager.cc:1434] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.003s
I20260709 13:06:53.099402 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.102887 13385 tablet_bootstrap.cc:492] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
W20260709 13:06:53.103924 13301 leader_election.cc:343] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.106631 13428 leader_election.cc:290] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:53.113920 13385 tablet_bootstrap.cc:654] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.114928 13188 raft_consensus.cc:2468] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:53.116433 13303 leader_election.cc:304] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:53.116863 13411 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.117550 13411 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.118178 13428 raft_consensus.cc:2804] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.119143 13428 raft_consensus.cc:697] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.120128 13411 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.121227 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:53.120677 13085 catalog_manager.cc:5697] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 0 to 1, leader changed from <none> to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.121508 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:53.121992 13188 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:53.122300 13338 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:53.123942 13229 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:53.125278 13411 raft_consensus.cc:2804] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.125741 13411 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.122231 13428 consensus_queue.cc:237] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.126150 13411 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.136204 13411 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.140106 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.140949 13189 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.141074 13405 consensus_queue.cc:1048] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.145480 13411 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.147472 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.148425 13336 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.151340 13189 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:53.154423 13229 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:53.159684 13384 raft_consensus.cc:493] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.160717 13384 raft_consensus.cc:515] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.162236 13086 catalog_manager.cc:5697] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.168296 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:53.169358 13188 raft_consensus.cc:2468] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:53.167032 13411 raft_consensus.cc:2804] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.171584 13411 raft_consensus.cc:697] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.171625 13303 leader_election.cc:304] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.173329 13428 raft_consensus.cc:2804] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.173878 13428 raft_consensus.cc:493] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.172947 13411 consensus_queue.cc:237] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.166271 13384 leader_election.cc:290] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:53.178269 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:53.175945 13428 raft_consensus.cc:3060] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:53.181460 13301 leader_election.cc:343] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.182566 13405 consensus_queue.cc:1048] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.183848 13385 tablet_bootstrap.cc:492] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:53.184583 13385 ts_tablet_manager.cc:1403] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.082s	user 0.039s	sys 0.028s
I20260709 13:06:53.188771 13385 raft_consensus.cc:359] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.189723 13385 raft_consensus.cc:385] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.179129 13336 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:53.190083 13385 raft_consensus.cc:740] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.195499 13428 raft_consensus.cc:515] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.203095 13385 consensus_queue.cc:260] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.209127 13385 ts_tablet_manager.cc:1434] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.003s
I20260709 13:06:53.210914 13385 tablet_bootstrap.cc:492] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:53.208796 13085 catalog_manager.cc:5697] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 0 to 1, leader changed from <none> to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.215229 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.216496 13188 raft_consensus.cc:3060] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.219880 13385 tablet_bootstrap.cc:654] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.220324 13086 catalog_manager.cc:5697] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.232316 13188 raft_consensus.cc:2468] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:53.234385 13303 leader_election.cc:304] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.235500 13384 raft_consensus.cc:2804] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.236035 13384 raft_consensus.cc:697] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.236979 13384 consensus_queue.cc:237] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.257664 13411 raft_consensus.cc:493] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.258627 13411 raft_consensus.cc:515] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.259397 13428 leader_election.cc:290] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:53.263047 13411 leader_election.cc:290] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.264199 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:53.264225 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:53.278949 13385 tablet_bootstrap.cc:492] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:53.278858 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.280010 13188 raft_consensus.cc:2393] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eecc19d37a4940b6997f35cd9ffb574f in current term 1: Already voted for candidate ff2c85391bff48d5bafcf26c004ad30e in this term.
I20260709 13:06:53.280077 13385 ts_tablet_manager.cc:1403] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.070s	user 0.026s	sys 0.041s
W20260709 13:06:53.281003 13301 leader_election.cc:343] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.281297 13229 leader_election.cc:304] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:53.282784 13411 raft_consensus.cc:3060] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.283759 13385 raft_consensus.cc:359] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.283057 13087 catalog_manager.cc:5697] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.284651 13385 raft_consensus.cc:385] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.285110 13385 raft_consensus.cc:740] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.286149 13385 consensus_queue.cc:260] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.288887 13385 ts_tablet_manager.cc:1434] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:53.290501 13385 tablet_bootstrap.cc:492] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:53.295075 13411 raft_consensus.cc:2749] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.299651 13385 tablet_bootstrap.cc:654] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.306135 13385 tablet_bootstrap.cc:492] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:53.306798 13385 ts_tablet_manager.cc:1403] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:06:53.310173 13385 raft_consensus.cc:359] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.310921 13385 raft_consensus.cc:385] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.311276 13385 raft_consensus.cc:740] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.312037 13385 consensus_queue.cc:260] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.314406 13385 ts_tablet_manager.cc:1434] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:53.328637 13405 consensus_queue.cc:1048] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.333923 13387 consensus_queue.cc:1048] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.370302 13411 raft_consensus.cc:493] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.371004 13411 raft_consensus.cc:515] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.373915 13411 leader_election.cc:290] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.374646 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:53.375165 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:53.375530 13188 raft_consensus.cc:2393] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eecc19d37a4940b6997f35cd9ffb574f in current term 1: Already voted for candidate ff2c85391bff48d5bafcf26c004ad30e in this term.
I20260709 13:06:53.376911 13229 leader_election.cc:304] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:53.377782 13411 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.384438 13411 raft_consensus.cc:2749] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.459172 13263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
mode: GRACEFUL
new_leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:35010
I20260709 13:06:53.460022 13263 raft_consensus.cc:606] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Received request to transfer leadership to TS 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:06:53.466657 13411 raft_consensus.cc:493] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.467370 13411 raft_consensus.cc:515] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.468465 13263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
mode: GRACEFUL
new_leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:35010
I20260709 13:06:53.469584 13263 raft_consensus.cc:606] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Received request to transfer leadership to TS 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:06:53.469960 13411 leader_election.cc:290] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.471314 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:53.471817 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:53.473332 13188 raft_consensus.cc:2393] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eecc19d37a4940b6997f35cd9ffb574f in current term 1: Already voted for candidate ff2c85391bff48d5bafcf26c004ad30e in this term.
I20260709 13:06:53.475502 13229 leader_election.cc:304] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:53.476578 13411 raft_consensus.cc:3060] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.477401 13415 consensus_queue.cc:1048] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.482409 13384 consensus_queue.cc:1048] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.488713 13411 raft_consensus.cc:2749] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.496881 13338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:53.497884 13338 raft_consensus.cc:606] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:06:53.499356 13415 raft_consensus.cc:993] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: : Instructing follower 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.500022 13415 raft_consensus.cc:1081] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Signalling peer 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.504153 13422 consensus_queue.cc:1048] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.507993 13411 consensus_queue.cc:1048] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.509351 13189 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:37602
I20260709 13:06:53.510142 13189 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.510638 13189 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.518589 13338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:53.519472 13338 raft_consensus.cc:606] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:06:53.520633 13264 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:53.559305 13229 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42139, user_credentials={real_user=slave}} blocked reactor thread for 46644.9us
I20260709 13:06:53.562299 13422 consensus_queue.cc:1048] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.570420 13428 consensus_queue.cc:1048] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.571991 13189 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:53.575848 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.576385 13189 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Requested vote from peers ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:53.576709 13338 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.580257 13262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.581059 13262 raft_consensus.cc:3055] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.581457 13262 raft_consensus.cc:740] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.582403 13262 consensus_queue.cc:260] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:53.585239 13397 consensus_queue.cc:1048] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.586979 13262 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.590678 13428 consensus_queue.cc:1048] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.592681 13189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:53746
I20260709 13:06:53.606307 13262 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.608549 13189 raft_consensus.cc:606] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:53.609973 13338 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.613255 13152 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.615165 13405 raft_consensus.cc:2804] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.616647 13405 raft_consensus.cc:697] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:53.618592 13428 raft_consensus.cc:993] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.618335 13405 consensus_queue.cc:237] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:06:53.619659 13422 raft_consensus.cc:1081] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.625172 13188 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:37594
I20260709 13:06:53.626787 13188 raft_consensus.cc:493] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.627460 13188 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.631956 13337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:53.633211 13337 raft_consensus.cc:606] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:53.636202 13453 consensus_queue.cc:1048] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.651108 13338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:53.658372 13428 consensus_queue.cc:1048] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.655470 13338 raft_consensus.cc:606] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:53.669683 13428 consensus_queue.cc:1048] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.670621 13188 raft_consensus.cc:515] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.674110 13188 leader_election.cc:290] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.676297 13453 consensus_queue.cc:1048] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.700189 13085 catalog_manager.cc:5697] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 1 to 2, leader changed from eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194) to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.705461 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.706688 13338 raft_consensus.cc:3055] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.707190 13338 raft_consensus.cc:740] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.676929 13262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.708549 13262 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.708290 13338 consensus_queue.cc:260] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.710523 13338 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.714807 13189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
mode: GRACEFUL
new_leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:53746
I20260709 13:06:53.722189 13189 raft_consensus.cc:606] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Received request to transfer leadership to TS ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:53.730475 13384 raft_consensus.cc:993] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:53.731076 13384 raft_consensus.cc:1081] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:53.736011 13262 raft_consensus.cc:2468] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.743158 13152 leader_election.cc:304] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:53.744402 13405 raft_consensus.cc:2804] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.744207 13264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32834
I20260709 13:06:53.747509 13264 raft_consensus.cc:493] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.747777 13405 raft_consensus.cc:697] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:53.747941 13264 raft_consensus.cc:3060] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.749763 13405 consensus_queue.cc:237] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.755479 13422 consensus_queue.cc:1048] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.762389 13422 raft_consensus.cc:993] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.763726 13422 raft_consensus.cc:1081] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.763798 13264 raft_consensus.cc:515] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.747936 13338 raft_consensus.cc:2468] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.777140 13188 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:37594
I20260709 13:06:53.781700 13188 raft_consensus.cc:493] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.782151 13188 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.790189 13264 leader_election.cc:290] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.792143 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.793156 13336 raft_consensus.cc:3055] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.792917 13453 consensus_queue.cc:1048] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:53.798497 13188 raft_consensus.cc:515] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.799851 13189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30321d144d6140b0a45e60b81c9379d3" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.800639 13189 raft_consensus.cc:3060] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.804270 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.793660 13336 raft_consensus.cc:740] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.809043 13264 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.810351 13336 consensus_queue.cc:260] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.816602 13336 raft_consensus.cc:3060] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.804554 13086 catalog_manager.cc:5697] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.819165 13188 leader_election.cc:290] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.821624 13411 raft_consensus.cc:993] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: : Instructing follower 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.822177 13411 raft_consensus.cc:1081] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Signalling peer 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:53.824501 13187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:37602
I20260709 13:06:53.825333 13187 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.825878 13187 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.821858 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.828152 13338 raft_consensus.cc:3055] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.828619 13338 raft_consensus.cc:740] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.830001 13338 consensus_queue.cc:260] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.831588 13336 raft_consensus.cc:2468] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:53.833772 13227 leader_election.cc:304] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.835053 13453 raft_consensus.cc:2804] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.835629 13453 raft_consensus.cc:697] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.836850 13453 consensus_queue.cc:237] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.837435 13187 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.838953 13189 raft_consensus.cc:2468] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:53.831878 13338 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.841333 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:53.842100 13263 raft_consensus.cc:3055] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.842226 13187 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.842526 13263 raft_consensus.cc:740] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.843637 13263 consensus_queue.cc:260] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.843990 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.844810 13337 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.845268 13263 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.846441 13264 raft_consensus.cc:2468] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.850581 13152 leader_election.cc:304] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:53.851788 13447 raft_consensus.cc:2804] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.852566 13447 raft_consensus.cc:697] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:53.853504 13447 consensus_queue.cc:237] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.856781 13263 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.857165 13337 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.858367 13338 raft_consensus.cc:2468] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:53.859112 13152 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.862685 13447 raft_consensus.cc:2804] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.863337 13447 raft_consensus.cc:697] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:53.864799 13447 consensus_queue.cc:237] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.869902 13086 catalog_manager.cc:5697] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.878461 13084 catalog_manager.cc:5697] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 1 to 2, leader changed from eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194) to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.881263 13084 catalog_manager.cc:5697] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.920713 13467 raft_consensus.cc:993] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:53.921329 13467 raft_consensus.cc:1081] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:53.923354 13263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32834
I20260709 13:06:53.924016 13263 raft_consensus.cc:493] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.924420 13263 raft_consensus.cc:3060] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.931197 13263 raft_consensus.cc:515] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.933422 13263 leader_election.cc:290] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.934793 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.934795 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba35c9ea187e482cbeb39f6b219c7030" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.935596 13338 raft_consensus.cc:3055] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.936100 13338 raft_consensus.cc:740] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:53.936124 13187 raft_consensus.cc:3060] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.937284 13338 consensus_queue.cc:260] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.938733 13338 raft_consensus.cc:3060] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.943792 13187 raft_consensus.cc:2468] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:53.945343 13229 leader_election.cc:304] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:53.945571 13338 raft_consensus.cc:2468] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:53.946283 13453 raft_consensus.cc:2804] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.946841 13453 raft_consensus.cc:697] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.947816 13453 consensus_queue.cc:237] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.958981 13405 raft_consensus.cc:993] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:53.957916 13084 catalog_manager.cc:5697] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.959831 13405 raft_consensus.cc:1081] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:53.961859 13263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32818
I20260709 13:06:53.962468 13263 raft_consensus.cc:493] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:53.962817 13263 raft_consensus.cc:3060] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.970508 13263 raft_consensus.cc:515] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.972711 13263 leader_election.cc:290] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:53.974191 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:53.974355 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b6fbf23640a4e858af7f28bddd6ad2e" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:53.975154 13338 raft_consensus.cc:3060] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.975154 13187 raft_consensus.cc:3055] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:53.976112 13187 raft_consensus.cc:740] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:53.977509 13187 consensus_queue.cc:260] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.978989 13187 raft_consensus.cc:3060] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:53.985685 13338 raft_consensus.cc:2468] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:53.987390 13227 leader_election.cc:304] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:53.988559 13187 raft_consensus.cc:2468] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:53.988560 13453 raft_consensus.cc:2804] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:53.989493 13453 raft_consensus.cc:697] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:53.990499 13453 consensus_queue.cc:237] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:53.999732 13084 catalog_manager.cc:5697] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.091954 13447 raft_consensus.cc:993] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: : Instructing follower ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:06:54.092823 13405 raft_consensus.cc:1081] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Signalling peer ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:06:54.095178 13336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:38552
I20260709 13:06:54.096076 13336 raft_consensus.cc:493] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.096541 13336 raft_consensus.cc:3060] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.103806 13336 raft_consensus.cc:515] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:54.105786 13336 leader_election.cc:290] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:54.107098 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:54.107146 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caef7e2d7c474ee1a9b8f91327426099" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:54.107882 13187 raft_consensus.cc:3055] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.107954 13263 raft_consensus.cc:3060] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.108245 13187 raft_consensus.cc:740] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:54.109269 13187 consensus_queue.cc:260] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:54.110469 13187 raft_consensus.cc:3060] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.115435 13263 raft_consensus.cc:2468] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:06:54.117098 13301 leader_election.cc:304] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:54.117945 13467 raft_consensus.cc:2804] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.118462 13467 raft_consensus.cc:697] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:54.118820 13187 raft_consensus.cc:2468] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:06:54.119356 13467 consensus_queue.cc:237] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:54.128151 13084 catalog_manager.cc:5697] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 1 to 2, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.136207 13336 raft_consensus.cc:1275] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.138415 13405 consensus_queue.cc:1048] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.149328 13263 raft_consensus.cc:1275] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.151563 13447 consensus_queue.cc:1048] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.270843 13336 raft_consensus.cc:1275] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.272521 13453 consensus_queue.cc:1048] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.282164 13187 raft_consensus.cc:1275] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.284132 13411 consensus_queue.cc:1048] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.300045 13263 raft_consensus.cc:1275] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.301790 13447 consensus_queue.cc:1048] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.307310 13336 raft_consensus.cc:1275] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.312055 13338 raft_consensus.cc:1275] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.315425 13405 consensus_queue.cc:1048] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.318018 13405 consensus_queue.cc:1048] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.339058 13263 raft_consensus.cc:1275] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.341068 13405 consensus_queue.cc:1048] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.384459 13336 raft_consensus.cc:1275] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.386102 13405 consensus_queue.cc:1048] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.397725 13336 raft_consensus.cc:1275] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.398332 13263 raft_consensus.cc:1275] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.401336 13405 consensus_queue.cc:1048] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.402740 13470 consensus_queue.cc:1048] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.420135 13187 raft_consensus.cc:1275] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.422760 13453 consensus_queue.cc:1048] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.431559 13189 raft_consensus.cc:1275] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.434019 13453 consensus_queue.cc:1048] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.446171 13338 raft_consensus.cc:1275] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.450018 13453 consensus_queue.cc:1048] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.533404 13187 raft_consensus.cc:1275] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.535212 13467 consensus_queue.cc:1048] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:54.547302 13263 raft_consensus.cc:1275] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.550171 13476 consensus_queue.cc:1048] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:56.773200 13086 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60848:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:56.780128 13086 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:56.876467 13178 tablet_service.cc:1511] Processing CreateTablet for tablet d1ea922ffb9f4f3497be21b8e2ffc4e7 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:56.876765 13249 tablet_service.cc:1511] Processing CreateTablet for tablet d1ea922ffb9f4f3497be21b8e2ffc4e7 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:56.879666 13249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ea922ffb9f4f3497be21b8e2ffc4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.885768 13323 tablet_service.cc:1511] Processing CreateTablet for tablet d1ea922ffb9f4f3497be21b8e2ffc4e7 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:56.887951 13173 tablet_service.cc:1511] Processing CreateTablet for tablet d4b2212ec052479c8be0c5e7fada3c47 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:56.893040 13251 tablet_service.cc:1511] Processing CreateTablet for tablet d4b2212ec052479c8be0c5e7fada3c47 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:56.895399 13176 tablet_service.cc:1511] Processing CreateTablet for tablet 127bb154d697451db026a06b2507cc7a (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:56.897001 13325 tablet_service.cc:1511] Processing CreateTablet for tablet d4b2212ec052479c8be0c5e7fada3c47 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:56.902362 13179 tablet_service.cc:1511] Processing CreateTablet for tablet a016a7f78fc34769933e2e7eb10eb89e (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:56.879666 13178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ea922ffb9f4f3497be21b8e2ffc4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.910459 13252 tablet_service.cc:1511] Processing CreateTablet for tablet 127bb154d697451db026a06b2507cc7a (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:56.910668 13175 tablet_service.cc:1511] Processing CreateTablet for tablet 9a407ca0fe144cc78e7b1e96f96ac585 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:56.912002 13173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4b2212ec052479c8be0c5e7fada3c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.913137 13175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a407ca0fe144cc78e7b1e96f96ac585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.914708 13176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127bb154d697451db026a06b2507cc7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.915822 13252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127bb154d697451db026a06b2507cc7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.921278 13322 tablet_service.cc:1511] Processing CreateTablet for tablet a016a7f78fc34769933e2e7eb10eb89e (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:56.917954 13174 tablet_service.cc:1511] Processing CreateTablet for tablet 4c61c027bd134db48f7fdf4cf845e009 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:56.923729 13174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c61c027bd134db48f7fdf4cf845e009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.936895 13251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4b2212ec052479c8be0c5e7fada3c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.914574 13248 tablet_service.cc:1511] Processing CreateTablet for tablet a016a7f78fc34769933e2e7eb10eb89e (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:56.940440 13248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a016a7f78fc34769933e2e7eb10eb89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.941007 13177 tablet_service.cc:1511] Processing CreateTablet for tablet 427001fe3c8244c5a6519c100eaad393 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:56.945014 13172 tablet_service.cc:1511] Processing CreateTablet for tablet 64112aad1a0a4275b6edaa9ece28a9d4 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:56.949302 13171 tablet_service.cc:1511] Processing CreateTablet for tablet 9dfcbf3c5e504e68bee46c6d02010c38 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:56.954295 13179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a016a7f78fc34769933e2e7eb10eb89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.954036 13321 tablet_service.cc:1511] Processing CreateTablet for tablet 4c61c027bd134db48f7fdf4cf845e009 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:56.951710 13328 tablet_service.cc:1511] Processing CreateTablet for tablet 9a407ca0fe144cc78e7b1e96f96ac585 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:56.907156 13324 tablet_service.cc:1511] Processing CreateTablet for tablet 127bb154d697451db026a06b2507cc7a (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:56.959524 13323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ea922ffb9f4f3497be21b8e2ffc4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.961050 13322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a016a7f78fc34769933e2e7eb10eb89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.967813 13324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127bb154d697451db026a06b2507cc7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.969125 13328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a407ca0fe144cc78e7b1e96f96ac585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:56.972065 13325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4b2212ec052479c8be0c5e7fada3c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.001179 13177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427001fe3c8244c5a6519c100eaad393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.005612 13172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64112aad1a0a4275b6edaa9ece28a9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.008569 13171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dfcbf3c5e504e68bee46c6d02010c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.009975 13321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c61c027bd134db48f7fdf4cf845e009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.019311 13327 tablet_service.cc:1511] Processing CreateTablet for tablet 427001fe3c8244c5a6519c100eaad393 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:57.021250 13327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427001fe3c8244c5a6519c100eaad393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.034134 13326 tablet_service.cc:1511] Processing CreateTablet for tablet 64112aad1a0a4275b6edaa9ece28a9d4 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:57.036566 13326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64112aad1a0a4275b6edaa9ece28a9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.042219 13247 tablet_service.cc:1511] Processing CreateTablet for tablet 4c61c027bd134db48f7fdf4cf845e009 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:57.044073 13247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c61c027bd134db48f7fdf4cf845e009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.046013 13254 tablet_service.cc:1511] Processing CreateTablet for tablet 64112aad1a0a4275b6edaa9ece28a9d4 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:57.049199 13246 tablet_service.cc:1511] Processing CreateTablet for tablet 9dfcbf3c5e504e68bee46c6d02010c38 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:57.053329 13253 tablet_service.cc:1511] Processing CreateTablet for tablet 9a407ca0fe144cc78e7b1e96f96ac585 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:57.042281 13250 tablet_service.cc:1511] Processing CreateTablet for tablet 427001fe3c8244c5a6519c100eaad393 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:57.067442 13246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dfcbf3c5e504e68bee46c6d02010c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.068666 13250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427001fe3c8244c5a6519c100eaad393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.069548 13254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64112aad1a0a4275b6edaa9ece28a9d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.069202 13320 tablet_service.cc:1511] Processing CreateTablet for tablet 9dfcbf3c5e504e68bee46c6d02010c38 (DEFAULT_TABLE table=soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:57.071082 13320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dfcbf3c5e504e68bee46c6d02010c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.072963 13253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a407ca0fe144cc78e7b1e96f96ac585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:57.099386 13491 tablet_bootstrap.cc:492] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:57.137970 13491 tablet_bootstrap.cc:654] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.188656 13492 tablet_bootstrap.cc:492] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.191179 13493 tablet_bootstrap.cc:492] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.243264 13491 tablet_bootstrap.cc:492] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:57.244705 13491 ts_tablet_manager.cc:1403] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.146s	user 0.016s	sys 0.005s
I20260709 13:06:57.249099 13493 tablet_bootstrap.cc:654] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.251770 13492 tablet_bootstrap.cc:654] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.266487 13491 raft_consensus.cc:359] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.283387 13491 raft_consensus.cc:385] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.287971 13493 tablet_bootstrap.cc:492] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:57.288826 13493 ts_tablet_manager.cc:1403] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.102s	user 0.009s	sys 0.013s
I20260709 13:06:57.283994 13491 raft_consensus.cc:740] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.292970 13491 consensus_queue.cc:260] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.293555 13493 raft_consensus.cc:359] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.294417 13493 raft_consensus.cc:385] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.294912 13493 raft_consensus.cc:740] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.295928 13491 ts_tablet_manager.cc:1434] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.003s
I20260709 13:06:57.296223 13493 consensus_queue.cc:260] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.297307 13491 tablet_bootstrap.cc:492] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:57.299778 13492 tablet_bootstrap.cc:492] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.300483 13492 ts_tablet_manager.cc:1403] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.112s	user 0.018s	sys 0.004s
I20260709 13:06:57.304710 13492 raft_consensus.cc:359] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.305686 13492 raft_consensus.cc:385] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.306053 13492 raft_consensus.cc:740] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.307529 13491 tablet_bootstrap.cc:654] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.310480 13493 ts_tablet_manager.cc:1434] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.001s
I20260709 13:06:57.316424 13493 tablet_bootstrap.cc:492] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.315285 13492 consensus_queue.cc:260] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.322469 13492 ts_tablet_manager.cc:1434] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.004s
I20260709 13:06:57.324141 13492 tablet_bootstrap.cc:492] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.328401 13491 tablet_bootstrap.cc:492] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:57.329721 13491 ts_tablet_manager.cc:1403] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.013s
I20260709 13:06:57.334064 13491 raft_consensus.cc:359] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.335409 13492 tablet_bootstrap.cc:654] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.335631 13491 raft_consensus.cc:385] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.336988 13491 raft_consensus.cc:740] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.337081 13493 tablet_bootstrap.cc:654] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.339170 13491 consensus_queue.cc:260] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.347174 13492 tablet_bootstrap.cc:492] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.347952 13492 ts_tablet_manager.cc:1403] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:06:57.352895 13491 ts_tablet_manager.cc:1434] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.005s
I20260709 13:06:57.352509 13492 raft_consensus.cc:359] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.354221 13492 raft_consensus.cc:385] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.354420 13491 tablet_bootstrap.cc:492] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:57.354568 13492 raft_consensus.cc:740] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.356171 13492 consensus_queue.cc:260] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.359578 13492 ts_tablet_manager.cc:1434] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:06:57.363018 13493 tablet_bootstrap.cc:492] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:57.363590 13491 tablet_bootstrap.cc:654] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.363646 13493 ts_tablet_manager.cc:1403] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.009s
I20260709 13:06:57.364068 13492 tablet_bootstrap.cc:492] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.368647 13493 raft_consensus.cc:359] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.369863 13493 raft_consensus.cc:385] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.370363 13493 raft_consensus.cc:740] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.372206 13493 consensus_queue.cc:260] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.375891 13492 tablet_bootstrap.cc:654] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.376468 13493 ts_tablet_manager.cc:1434] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.002s
I20260709 13:06:57.381814 13493 tablet_bootstrap.cc:492] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.384222 13476 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.385125 13476 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.390125 13493 tablet_bootstrap.cc:654] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.392719 13492 tablet_bootstrap.cc:492] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.393659 13492 ts_tablet_manager.cc:1403] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.033s	user 0.019s	sys 0.000s
I20260709 13:06:57.395138 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
W20260709 13:06:57.401312 13135 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:06:57.402488 13303 leader_election.cc:343] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.438858 13491 tablet_bootstrap.cc:492] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:57.440157 13491 ts_tablet_manager.cc:1403] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.014s
I20260709 13:06:57.443105 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.443694 13476 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
W20260709 13:06:57.445479 13301 leader_election.cc:343] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.446098 13301 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:57.447608 13476 raft_consensus.cc:2749] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.450614 13492 raft_consensus.cc:359] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.451049 13491 raft_consensus.cc:359] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.451848 13492 raft_consensus.cc:385] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.452015 13491 raft_consensus.cc:385] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.452320 13492 raft_consensus.cc:740] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.452488 13491 raft_consensus.cc:740] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.452795 13493 tablet_bootstrap.cc:492] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:57.453625 13493 ts_tablet_manager.cc:1403] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.019s
I20260709 13:06:57.453732 13491 consensus_queue.cc:260] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.453853 13492 consensus_queue.cc:260] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.458047 13480 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.459172 13480 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.459409 13493 raft_consensus.cc:359] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.460701 13493 raft_consensus.cc:385] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.461187 13493 raft_consensus.cc:740] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.462194 13493 consensus_queue.cc:260] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.464039 13492 ts_tablet_manager.cc:1434] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.070s	user 0.017s	sys 0.005s
I20260709 13:06:57.465372 13492 tablet_bootstrap.cc:492] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.465317 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:57.466583 13480 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.465387 13491 ts_tablet_manager.cc:1434] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.005s
I20260709 13:06:57.467442 13493 ts_tablet_manager.cc:1434] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
W20260709 13:06:57.468837 13227 leader_election.cc:343] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.469218 13493 tablet_bootstrap.cc:492] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.475726 13492 tablet_bootstrap.cc:654] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.478960 13493 tablet_bootstrap.cc:654] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.485807 13492 tablet_bootstrap.cc:492] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.486563 13492 ts_tablet_manager.cc:1403] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:06:57.491063 13492 raft_consensus.cc:359] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.492491 13492 raft_consensus.cc:385] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.492856 13492 raft_consensus.cc:740] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.493999 13492 consensus_queue.cc:260] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.496605 13493 tablet_bootstrap.cc:492] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:57.497052 13492 ts_tablet_manager.cc:1434] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 13:06:57.497392 13493 ts_tablet_manager.cc:1403] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
I20260709 13:06:57.498603 13492 tablet_bootstrap.cc:492] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.506526 13493 raft_consensus.cc:359] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.508088 13493 raft_consensus.cc:385] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.508590 13493 raft_consensus.cc:740] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.509562 13493 consensus_queue.cc:260] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.510401 13492 tablet_bootstrap.cc:654] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.512624 13493 ts_tablet_manager.cc:1434] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:06:57.514142 13493 tablet_bootstrap.cc:492] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.518436 13492 tablet_bootstrap.cc:492] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.519337 13492 ts_tablet_manager.cc:1403] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.004s
I20260709 13:06:57.522925 13493 tablet_bootstrap.cc:654] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.523787 13405 raft_consensus.cc:493] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.524557 13405 raft_consensus.cc:515] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.524617 13492 raft_consensus.cc:359] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.525753 13492 raft_consensus.cc:385] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.526211 13492 raft_consensus.cc:740] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.527645 13492 consensus_queue.cc:260] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.528963 13405 leader_election.cc:290] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.529902 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.531527 13492 ts_tablet_manager.cc:1434] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
W20260709 13:06:57.532379 13229 leader_election.cc:343] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.532943 13492 tablet_bootstrap.cc:492] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.533622 13493 tablet_bootstrap.cc:492] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:57.532945 13229 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:57.534302 13493 ts_tablet_manager.cc:1403] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 13:06:57.535758 13478 raft_consensus.cc:2749] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.538969 13493 raft_consensus.cc:359] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.542634 13493 raft_consensus.cc:385] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.542999 13493 raft_consensus.cc:740] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.543956 13492 tablet_bootstrap.cc:654] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.544420 13493 consensus_queue.cc:260] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.547858 13405 raft_consensus.cc:493] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.548503 13405 raft_consensus.cc:515] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.550880 13405 leader_election.cc:290] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.551638 13405 raft_consensus.cc:493] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.532939 13491 tablet_bootstrap.cc:492] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:57.552134 13405 raft_consensus.cc:515] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.577412 13405 leader_election.cc:290] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.608199 13493 ts_tablet_manager.cc:1434] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.073s	user 0.026s	sys 0.006s
I20260709 13:06:57.760921 13491 tablet_bootstrap.cc:654] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.761890 13493 tablet_bootstrap.cc:492] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.707307 13476 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.722411 13262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.765103 13476 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.743070 13492 tablet_bootstrap.cc:492] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.753506 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.754478 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:57.770645 13259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.755024 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:57.724217 13508 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.773087 13301 leader_election.cc:343] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.775719 13508 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.776731 13492 ts_tablet_manager.cc:1403] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.244s	user 0.030s	sys 0.042s
I20260709 13:06:57.778656 13508 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.773594 13337 raft_consensus.cc:2468] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:57.780915 13511 raft_consensus.cc:493] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.708205 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.781644 13511 raft_consensus.cc:515] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.782124 13261 raft_consensus.cc:2468] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:57.780723 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.783390 13187 raft_consensus.cc:2468] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:57.784004 13493 tablet_bootstrap.cc:654] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.725943 13480 raft_consensus.cc:493] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.785012 13480 raft_consensus.cc:515] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.707962 13510 raft_consensus.cc:493] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.789279 13510 raft_consensus.cc:515] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.791199 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
W20260709 13:06:57.793802 13227 leader_election.cc:343] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.768082 13476 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.794018 13511 leader_election.cc:290] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.755612 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:57.793824 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
W20260709 13:06:57.798008 13301 leader_election.cc:343] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.798403 13510 leader_election.cc:290] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.800449 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.801795 13261 raft_consensus.cc:2468] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:57.803687 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.803848 13301 leader_election.cc:304] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
W20260709 13:06:57.806262 13229 leader_election.cc:343] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.806903 13229 leader_election.cc:304] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:57.796068 13335 raft_consensus.cc:2468] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:57.807946 13478 raft_consensus.cc:2749] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.809244 13510 raft_consensus.cc:2804] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.809913 13491 tablet_bootstrap.cc:492] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:57.808228 13480 leader_election.cc:290] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.810559 13491 ts_tablet_manager.cc:1403] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.278s	user 0.017s	sys 0.011s
I20260709 13:06:57.812934 13476 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.813637 13476 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.815102 13491 raft_consensus.cc:359] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.816004 13491 raft_consensus.cc:385] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.816282 13476 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.816370 13491 raft_consensus.cc:740] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.818017 13491 consensus_queue.cc:260] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.821638 13510 raft_consensus.cc:493] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.822258 13510 raft_consensus.cc:3060] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.823957 13491 ts_tablet_manager.cc:1434] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:06:57.826040 13491 tablet_bootstrap.cc:492] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:57.827728 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.829169 13261 raft_consensus.cc:2468] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
W20260709 13:06:57.780800 13152 leader_election.cc:343] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:57.833485 13152 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46015, user_credentials={real_user=slave}} blocked reactor thread for 52899.1us
W20260709 13:06:57.834834 13152 leader_election.cc:343] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.835865 13491 tablet_bootstrap.cc:654] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.838762 13510 raft_consensus.cc:515] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.835721 13152 leader_election.cc:304] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:57.841781 13301 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:57.843194 13507 raft_consensus.cc:2804] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.843667 13507 raft_consensus.cc:493] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.843770 13511 raft_consensus.cc:2804] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.844122 13507 raft_consensus.cc:3060] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.844440 13511 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.844962 13511 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:57.845816 13152 leader_election.cc:343] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.846531 13152 leader_election.cc:304] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31; no voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:57.845906 13492 raft_consensus.cc:359] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.848232 13492 raft_consensus.cc:385] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.848902 13492 raft_consensus.cc:740] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.849493 13491 tablet_bootstrap.cc:492] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:57.850297 13491 ts_tablet_manager.cc:1403] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.013s
I20260709 13:06:57.850235 13508 raft_consensus.cc:2749] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.852082 13152 leader_election.cc:304] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:57.855914 13508 raft_consensus.cc:2804] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.855852 13491 raft_consensus.cc:359] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.856967 13508 raft_consensus.cc:493] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.857630 13508 raft_consensus.cc:3060] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.859688 13507 raft_consensus.cc:515] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.860615 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:57.861449 13337 raft_consensus.cc:2468] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:57.862221 13507 leader_election.cc:290] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.863281 13510 leader_election.cc:290] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.853822 13492 consensus_queue.cc:260] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.866564 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:57.868489 13491 raft_consensus.cc:385] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.868944 13491 raft_consensus.cc:740] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.871642 13303 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:57.872843 13492 ts_tablet_manager.cc:1434] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.095s	user 0.011s	sys 0.000s
I20260709 13:06:57.869675 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:57.874336 13493 tablet_bootstrap.cc:492] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:57.875249 13261 raft_consensus.cc:3060] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.870172 13491 consensus_queue.cc:260] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.875608 13493 ts_tablet_manager.cc:1403] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.114s	user 0.033s	sys 0.044s
I20260709 13:06:57.874614 13510 raft_consensus.cc:2804] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.880877 13510 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.881311 13510 raft_consensus.cc:3060] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.881433 13491 ts_tablet_manager.cc:1434] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.005s
I20260709 13:06:57.883123 13491 tablet_bootstrap.cc:492] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:57.886727 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.888785 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.889848 13188 raft_consensus.cc:2393] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff2c85391bff48d5bafcf26c004ad30e in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:57.873958 13508 raft_consensus.cc:515] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.893738 13492 tablet_bootstrap.cc:492] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:57.894143 13187 raft_consensus.cc:2393] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff2c85391bff48d5bafcf26c004ad30e in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:57.895948 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.898452 13508 leader_election.cc:290] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.901084 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:57.903474 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:57.904476 13188 raft_consensus.cc:2393] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ff2c85391bff48d5bafcf26c004ad30e in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:57.904551 13337 raft_consensus.cc:2393] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d05e27c97034a909efec0f5dd6b4b31 in current term 1: Already voted for candidate ff2c85391bff48d5bafcf26c004ad30e in this term.
I20260709 13:06:57.904320 13493 raft_consensus.cc:359] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.905181 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:57.905478 13493 raft_consensus.cc:385] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.905906 13493 raft_consensus.cc:740] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.903594 13511 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.906919 13493 consensus_queue.cc:260] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:06:57.909011 13303 leader_election.cc:343] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.909978 13493 ts_tablet_manager.cc:1434] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:06:57.911475 13493 tablet_bootstrap.cc:492] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.912372 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:57.915367 13511 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.920203 13261 raft_consensus.cc:2468] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:57.925647 13493 tablet_bootstrap.cc:654] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.926211 13301 leader_election.cc:304] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:57.927270 13511 raft_consensus.cc:2804] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.927830 13511 raft_consensus.cc:697] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:57.928936 13511 consensus_queue.cc:237] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.929771 13510 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.932605 13510 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:57.903483 13492 tablet_bootstrap.cc:654] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.920953 13491 tablet_bootstrap.cc:654] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.921248 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:57.934085 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:57.934556 13264 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:57.935789 13301 leader_election.cc:343] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.936308 13478 raft_consensus.cc:493] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.936929 13478 raft_consensus.cc:515] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.939726 13478 leader_election.cc:290] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.923085 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:57.941951 13337 raft_consensus.cc:3060] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.940981 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:57.940943 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:57.944813 13188 raft_consensus.cc:2468] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:57.946525 13229 leader_election.cc:304] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:57.947556 13478 raft_consensus.cc:2804] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.933137 13493 tablet_bootstrap.cc:492] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
W20260709 13:06:57.948117 13227 leader_election.cc:343] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.948120 13478 raft_consensus.cc:493] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.949096 13478 raft_consensus.cc:3060] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.949323 13493 ts_tablet_manager.cc:1403] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.014s
I20260709 13:06:57.924683 13152 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:57.952495 13508 raft_consensus.cc:2804] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.953073 13508 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.953753 13508 raft_consensus.cc:3060] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.957876 13493 raft_consensus.cc:359] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.957664 13264 raft_consensus.cc:2468] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:57.961432 13493 raft_consensus.cc:385] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.962373 13493 raft_consensus.cc:740] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.964097 13493 consensus_queue.cc:260] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.962452 13301 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:57.962046 13478 raft_consensus.cc:515] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.968093 13492 tablet_bootstrap.cc:492] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:57.953012 13337 raft_consensus.cc:2468] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
W20260709 13:06:57.953012 13152 leader_election.cc:343] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.981360 13493 ts_tablet_manager.cc:1434] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.004s
I20260709 13:06:57.982077 13152 leader_election.cc:304] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31; no voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:57.982844 13493 tablet_bootstrap.cc:492] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:57.983222 13507 raft_consensus.cc:2749] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.983346 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:57.984246 13492 ts_tablet_manager.cc:1403] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.091s	user 0.020s	sys 0.008s
I20260709 13:06:57.984578 13188 raft_consensus.cc:3060] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:57.985220 13152 leader_election.cc:343] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.986809 13303 leader_election.cc:304] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:57.975412 13085 catalog_manager.cc:5697] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.978152 13508 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.991173 13508 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:57.993389 13492 raft_consensus.cc:359] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:57.994388 13492 raft_consensus.cc:385] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.994628 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:57.997084 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:57.998198 13187 raft_consensus.cc:2393] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ff2c85391bff48d5bafcf26c004ad30e in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:58.000973 13491 tablet_bootstrap.cc:492] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:57.994689 13492 raft_consensus.cc:740] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.001688 13491 ts_tablet_manager.cc:1403] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.119s	user 0.014s	sys 0.017s
I20260709 13:06:58.002409 13492 consensus_queue.cc:260] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.003733 13510 raft_consensus.cc:2749] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.004884 13478 leader_election.cc:290] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.005744 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:57.999321 13152 leader_election.cc:304] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.008247 13510 raft_consensus.cc:2804] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.008870 13510 raft_consensus.cc:697] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
W20260709 13:06:58.009043 13227 leader_election.cc:343] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.010177 13510 consensus_queue.cc:237] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:06:58.010816 13303 leader_election.cc:343] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.010692 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.012118 13303 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.012436 13337 raft_consensus.cc:2393] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d05e27c97034a909efec0f5dd6b4b31 in current term 1: Already voted for candidate ff2c85391bff48d5bafcf26c004ad30e in this term.
I20260709 13:06:58.013464 13476 raft_consensus.cc:2749] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.010020 13508 raft_consensus.cc:2804] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.014354 13508 raft_consensus.cc:697] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.015726 13508 consensus_queue.cc:237] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.011032 13491 raft_consensus.cc:359] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.006551 13188 raft_consensus.cc:2468] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
W20260709 13:06:58.016526 13301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:46015, user_credentials={real_user=slave}} blocked reactor thread for 54209.5us
I20260709 13:06:58.020903 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.023425 13491 raft_consensus.cc:385] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.030705 13491 raft_consensus.cc:740] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.031734 13493 tablet_bootstrap.cc:654] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.031709 13491 consensus_queue.cc:260] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.037464 13264 raft_consensus.cc:3060] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.038399 13229 leader_election.cc:304] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.040058 13478 raft_consensus.cc:2804] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.040683 13478 raft_consensus.cc:697] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.041585 13478 consensus_queue.cc:237] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.032879 13492 ts_tablet_manager.cc:1434] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:06:58.045976 13492 tablet_bootstrap.cc:492] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:58.050071 13086 catalog_manager.cc:5697] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.058359 13492 tablet_bootstrap.cc:654] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.058595 13264 raft_consensus.cc:2468] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:58.059810 13480 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.061571 13480 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.067914 13508 raft_consensus.cc:493] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.068259 13493 tablet_bootstrap.cc:492] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:58.068590 13491 ts_tablet_manager.cc:1434] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.066s	user 0.012s	sys 0.003s
I20260709 13:06:58.068851 13493 ts_tablet_manager.cc:1403] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.086s	user 0.023s	sys 0.003s
I20260709 13:06:58.068822 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.069962 13491 tablet_bootstrap.cc:492] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:58.068655 13508 raft_consensus.cc:515] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.070223 13337 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:58.072147 13227 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:58.073022 13493 raft_consensus.cc:359] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.074081 13493 raft_consensus.cc:385] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.074349 13493 raft_consensus.cc:740] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.069113 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:58.078812 13493 consensus_queue.cc:260] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.079939 13188 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 0.
I20260709 13:06:58.073109 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:58.082976 13493 ts_tablet_manager.cc:1434] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:06:58.078912 13480 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.084199 13493 tablet_bootstrap.cc:492] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: Bootstrap starting.
I20260709 13:06:58.084471 13152 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.086319 13508 leader_election.cc:290] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.087116 13508 raft_consensus.cc:2804] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:58.087862 13152 leader_election.cc:343] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.087038 13084 catalog_manager.cc:5697] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 0 to 1, leader changed from <none> to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.088141 13508 raft_consensus.cc:697] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.089591 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.090425 13337 raft_consensus.cc:2468] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:58.089499 13508 consensus_queue.cc:237] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.091866 13152 leader_election.cc:304] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.093041 13507 raft_consensus.cc:2804] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.093501 13507 raft_consensus.cc:493] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.093873 13507 raft_consensus.cc:3060] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.094079 13478 raft_consensus.cc:2804] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.094601 13478 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.094995 13493 tablet_bootstrap.cc:654] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.094998 13478 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.097760 13491 tablet_bootstrap.cc:654] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.125043 13507 raft_consensus.cc:515] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.132637 13507 leader_election.cc:290] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.131729 13508 raft_consensus.cc:493] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.125216 13478 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.136411 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.137454 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.138371 13337 raft_consensus.cc:3060] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:58.139525 13152 leader_election.cc:343] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.143755 13493 tablet_bootstrap.cc:492] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: No bootstrap required, opened a new log
I20260709 13:06:58.144505 13493 ts_tablet_manager.cc:1403] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.018s
I20260709 13:06:58.145210 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.144575 13508 raft_consensus.cc:515] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.146173 13188 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.148042 13493 raft_consensus.cc:359] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.148981 13493 raft_consensus.cc:385] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.149274 13493 raft_consensus.cc:740] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.151022 13493 consensus_queue.cc:260] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.155746 13493 ts_tablet_manager.cc:1434] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 13:06:58.157085 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:58.158943 13261 raft_consensus.cc:2468] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:58.161059 13508 leader_election.cc:290] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.162190 13152 leader_election.cc:304] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:58.162636 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.163446 13335 raft_consensus.cc:2468] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 0.
I20260709 13:06:58.165691 13405 raft_consensus.cc:2804] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.166283 13405 raft_consensus.cc:493] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.167063 13478 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.167140 13405 raft_consensus.cc:3060] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.169942 13491 tablet_bootstrap.cc:492] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:58.169799 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.170735 13491 ts_tablet_manager.cc:1403] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.101s	user 0.014s	sys 0.013s
I20260709 13:06:58.170841 13335 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.177508 13492 tablet_bootstrap.cc:492] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:58.178181 13492 ts_tablet_manager.cc:1403] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.133s	user 0.022s	sys 0.010s
I20260709 13:06:58.181932 13188 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:58.181721 13405 raft_consensus.cc:515] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.187130 13337 raft_consensus.cc:2468] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:58.188160 13229 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:58.181777 13492 raft_consensus.cc:359] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.189805 13405 leader_election.cc:290] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.190161 13492 raft_consensus.cc:385] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.190505 13492 raft_consensus.cc:740] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.182837 13335 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 1.
I20260709 13:06:58.184156 13491 raft_consensus.cc:359] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.192862 13491 raft_consensus.cc:385] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.192747 13152 leader_election.cc:304] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.193337 13491 raft_consensus.cc:740] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.194211 13491 consensus_queue.cc:260] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.196950 13491 ts_tablet_manager.cc:1434] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 13:06:58.196795 13478 raft_consensus.cc:2804] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.191835 13492 consensus_queue.cc:260] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.196108 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.202657 13264 raft_consensus.cc:3060] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.203518 13492 ts_tablet_manager.cc:1434] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:06:58.204898 13492 tablet_bootstrap.cc:492] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: Bootstrap starting.
I20260709 13:06:58.206863 13491 tablet_bootstrap.cc:492] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:58.207647 13086 catalog_manager.cc:5697] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.215039 13405 raft_consensus.cc:2804] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.215339 13491 tablet_bootstrap.cc:654] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.215518 13264 raft_consensus.cc:2468] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:58.216490 13405 raft_consensus.cc:697] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.217468 13152 leader_election.cc:304] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:58.218245 13478 raft_consensus.cc:697] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.219466 13478 consensus_queue.cc:237] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.217671 13405 consensus_queue.cc:237] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.221835 13492 tablet_bootstrap.cc:654] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.219208 13508 raft_consensus.cc:2804] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.230772 13508 raft_consensus.cc:697] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.231052 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.230245 13085 catalog_manager.cc:5697] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.232148 13508 consensus_queue.cc:237] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.235697 13526 raft_consensus.cc:493] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.236449 13526 raft_consensus.cc:515] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.238642 13491 tablet_bootstrap.cc:492] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:58.239255 13491 ts_tablet_manager.cc:1403] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.006s
I20260709 13:06:58.242862 13491 raft_consensus.cc:359] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.243686 13491 raft_consensus.cc:385] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.243965 13491 raft_consensus.cc:740] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.247206 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.247587 13492 tablet_bootstrap.cc:492] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: No bootstrap required, opened a new log
I20260709 13:06:58.248332 13492 ts_tablet_manager.cc:1403] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.006s
I20260709 13:06:58.252058 13492 raft_consensus.cc:359] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.252988 13492 raft_consensus.cc:385] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.252611 13526 leader_election.cc:290] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.255908 13337 raft_consensus.cc:2393] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eecc19d37a4940b6997f35cd9ffb574f in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:58.253319 13492 raft_consensus.cc:740] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.257117 13492 consensus_queue.cc:260] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.253934 13491 consensus_queue.cc:260] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.259620 13491 ts_tablet_manager.cc:1434] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.001s
I20260709 13:06:58.260998 13491 tablet_bootstrap.cc:492] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: Bootstrap starting.
I20260709 13:06:58.265168 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:58.266292 13507 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.267210 13507 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.267834 13229 leader_election.cc:304] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.269418 13480 raft_consensus.cc:3060] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.233841 13335 raft_consensus.cc:3060] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.272825 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:58.269848 13507 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.274178 13264 raft_consensus.cc:2393] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6d05e27c97034a909efec0f5dd6b4b31 in current term 1: Already voted for candidate ff2c85391bff48d5bafcf26c004ad30e in this term.
I20260709 13:06:58.275477 13492 ts_tablet_manager.cc:1434] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:06:58.268185 13491 tablet_bootstrap.cc:654] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.274227 13510 raft_consensus.cc:493] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.277523 13510 raft_consensus.cc:515] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.273043 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.283538 13152 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31; no voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.285425 13507 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.297703 13510 leader_election.cc:290] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:58.298482 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:06:58.298921 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
W20260709 13:06:58.300256 13301 leader_election.cc:343] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.299801 13188 raft_consensus.cc:2468] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 0.
I20260709 13:06:58.302636 13303 leader_election.cc:304] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.304420 13510 raft_consensus.cc:2804] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.304973 13510 raft_consensus.cc:493] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.305544 13510 raft_consensus.cc:3060] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.316795 13510 raft_consensus.cc:515] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.321303 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.322104 13188 raft_consensus.cc:3060] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.322698 13510 leader_election.cc:290] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:58.317358 13480 raft_consensus.cc:2749] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.317978 13507 raft_consensus.cc:2749] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.321173 13335 raft_consensus.cc:2468] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 1.
I20260709 13:06:58.325784 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.325083 13087 catalog_manager.cc:5697] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:58.328111 13301 leader_election.cc:343] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.328838 13087 catalog_manager.cc:5697] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.330745 13084 catalog_manager.cc:5697] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 0 to 1, leader changed from <none> to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.335006 13188 raft_consensus.cc:2468] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
I20260709 13:06:58.337409 13303 leader_election.cc:304] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.338564 13491 tablet_bootstrap.cc:492] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: No bootstrap required, opened a new log
I20260709 13:06:58.338789 13510 raft_consensus.cc:2804] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.339324 13491 ts_tablet_manager.cc:1403] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: Time spent bootstrapping tablet: real 0.079s	user 0.022s	sys 0.044s
I20260709 13:06:58.339799 13510 raft_consensus.cc:697] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:58.340848 13510 consensus_queue.cc:237] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.343971 13491 raft_consensus.cc:359] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.344957 13491 raft_consensus.cc:385] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.345247 13491 raft_consensus.cc:740] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.346058 13491 consensus_queue.cc:260] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.348534 13491 ts_tablet_manager.cc:1434] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.002s
I20260709 13:06:58.354889 13511 consensus_queue.cc:1048] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.360229 13086 catalog_manager.cc:5697] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 0 to 1, leader changed from <none> to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.373921 13510 consensus_queue.cc:1048] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.400394 13480 raft_consensus.cc:493] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.400949 13480 raft_consensus.cc:515] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.403172 13480 leader_election.cc:290] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.404611 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:58.404783 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.405911 13335 raft_consensus.cc:2393] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eecc19d37a4940b6997f35cd9ffb574f in current term 1: Already voted for candidate 6d05e27c97034a909efec0f5dd6b4b31 in this term.
I20260709 13:06:58.407811 13227 leader_election.cc:304] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.409291 13480 raft_consensus.cc:3060] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.415518 13510 consensus_queue.cc:1048] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.416280 13264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
mode: GRACEFUL
new_leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:35010
I20260709 13:06:58.417280 13264 raft_consensus.cc:606] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Received request to transfer leadership to TS 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:06:58.419054 13480 raft_consensus.cc:2749] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.424947 13335 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "127bb154d697451db026a06b2507cc7a"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:58.425772 13335 raft_consensus.cc:606] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:06:58.435137 13511 consensus_queue.cc:1048] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.439087 13187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:53746
I20260709 13:06:58.439852 13187 raft_consensus.cc:606] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.448858 13335 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:58.449620 13335 raft_consensus.cc:606] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.453230 13507 consensus_queue.cc:1048] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.458202 13476 raft_consensus.cc:993] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:58.458330 13188 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:53746
I20260709 13:06:58.459136 13188 raft_consensus.cc:606] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.459707 13510 raft_consensus.cc:1081] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:58.461825 13188 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "127bb154d697451db026a06b2507cc7a"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:37594
I20260709 13:06:58.462670 13188 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.463119 13188 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.473997 13188 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.477138 13188 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.479473 13335 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
mode: GRACEFUL
new_leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:06:58.480470 13335 raft_consensus.cc:606] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Received request to transfer leadership to TS eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:06:58.481900 13512 consensus_queue.cc:1048] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.486747 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.487560 13264 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.498320 13262 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
mode: GRACEFUL
new_leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:35010
I20260709 13:06:58.499253 13262 raft_consensus.cc:606] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Received request to transfer leadership to TS ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.502381 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.503197 13335 raft_consensus.cc:3055] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.502967 13480 consensus_queue.cc:1048] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.503643 13335 raft_consensus.cc:740] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:58.508592 13335 consensus_queue.cc:260] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.510396 13335 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.520815 13264 raft_consensus.cc:2468] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:58.522753 13335 raft_consensus.cc:2468] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:58.522871 13152 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:58.528311 13512 raft_consensus.cc:2804] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.528946 13405 raft_consensus.cc:993] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.529604 13405 raft_consensus.cc:1081] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.530113 13512 raft_consensus.cc:697] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.531312 13512 consensus_queue.cc:237] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.536638 13335 raft_consensus.cc:3060] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.542934 13084 catalog_manager.cc:5697] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.546428 13264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32818
I20260709 13:06:58.547395 13264 raft_consensus.cc:493] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.547848 13264 raft_consensus.cc:3060] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.548913 13478 consensus_queue.cc:1048] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.558624 13264 raft_consensus.cc:515] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.561491 13264 leader_election.cc:290] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.562554 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.563320 13188 raft_consensus.cc:3055] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.563536 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a407ca0fe144cc78e7b1e96f96ac585" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.564278 13337 raft_consensus.cc:3060] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.563551 13188 raft_consensus.cc:740] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.565810 13478 raft_consensus.cc:993] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: : Instructing follower ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:06:58.565491 13188 consensus_queue.cc:260] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.566352 13478 raft_consensus.cc:1081] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Signalling peer ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:06:58.567093 13188 raft_consensus.cc:3060] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.569089 13335 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:38554
I20260709 13:06:58.570228 13335 raft_consensus.cc:493] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.570935 13335 raft_consensus.cc:3060] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.574633 13337 raft_consensus.cc:2468] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.576176 13227 leader_election.cc:304] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:58.578118 13478 raft_consensus.cc:2804] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.579649 13478 raft_consensus.cc:697] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.580875 13478 consensus_queue.cc:237] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.580871 13335 raft_consensus.cc:515] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.584728 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.584813 13188 raft_consensus.cc:2468] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.585464 13187 raft_consensus.cc:3060] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.586143 13335 leader_election.cc:290] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:06:58.586562 13264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dfcbf3c5e504e68bee46c6d02010c38" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.587395 13264 raft_consensus.cc:3055] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.587908 13264 raft_consensus.cc:740] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.588994 13264 consensus_queue.cc:260] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.591125 13264 raft_consensus.cc:3060] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.595496 13084 catalog_manager.cc:5697] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.601419 13264 raft_consensus.cc:2468] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:06:58.602826 13187 raft_consensus.cc:2468] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:06:58.603020 13301 leader_election.cc:304] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:58.604140 13549 raft_consensus.cc:2804] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.604738 13549 raft_consensus.cc:697] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:58.605696 13549 consensus_queue.cc:237] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:06:58.613524 13210 debug-util.cc:398] Leaking SignalData structure 0x7b08001144c0 after lost signal to thread 434
W20260709 13:06:58.615197 13210 debug-util.cc:398] Leaking SignalData structure 0x7b0800334740 after lost signal to thread 13121
I20260709 13:06:58.615772 13084 catalog_manager.cc:5697] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 1 to 2, leader changed from eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:58.617007 13210 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf3a0 after lost signal to thread 13213
W20260709 13:06:58.618561 13210 debug-util.cc:398] Leaking SignalData structure 0x7b080020f460 after lost signal to thread 13288
W20260709 13:06:58.620059 13210 debug-util.cc:398] Leaking SignalData structure 0x7b080021fa80 after lost signal to thread 13362
I20260709 13:06:58.630908 13512 consensus_queue.cc:1048] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.636620 13512 consensus_queue.cc:1048] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.638728 13508 consensus_queue.cc:1048] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.667615 13507 consensus_queue.cc:1048] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.707578 13512 consensus_queue.cc:1048] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.715332 13507 consensus_queue.cc:1048] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.720736 13478 consensus_queue.cc:1048] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.741088 13512 raft_consensus.cc:993] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.742007 13512 raft_consensus.cc:1081] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.748931 13264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32818
I20260709 13:06:58.749715 13264 raft_consensus.cc:493] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.750159 13264 raft_consensus.cc:3060] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.763615 13264 raft_consensus.cc:515] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.764422 13549 raft_consensus.cc:993] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.765087 13549 raft_consensus.cc:1081] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.764518 13480 raft_consensus.cc:993] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: : Instructing follower 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:58.766891 13264 leader_election.cc:290] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.766955 13480 raft_consensus.cc:1081] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Signalling peer 6d05e27c97034a909efec0f5dd6b4b31 to start an election
I20260709 13:06:58.768332 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.768783 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427001fe3c8244c5a6519c100eaad393" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.769249 13188 raft_consensus.cc:3055] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.769583 13336 raft_consensus.cc:3060] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.769711 13188 raft_consensus.cc:740] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.770947 13188 consensus_queue.cc:260] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.772538 13188 raft_consensus.cc:3060] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.775962 13187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
 from {username='slave'} at 127.0.0.1:37602
I20260709 13:06:58.776768 13187 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.777177 13187 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.783735 13188 raft_consensus.cc:2468] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.792804 13526 consensus_queue.cc:1048] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.807000 13264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32834
I20260709 13:06:58.807842 13264 raft_consensus.cc:493] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.808409 13264 raft_consensus.cc:3060] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.819303 13264 raft_consensus.cc:515] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.821681 13336 raft_consensus.cc:2468] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.828931 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.832930 13337 raft_consensus.cc:3055] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.833333 13337 raft_consensus.cc:740] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:58.834359 13337 consensus_queue.cc:260] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.835673 13337 raft_consensus.cc:3060] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.837390 13480 raft_consensus.cc:493] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.838107 13480 raft_consensus.cc:515] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.839160 13510 consensus_queue.cc:1048] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.842051 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:06:58.829712 13264 leader_election.cc:290] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.847925 13337 raft_consensus.cc:2468] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.830441 13229 leader_election.cc:304] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:58.853996 13227 leader_election.cc:304] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:58.855589 13562 raft_consensus.cc:2804] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.856285 13562 raft_consensus.cc:697] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.856065 13561 raft_consensus.cc:2804] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.857367 13561 raft_consensus.cc:697] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.857656 13562 consensus_queue.cc:237] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.858318 13561 consensus_queue.cc:237] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.858872 13480 leader_election.cc:290] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.863869 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c61c027bd134db48f7fdf4cf845e009" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.864794 13188 raft_consensus.cc:3060] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.822887 13187 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.876041 13264 raft_consensus.cc:3060] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.876315 13184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31" is_pre_election: true
I20260709 13:06:58.878988 13229 leader_election.cc:304] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:06:58.879813 13084 catalog_manager.cc:5697] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.885190 13084 catalog_manager.cc:5697] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.899539 13549 consensus_queue.cc:1048] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.900483 13561 raft_consensus.cc:2749] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.901777 13187 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Requested vote from peers eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.902144 13262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:06:58.902886 13262 raft_consensus.cc:3055] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.903220 13262 raft_consensus.cc:740] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.904395 13262 consensus_queue.cc:260] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.905853 13262 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.907716 13188 raft_consensus.cc:2468] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.908836 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "6d05e27c97034a909efec0f5dd6b4b31" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.909631 13335 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.919942 13335 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:58.921571 13152 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:06:58.922827 13405 raft_consensus.cc:2804] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.923367 13405 raft_consensus.cc:697] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Leader. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:06:58.924204 13262 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d05e27c97034a909efec0f5dd6b4b31 in term 2.
I20260709 13:06:58.924288 13405 consensus_queue.cc:237] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.930497 13549 raft_consensus.cc:993] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: : Instructing follower eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.932936 13549 raft_consensus.cc:1081] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Signalling peer eecc19d37a4940b6997f35cd9ffb574f to start an election
I20260709 13:06:58.935371 13259 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
 from {username='slave'} at 127.0.0.1:32834
I20260709 13:06:58.936129 13259 raft_consensus.cc:493] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.936580 13259 raft_consensus.cc:3060] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.938617 13084 catalog_manager.cc:5697] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 reported cstate change: term changed from 1 to 2, leader changed from eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194) to 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "6d05e27c97034a909efec0f5dd6b4b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.944464 13259 raft_consensus.cc:515] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.949556 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:06:58.950353 13335 raft_consensus.cc:3055] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:58.950717 13335 raft_consensus.cc:740] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:06:58.951625 13335 consensus_queue.cc:260] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.952594 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64112aad1a0a4275b6edaa9ece28a9d4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:06:58.953136 13335 raft_consensus.cc:3060] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.953415 13188 raft_consensus.cc:3060] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.955394 13259 leader_election.cc:290] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:06:58.964387 13188 raft_consensus.cc:2468] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.964387 13335 raft_consensus.cc:2468] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:06:58.965991 13229 leader_election.cc:304] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f; no voters: 
I20260709 13:06:58.968240 13561 raft_consensus.cc:2804] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:58.968917 13561 raft_consensus.cc:697] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:06:58.969908 13561 consensus_queue.cc:237] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:06:58.979234 13086 catalog_manager.cc:5697] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 1 to 2, leader changed from ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.010571 13259 raft_consensus.cc:1275] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.012609 13405 consensus_queue.cc:1048] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.030009 13335 raft_consensus.cc:1275] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.033638 13405 consensus_queue.cc:1048] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.065042 13259 raft_consensus.cc:1275] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.066757 13549 consensus_queue.cc:1048] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.082914 13188 raft_consensus.cc:1275] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.085381 13549 consensus_queue.cc:1048] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.166581 13337 raft_consensus.cc:1275] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.168668 13562 consensus_queue.cc:1048] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.189921 13188 raft_consensus.cc:1275] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.191643 13480 consensus_queue.cc:1048] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.304160 13337 raft_consensus.cc:1275] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.306059 13480 consensus_queue.cc:1048] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.317188 13188 raft_consensus.cc:1275] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.319458 13562 consensus_queue.cc:1048] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.364030 13337 raft_consensus.cc:1275] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.365974 13480 consensus_queue.cc:1048] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.379566 13188 raft_consensus.cc:1275] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.381659 13478 consensus_queue.cc:1048] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.393868 13188 raft_consensus.cc:1275] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.399309 13480 consensus_queue.cc:1048] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.417394 13337 raft_consensus.cc:1275] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer eecc19d37a4940b6997f35cd9ffb574f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.417394 13335 raft_consensus.cc:1275] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.419606 13480 consensus_queue.cc:1048] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.419956 13512 consensus_queue.cc:1048] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.437949 13259 raft_consensus.cc:1275] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer 6d05e27c97034a909efec0f5dd6b4b31: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.441524 13508 consensus_queue.cc:1048] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:01.505239 13087 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60848:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:07:01.538064 13087 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60848:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:07:01.567193   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:01.828420 13261 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "383dda1d741941398461d227aee2b621"
dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
mode: GRACEFUL
new_leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:35026
I20260709 13:07:01.837407 13261 raft_consensus.cc:606] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Received request to transfer leadership to TS ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:07:01.920125 13612 tablet.cc:1724] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:01.927589 13606 log.cc:927] Time spent T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: Append to log took a long time: real 0.124s	user 0.005s	sys 0.000s
I20260709 13:07:01.929852 13571 raft_consensus.cc:993] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: : Instructing follower ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:07:01.934649 13571 raft_consensus.cc:1081] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Signalling peer ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:07:01.938479 13333 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "383dda1d741941398461d227aee2b621"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:38554
I20260709 13:07:01.944804 13333 raft_consensus.cc:493] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:01.949967 13333 raft_consensus.cc:3060] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:01.983202 13602 log.cc:927] Time spent T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: Append to log took a long time: real 0.180s	user 0.003s	sys 0.000s
W20260709 13:07:01.987298 13609 log.cc:927] Time spent T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 13:07:02.037489 13628 tablet.cc:1724] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.041857 13632 tablet.cc:1724] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.045425 13629 tablet.cc:1724] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.056074 13333 raft_consensus.cc:515] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:07:02.063434 13152 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:46015, user_credentials={real_user=slave}} blocked reactor thread for 135607us
I20260709 13:07:02.088406 13634 tablet.cc:1724] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.105722 13636 tablet.cc:1724] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.051316 13633 tablet.cc:1724] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.118526 13188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:07:02.119561 13188 raft_consensus.cc:3060] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.118432 13630 tablet.cc:1724] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.122910 13627 tablet.cc:1724] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.134040 13333 leader_election.cc:290] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:02.138710 13263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "383dda1d741941398461d227aee2b621" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:07:02.139508 13263 raft_consensus.cc:3055] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:02.139878 13263 raft_consensus.cc:740] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:07:02.140811 13263 consensus_queue.cc:260] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:02.142387 13263 raft_consensus.cc:3060] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.145664 13188 raft_consensus.cc:2468] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:07:02.153256 13303 leader_election.cc:304] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:07:02.161461 13549 raft_consensus.cc:2804] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.162145 13549 raft_consensus.cc:697] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:07:02.161994 13628 tablet.cc:1724] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.163499 13549 consensus_queue.cc:237] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:07:02.182997 13631 log.cc:927] Time spent T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:07:02.189715 13263 raft_consensus.cc:2468] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:07:02.208011 13635 tablet.cc:1724] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.208158 13637 tablet.cc:1724] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.210968 13630 tablet.cc:1724] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.220301 13087 catalog_manager.cc:5789] TS 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139) does not have the latest schema for tablet d4b2212ec052479c8be0c5e7fada3c47 (table soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]). Expected version 1 got 0
I20260709 13:07:02.222110 13087 catalog_manager.cc:5789] TS 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139) does not have the latest schema for tablet 127bb154d697451db026a06b2507cc7a (table soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]). Expected version 1 got 0
I20260709 13:07:02.225464 13087 catalog_manager.cc:5789] TS 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139) does not have the latest schema for tablet a016a7f78fc34769933e2e7eb10eb89e (table soft_deleted_table [id=559a18b24f814112b011d94978eab2ef]). Expected version 1 got 0
I20260709 13:07:02.261286 13640 tablet.cc:1724] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.287179 13629 tablet.cc:1724] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.292516 13187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b"
dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
mode: GRACEFUL
new_leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:53750
I20260709 13:07:02.291994 13633 tablet.cc:1724] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.292320 13087 catalog_manager.cc:5697] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 1 to 2, leader changed from eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.293341 13187 raft_consensus.cc:606] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Received request to transfer leadership to TS ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:07:02.295476   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:02.296250   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:02.237864 13612 tablet.cc:1724] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.308471 13634 tablet.cc:1724] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.361476 13636 tablet.cc:1724] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.364535 13640 tablet.cc:1724] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.321401 13628 tablet.cc:1724] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.370592 13630 tablet.cc:1724] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.390022 13632 tablet.cc:1724] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.394702 13612 tablet.cc:1724] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.426549 13635 tablet.cc:1724] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.406978 13637 tablet.cc:1724] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.436115 13633 tablet.cc:1724] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:02.496884 13086 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f: soft_deleted_table [id=559a18b24f814112b011d94978eab2ef] alter complete (version 1)
I20260709 13:07:02.611457 13261 raft_consensus.cc:1275] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.614054 13549 consensus_queue.cc:1048] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:02.630483 13187 raft_consensus.cc:1275] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.636209 13549 consensus_queue.cc:1048] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:02.663336 13591 raft_consensus.cc:993] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: : Instructing follower ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:07:02.664070 13626 raft_consensus.cc:1081] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Signalling peer ff2c85391bff48d5bafcf26c004ad30e to start an election
I20260709 13:07:02.667852 13337 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b"
dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
 from {username='slave'} at 127.0.0.1:38552
I20260709 13:07:02.668460 13337 raft_consensus.cc:493] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:02.668919 13337 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:02.676491 13337 raft_consensus.cc:515] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:02.679510 13337 leader_election.cc:290] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:02.681345 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:07:02.682217 13261 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:02.683131 13187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940faeaa383542e1ac59b86fa4167a8b" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6d05e27c97034a909efec0f5dd6b4b31"
I20260709 13:07:02.685943 13187 raft_consensus.cc:3055] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:02.686350 13187 raft_consensus.cc:740] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6d05e27c97034a909efec0f5dd6b4b31, State: Running, Role: LEADER
I20260709 13:07:02.687326 13187 consensus_queue.cc:260] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:02.688906 13187 raft_consensus.cc:3060] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:02.698628 13261 raft_consensus.cc:2468] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 3.
I20260709 13:07:02.699710 13187 raft_consensus.cc:2468] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 3.
I20260709 13:07:02.700879 13301 leader_election.cc:304] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:07:02.702121 13649 raft_consensus.cc:2804] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:02.702569 13649 raft_consensus.cc:697] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 3 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:07:02.703629 13649 consensus_queue.cc:237] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:02.713555 13087 catalog_manager.cc:5697] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 2 to 3, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:03.116653 13261 raft_consensus.cc:1275] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:03.118727 13649 consensus_queue.cc:1048] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:03.139550 13187 raft_consensus.cc:1275] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 3 FOLLOWER]: Refusing update from remote peer ff2c85391bff48d5bafcf26c004ad30e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:03.141536 13648 consensus_queue.cc:1048] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:03.389056   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:07:03.397037 13229 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20260709 13:07:03.404551 13229 consensus_peers.cc:597] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.429553 13229 consensus_peers.cc:597] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.460407   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:03.461491   431 tablet_replica.cc:333] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.462332   431 raft_consensus.cc:2243] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.463351   431 raft_consensus.cc:2272] T d4b2212ec052479c8be0c5e7fada3c47 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.465907   431 tablet_replica.cc:333] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.466692   431 raft_consensus.cc:2243] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.467288   431 raft_consensus.cc:2272] T 9dfcbf3c5e504e68bee46c6d02010c38 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.469877   431 tablet_replica.cc:333] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.470634   431 raft_consensus.cc:2243] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.471215   431 raft_consensus.cc:2272] T 427001fe3c8244c5a6519c100eaad393 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.477931   431 tablet_replica.cc:333] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.479185   431 raft_consensus.cc:2243] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.479825   431 raft_consensus.cc:2272] T 9a407ca0fe144cc78e7b1e96f96ac585 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.482414   431 tablet_replica.cc:333] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.483143   431 raft_consensus.cc:2243] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.483767   431 raft_consensus.cc:2272] T 4c61c027bd134db48f7fdf4cf845e009 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.486402   431 tablet_replica.cc:333] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.487192   431 raft_consensus.cc:2243] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.487726   431 raft_consensus.cc:2272] T 940faeaa383542e1ac59b86fa4167a8b P 6d05e27c97034a909efec0f5dd6b4b31 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.491708   431 tablet_replica.cc:333] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.492569   431 raft_consensus.cc:2243] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.493158   431 raft_consensus.cc:2272] T 64112aad1a0a4275b6edaa9ece28a9d4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.497159   431 tablet_replica.cc:333] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.498286   431 raft_consensus.cc:2243] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.498872   431 raft_consensus.cc:2272] T 1b6fbf23640a4e858af7f28bddd6ad2e P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.501579   431 tablet_replica.cc:333] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.502380   431 raft_consensus.cc:2243] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.503345   431 raft_consensus.cc:2272] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.505980   431 tablet_replica.cc:333] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.506831   431 raft_consensus.cc:2243] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.508111   431 raft_consensus.cc:2272] T 34e536fb23464ff3b1396e46945ac5ea P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.512012   431 tablet_replica.cc:333] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.512837   431 raft_consensus.cc:2243] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.514091   431 raft_consensus.cc:2272] T a016a7f78fc34769933e2e7eb10eb89e P 6d05e27c97034a909efec0f5dd6b4b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.516909   431 tablet_replica.cc:333] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.517591   431 raft_consensus.cc:2243] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.518134   431 raft_consensus.cc:2272] T 383dda1d741941398461d227aee2b621 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.521309   431 tablet_replica.cc:333] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.522006   431 raft_consensus.cc:2243] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.523006   431 raft_consensus.cc:2272] T 127bb154d697451db026a06b2507cc7a P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.525696   431 tablet_replica.cc:333] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.526532   431 raft_consensus.cc:2243] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.527814   431 raft_consensus.cc:2272] T 68a72ac98d9f4748bc8466b080e758e4 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:03.530655 13229 consensus_peers.cc:597] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.531621   431 tablet_replica.cc:333] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.532850   431 raft_consensus.cc:2243] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.533771   431 raft_consensus.cc:2272] T caef7e2d7c474ee1a9b8f91327426099 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.537223   431 tablet_replica.cc:333] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.538465   431 raft_consensus.cc:2243] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.539780   431 raft_consensus.cc:2272] T ba35c9ea187e482cbeb39f6b219c7030 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.542796   431 tablet_replica.cc:333] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.543618   431 raft_consensus.cc:2243] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:03.544720   431 raft_consensus.cc:2272] T 1cd9455d9af4433d9e2f66e3018c86e8 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:03.547454   431 tablet_replica.cc:333] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31: stopping tablet replica
I20260709 13:07:03.548110   431 raft_consensus.cc:2243] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:03.548709   431 raft_consensus.cc:2272] T 30321d144d6140b0a45e60b81c9379d3 P 6d05e27c97034a909efec0f5dd6b4b31 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:03.568928 13229 consensus_peers.cc:597] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.575873 13303 consensus_peers.cc:597] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.609916 13229 consensus_peers.cc:597] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.627697 13303 consensus_peers.cc:597] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.670974 13303 consensus_peers.cc:597] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.710892 13229 consensus_peers.cc:597] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.780376 13229 consensus_peers.cc:597] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.786706 13303 consensus_peers.cc:597] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:04.456223 13657 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.456943 13657 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.459213 13657 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.460600 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
W20260709 13:07:04.464031 13229 leader_election.cc:336] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.465080 13229 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:07:04.465826 13593 raft_consensus.cc:2749] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:04.496940 13549 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.497859 13549 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.500711 13549 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.502235 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
W20260709 13:07:04.505292 13303 leader_election.cc:336] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.505725 13303 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:07:04.506436 13549 raft_consensus.cc:2749] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:04.597436 13549 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.597957 13549 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.600988 13549 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
W20260709 13:07:04.602747   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13210)
I20260709 13:07:04.602427 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
W20260709 13:07:04.605985 13303 leader_election.cc:336] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.606540 13303 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:07:04.607609 13549 raft_consensus.cc:2749] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:04.608484 13593 raft_consensus.cc:493] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.609229 13593 raft_consensus.cc:515] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.613901 13593 leader_election.cc:290] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.618208 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
W20260709 13:07:04.618533 13229 leader_election.cc:336] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.620565 13549 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.621140 13227 leader_election.cc:304] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:07:04.621382 13549 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.622932 13657 raft_consensus.cc:2749] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:04.625219 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
W20260709 13:07:04.630205 13303 leader_election.cc:336] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.631251 13303 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:07:04.632483 13649 raft_consensus.cc:2749] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:04.627612 13549 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.650003 13549 raft_consensus.cc:493] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.650837 13549 raft_consensus.cc:515] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.656512 13549 leader_election.cc:290] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.657920 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:07:04.658872 13261 raft_consensus.cc:2468] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
W20260709 13:07:04.659338 13303 leader_election.cc:336] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.660434 13301 leader_election.cc:304] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.661284 13549 raft_consensus.cc:2804] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:04.662070 13549 raft_consensus.cc:493] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.662564 13549 raft_consensus.cc:3060] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.671753 13549 raft_consensus.cc:515] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.674119 13549 leader_election.cc:290] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.675666 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ea922ffb9f4f3497be21b8e2ffc4e7" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:07:04.676726 13261 raft_consensus.cc:3060] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.678169 13549 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
W20260709 13:07:04.678977 13303 leader_election.cc:336] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.678936 13549 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:07:04.682687 13549 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.682992 13262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
W20260709 13:07:04.687301 13303 leader_election.cc:336] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.687814 13303 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31, eecc19d37a4940b6997f35cd9ffb574f
I20260709 13:07:04.687928 13261 raft_consensus.cc:2468] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:07:04.688771 13549 raft_consensus.cc:2749] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:04.689919 13301 leader_election.cc:304] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.691128 13549 raft_consensus.cc:2804] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:04.691875 13549 raft_consensus.cc:697] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:07:04.692801 13549 consensus_queue.cc:237] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.703658 13084 catalog_manager.cc:5697] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 1 to 2, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:04.766088 13593 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.767146 13593 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.771296 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:07:04.772128 13337 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:07:04.773836 13227 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:07:04.774226 13593 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.774775 13657 raft_consensus.cc:2804] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.775259 13657 raft_consensus.cc:493] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.775640 13657 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:04.776589 13229 leader_election.cc:336] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.780298 13593 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.781168 13593 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.784832 13549 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.785655 13549 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.788753 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:07:04.789726 13337 raft_consensus.cc:2468] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:07:04.790256 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:07:04.791129 13261 raft_consensus.cc:2468] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 1.
W20260709 13:07:04.791244 13229 leader_election.cc:336] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.792923 13301 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:07:04.785687 13657 raft_consensus.cc:515] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.794163 13649 raft_consensus.cc:2804] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:04.794633 13649 raft_consensus.cc:493] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.795081 13649 raft_consensus.cc:3060] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.795019 13227 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.800715 13549 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.802413 13593 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.804297 13593 raft_consensus.cc:2804] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.804795 13593 raft_consensus.cc:493] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.805200 13593 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:04.805748 13229 leader_election.cc:336] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.812716 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4b2212ec052479c8be0c5e7fada3c47" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:07:04.816524 13337 raft_consensus.cc:3060] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.823424 13593 raft_consensus.cc:515] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.826176 13337 raft_consensus.cc:2468] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 3.
W20260709 13:07:04.815339 13303 leader_election.cc:336] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.828151 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bb154d697451db026a06b2507cc7a" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:07:04.829010 13335 raft_consensus.cc:3060] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.827248 13593 leader_election.cc:290] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.809948 13649 raft_consensus.cc:515] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.829649 13227 leader_election.cc:304] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.811034 13657 leader_election.cc:290] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.831482 13657 raft_consensus.cc:2804] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:04.832077 13657 raft_consensus.cc:697] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:07:04.833117 13657 consensus_queue.cc:237] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.838407 13649 leader_election.cc:290] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.838853 13335 raft_consensus.cc:2468] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 3.
I20260709 13:07:04.840600 13227 leader_election.cc:304] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:07:04.841662 13668 raft_consensus.cc:2804] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:04.840611 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a016a7f78fc34769933e2e7eb10eb89e" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
W20260709 13:07:04.841454 13229 leader_election.cc:336] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.842288 13668 raft_consensus.cc:697] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:07:04.842345 13261 raft_consensus.cc:3060] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.847052 13668 consensus_queue.cc:237] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
W20260709 13:07:04.855720 13303 leader_election.cc:336] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.865553 13084 catalog_manager.cc:5697] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 2 to 3, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.865418 13261 raft_consensus.cc:2468] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:07:04.873613 13301 leader_election.cc:304] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.874754 13649 raft_consensus.cc:2804] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:04.875432 13649 raft_consensus.cc:697] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:07:04.876502 13649 consensus_queue.cc:237] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.881201 13668 raft_consensus.cc:493] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.881980 13668 raft_consensus.cc:515] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.885907 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:07:04.888396 13668 leader_election.cc:290] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
W20260709 13:07:04.888628 13229 leader_election.cc:336] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.889324 13229 leader_election.cc:304] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f; no voters: 6d05e27c97034a909efec0f5dd6b4b31, ff2c85391bff48d5bafcf26c004ad30e
I20260709 13:07:04.890177 13668 raft_consensus.cc:2749] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:04.891266 13084 catalog_manager.cc:5697] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 2 to 3, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.900238 13087 catalog_manager.cc:5697] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 1 to 2, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:04.919147 13668 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.919852 13668 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.922286 13668 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.923854 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:07:04.924757 13335 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:07:04.926069 13227 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
W20260709 13:07:04.926748 13229 leader_election.cc:336] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.926952 13668 raft_consensus.cc:2804] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.927496 13668 raft_consensus.cc:493] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.927838 13668 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.935876 13668 raft_consensus.cc:515] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.939021 13668 leader_election.cc:290] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.941013 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a72ac98d9f4748bc8466b080e758e4" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:07:04.941910 13335 raft_consensus.cc:3060] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:04.943740 13229 leader_election.cc:336] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.945926   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:07:04.949530 13335 raft_consensus.cc:2468] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 3.
I20260709 13:07:04.951040 13668 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.951363 13227 leader_election.cc:304] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.952113 13668 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:07:04.952572 13657 raft_consensus.cc:2804] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:04.953254 13657 raft_consensus.cc:697] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:07:04.954861 13668 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.954262 13657 consensus_queue.cc:237] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.957404 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e" is_pre_election: true
I20260709 13:07:04.958488 13335 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 2.
I20260709 13:07:04.961716 13227 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 
I20260709 13:07:04.963047 13668 raft_consensus.cc:2804] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.964198 13668 raft_consensus.cc:493] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.964804 13668 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:04.967835 13229 leader_election.cc:336] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.976068 13649 raft_consensus.cc:493] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.975139 13668 raft_consensus.cc:515] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:07:04.976917 13649 raft_consensus.cc:515] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:04.978869 13668 leader_election.cc:290] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195:36547)
I20260709 13:07:04.980276 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd9455d9af4433d9e2f66e3018c86e8" candidate_uuid: "eecc19d37a4940b6997f35cd9ffb574f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ff2c85391bff48d5bafcf26c004ad30e"
I20260709 13:07:04.984337 13335 raft_consensus.cc:3060] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:04.985656 13229 leader_election.cc:336] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.980067 13087 catalog_manager.cc:5697] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 2 to 3, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:04.987195 13303 leader_election.cc:336] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:04.982244 13649 leader_election.cc:290] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:04.983139 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f" is_pre_election: true
I20260709 13:07:04.988828 13261 raft_consensus.cc:2468] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 2.
I20260709 13:07:04.990497 13301 leader_election.cc:304] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:04.991959 13649 raft_consensus.cc:2804] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.992419 13649 raft_consensus.cc:493] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6d05e27c97034a909efec0f5dd6b4b31)
I20260709 13:07:04.992872 13649 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.998956 13335 raft_consensus.cc:2468] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecc19d37a4940b6997f35cd9ffb574f in term 3.
I20260709 13:07:05.001333 13227 leader_election.cc:304] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:05.003489 13657 raft_consensus.cc:2804] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:05.004155 13657 raft_consensus.cc:697] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Becoming Leader. State: Replica: eecc19d37a4940b6997f35cd9ffb574f, State: Running, Role: LEADER
I20260709 13:07:05.004290 13649 raft_consensus.cc:515] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:05.005311 13657 consensus_queue.cc:237] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } }
I20260709 13:07:05.008278 13649 leader_election.cc:290] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 election: Requested vote from peers 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139), eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015)
I20260709 13:07:05.008646 13261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e536fb23464ff3b1396e46945ac5ea" candidate_uuid: "ff2c85391bff48d5bafcf26c004ad30e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "eecc19d37a4940b6997f35cd9ffb574f"
I20260709 13:07:05.009552 13261 raft_consensus.cc:3060] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:05.015872   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:07:05.022082 13261 raft_consensus.cc:2468] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff2c85391bff48d5bafcf26c004ad30e in term 3.
W20260709 13:07:05.023546 13303 leader_election.cc:336] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111)
I20260709 13:07:05.025390 13301 leader_election.cc:304] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecc19d37a4940b6997f35cd9ffb574f, ff2c85391bff48d5bafcf26c004ad30e; no voters: 6d05e27c97034a909efec0f5dd6b4b31
I20260709 13:07:05.026450 13649 raft_consensus.cc:2804] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:05.027149 13649 raft_consensus.cc:697] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 3 LEADER]: Becoming Leader. State: Replica: ff2c85391bff48d5bafcf26c004ad30e, State: Running, Role: LEADER
I20260709 13:07:05.031821 13649 consensus_queue.cc:237] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } }
I20260709 13:07:05.031240 13087 catalog_manager.cc:5697] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f reported cstate change: term changed from 2 to 3, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "eecc19d37a4940b6997f35cd9ffb574f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:05.046684 13084 catalog_manager.cc:5697] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e reported cstate change: term changed from 2 to 3, leader changed from 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193) to ff2c85391bff48d5bafcf26c004ad30e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "ff2c85391bff48d5bafcf26c004ad30e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d05e27c97034a909efec0f5dd6b4b31" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecc19d37a4940b6997f35cd9ffb574f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff2c85391bff48d5bafcf26c004ad30e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:05.086102   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:05.087181   431 tablet_replica.cc:333] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.088037   431 raft_consensus.cc:2243] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.089205   431 raft_consensus.cc:2272] T 9a407ca0fe144cc78e7b1e96f96ac585 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.092269   431 tablet_replica.cc:333] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.093334   431 raft_consensus.cc:2243] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.095118   431 raft_consensus.cc:2272] T 427001fe3c8244c5a6519c100eaad393 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.098591   431 tablet_replica.cc:333] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.099776   431 raft_consensus.cc:2243] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.100505   431 raft_consensus.cc:2272] T 9dfcbf3c5e504e68bee46c6d02010c38 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.103487   431 tablet_replica.cc:333] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.104413   431 raft_consensus.cc:2243] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.105259   431 pending_rounds.cc:70] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Trying to abort 1 pending ops.
I20260709 13:07:05.105525   431 pending_rounds.cc:77] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f: Aborting op as it isn't in flight: id { term: 3 index: 4 } timestamp: 7305635532127199232 op_type: NO_OP noop_request { }
I20260709 13:07:05.105979   431 raft_consensus.cc:2889] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.106362   431 raft_consensus.cc:2272] T d4b2212ec052479c8be0c5e7fada3c47 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:05.107532 13301 consensus_peers.cc:597] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e -> Peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Couldn't send request to peer eecc19d37a4940b6997f35cd9ffb574f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:05.108527 13301 consensus_peers.cc:597] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e -> Peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Couldn't send request to peer eecc19d37a4940b6997f35cd9ffb574f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:05.109414   431 tablet_replica.cc:333] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.110260   431 raft_consensus.cc:2243] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.111506   431 raft_consensus.cc:2272] T 4c61c027bd134db48f7fdf4cf845e009 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.114867   431 tablet_replica.cc:333] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.115686   431 raft_consensus.cc:2243] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.116271   431 raft_consensus.cc:2272] T 34e536fb23464ff3b1396e46945ac5ea P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.119482   431 tablet_replica.cc:333] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.120220   431 raft_consensus.cc:2243] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.120792   431 raft_consensus.cc:2272] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.123401   431 tablet_replica.cc:333] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.124123   431 raft_consensus.cc:2243] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.124732   431 raft_consensus.cc:2272] T 940faeaa383542e1ac59b86fa4167a8b P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.127125   431 tablet_replica.cc:333] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.127750   431 raft_consensus.cc:2243] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.128206   431 raft_consensus.cc:2272] T caef7e2d7c474ee1a9b8f91327426099 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.131232   431 tablet_replica.cc:333] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.132125   431 raft_consensus.cc:2243] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.133380   431 pending_rounds.cc:70] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Trying to abort 1 pending ops.
I20260709 13:07:05.133574   431 pending_rounds.cc:77] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f: Aborting op as it isn't in flight: id { term: 3 index: 4 } timestamp: 7305635532216139776 op_type: NO_OP noop_request { }
I20260709 13:07:05.133939   431 raft_consensus.cc:2889] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.134289   431 raft_consensus.cc:2272] T 127bb154d697451db026a06b2507cc7a P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.137411   431 tablet_replica.cc:333] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.138182   431 raft_consensus.cc:2243] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.139050   431 pending_rounds.cc:70] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: Trying to abort 1 pending ops.
I20260709 13:07:05.139231   431 pending_rounds.cc:77] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635532635021312 op_type: NO_OP noop_request { }
I20260709 13:07:05.139599   431 raft_consensus.cc:2889] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.139935   431 raft_consensus.cc:2272] T 68a72ac98d9f4748bc8466b080e758e4 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.142884   431 tablet_replica.cc:333] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.143743   431 raft_consensus.cc:2243] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.144951   431 raft_consensus.cc:2272] T 64112aad1a0a4275b6edaa9ece28a9d4 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.147348   431 tablet_replica.cc:333] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.148043   431 raft_consensus.cc:2243] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.149047   431 raft_consensus.cc:2272] T 1b6fbf23640a4e858af7f28bddd6ad2e P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.151467   431 tablet_replica.cc:333] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.152128   431 raft_consensus.cc:2243] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.153105   431 raft_consensus.cc:2272] T 30321d144d6140b0a45e60b81c9379d3 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.155674   431 tablet_replica.cc:333] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.156287   431 raft_consensus.cc:2243] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.157260   431 raft_consensus.cc:2272] T ba35c9ea187e482cbeb39f6b219c7030 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.159632   431 tablet_replica.cc:333] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.160233   431 raft_consensus.cc:2243] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.161084   431 pending_rounds.cc:70] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: Trying to abort 1 pending ops.
I20260709 13:07:05.161255   431 pending_rounds.cc:77] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635532832890880 op_type: NO_OP noop_request { }
I20260709 13:07:05.161586   431 raft_consensus.cc:2889] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.161883   431 raft_consensus.cc:2272] T 1cd9455d9af4433d9e2f66e3018c86e8 P eecc19d37a4940b6997f35cd9ffb574f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.165387   431 tablet_replica.cc:333] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
I20260709 13:07:05.166215   431 raft_consensus.cc:2243] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.166827   431 raft_consensus.cc:2272] T a016a7f78fc34769933e2e7eb10eb89e P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.169680   431 tablet_replica.cc:333] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f: stopping tablet replica
W20260709 13:07:05.169667 13303 consensus_peers.cc:597] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e -> Peer 6d05e27c97034a909efec0f5dd6b4b31 (127.0.107.193:42139): Couldn't send request to peer 6d05e27c97034a909efec0f5dd6b4b31. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:05.170811   431 raft_consensus.cc:2243] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.171380   431 raft_consensus.cc:2272] T 383dda1d741941398461d227aee2b621 P eecc19d37a4940b6997f35cd9ffb574f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.220882   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:07:05.226564 13301 consensus_peers.cc:597] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e -> Peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Couldn't send request to peer eecc19d37a4940b6997f35cd9ffb574f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:05.264744   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:07:05.274783 13301 consensus_peers.cc:597] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e -> Peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Couldn't send request to peer eecc19d37a4940b6997f35cd9ffb574f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:05.282486 13301 consensus_peers.cc:597] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e -> Peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Couldn't send request to peer eecc19d37a4940b6997f35cd9ffb574f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:05.319082   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:05.320097   431 tablet_replica.cc:333] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.320873   431 raft_consensus.cc:2243] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.322382   431 raft_consensus.cc:2272] T 9dfcbf3c5e504e68bee46c6d02010c38 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.325414   431 tablet_replica.cc:333] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.326373   431 raft_consensus.cc:2243] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.326957   431 raft_consensus.cc:2272] T 427001fe3c8244c5a6519c100eaad393 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:05.328286 13301 consensus_peers.cc:597] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e -> Peer eecc19d37a4940b6997f35cd9ffb574f (127.0.107.194:46015): Couldn't send request to peer eecc19d37a4940b6997f35cd9ffb574f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:46015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:05.329958   431 tablet_replica.cc:333] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.330868   431 raft_consensus.cc:2243] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.331449   431 raft_consensus.cc:2272] T 4c61c027bd134db48f7fdf4cf845e009 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.333959   431 tablet_replica.cc:333] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.334704   431 raft_consensus.cc:2243] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.335259   431 raft_consensus.cc:2272] T 9a407ca0fe144cc78e7b1e96f96ac585 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.337886   431 tablet_replica.cc:333] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.338625   431 raft_consensus.cc:2243] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.339879   431 raft_consensus.cc:2272] T 940faeaa383542e1ac59b86fa4167a8b P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.342675   431 tablet_replica.cc:333] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.343420   431 raft_consensus.cc:2243] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.343981   431 raft_consensus.cc:2272] T 64112aad1a0a4275b6edaa9ece28a9d4 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.346331   431 tablet_replica.cc:333] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.347000   431 raft_consensus.cc:2243] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.347524   431 raft_consensus.cc:2272] T 1b6fbf23640a4e858af7f28bddd6ad2e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.350100   431 tablet_replica.cc:333] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.351094   431 raft_consensus.cc:2243] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.352694   431 pending_rounds.cc:70] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Trying to abort 1 pending ops.
I20260709 13:07:05.352916   431 pending_rounds.cc:77] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e: Aborting op as it isn't in flight: id { term: 2 index: 3 } timestamp: 7305635532307599360 op_type: NO_OP noop_request { }
I20260709 13:07:05.353294   431 raft_consensus.cc:2889] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.353821   431 raft_consensus.cc:2272] T a016a7f78fc34769933e2e7eb10eb89e P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.359045   431 tablet_replica.cc:333] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.359901   431 raft_consensus.cc:2243] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.360998   431 raft_consensus.cc:2272] T 383dda1d741941398461d227aee2b621 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.363816   431 tablet_replica.cc:333] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.364602   431 raft_consensus.cc:2243] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.365756   431 raft_consensus.cc:2272] T caef7e2d7c474ee1a9b8f91327426099 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.368386   431 tablet_replica.cc:333] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.369076   431 raft_consensus.cc:2243] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.369658   431 raft_consensus.cc:2272] T 127bb154d697451db026a06b2507cc7a P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.372793   431 tablet_replica.cc:333] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.373848   431 raft_consensus.cc:2243] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.374775   431 raft_consensus.cc:2272] T 68a72ac98d9f4748bc8466b080e758e4 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.377928   431 tablet_replica.cc:333] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.378885   431 raft_consensus.cc:2243] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.379881   431 pending_rounds.cc:70] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Trying to abort 1 pending ops.
I20260709 13:07:05.380072   431 pending_rounds.cc:77] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e: Aborting op as it isn't in flight: id { term: 2 index: 3 } timestamp: 7305635531551412224 op_type: NO_OP noop_request { }
I20260709 13:07:05.380625   431 raft_consensus.cc:2889] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.381029   431 raft_consensus.cc:2272] T d1ea922ffb9f4f3497be21b8e2ffc4e7 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.384560   431 tablet_replica.cc:333] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.385268   431 raft_consensus.cc:2243] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.386123   431 pending_rounds.cc:70] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: Trying to abort 1 pending ops.
I20260709 13:07:05.386301   431 pending_rounds.cc:77] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635532946276352 op_type: NO_OP noop_request { }
I20260709 13:07:05.386624   431 raft_consensus.cc:2889] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:05.386932   431 raft_consensus.cc:2272] T 34e536fb23464ff3b1396e46945ac5ea P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.390194   431 tablet_replica.cc:333] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.391120   431 raft_consensus.cc:2243] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.391834   431 raft_consensus.cc:2272] T d4b2212ec052479c8be0c5e7fada3c47 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.394560   431 tablet_replica.cc:333] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.395252   431 raft_consensus.cc:2243] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.395743   431 raft_consensus.cc:2272] T 30321d144d6140b0a45e60b81c9379d3 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.398677   431 tablet_replica.cc:333] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.399510   431 raft_consensus.cc:2243] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.400187   431 raft_consensus.cc:2272] T ba35c9ea187e482cbeb39f6b219c7030 P ff2c85391bff48d5bafcf26c004ad30e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.402868   431 tablet_replica.cc:333] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e: stopping tablet replica
I20260709 13:07:05.403558   431 raft_consensus.cc:2243] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:05.404099   431 raft_consensus.cc:2272] T 1cd9455d9af4433d9e2f66e3018c86e8 P ff2c85391bff48d5bafcf26c004ad30e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.453899   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:07:05.495860   431 master.cc:562] Master@127.0.107.254:36441 shutting down...
I20260709 13:07:05.540768   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:05.541553   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:05.541926   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7b2b5f93978b44e7a360e4285d523c0f: stopping tablet replica
I20260709 13:07:05.565359   431 master.cc:584] Master@127.0.107.254:36441 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (14511 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (28735 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (422844 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:07:05.629840   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 518 messages since previous log ~17 seconds ago
I20260709 13:07:05.630159   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 523 messages since previous log ~17 seconds ago
I20260709 13:07:05.630354   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 520 messages since previous log ~17 seconds ago
I20260709 13:07:05.630573   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:07:05.630791   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:07:05.631045   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 10 messages since previous log ~208 seconds ago
I20260709 13:07:05.631359   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 68 messages since previous log ~2 seconds ago
I20260709 13:07:05.631878   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~13 seconds ago
